Soil is a file-based, transactional object database for Pharo — it stores object graphs directly, with no mapping layer between your objects and the disk. This series builds it from the ground up: it starts with turning a single object into bytes and works outward through object graphs, identity, transactions, MVCC, indexes and crash recovery. It assumes you know databases in general but nothing about Soil's internals, and it trades grand theory for how one concrete system actually works.
Soil: turning an object into bytes
Soil: turning an object into bytes A few years ago I wrote two posts about building an object database from scratch, BYOB - build your own (data)base and BYOB - object serialization, and then went quiet and just built the thing. Soil is real now: a file-based, transactional object database for Pharo, running in production. I want to go through its architecture properly this time, from the beginnin
22 July 2026
Soil: serializing a graph
Soil: serializing a graph In the last post we turned a single object into a self-describing stream of bytes and read it back. That was the easy half. Real objects don't live alone. They point at other objects, those point at more, and before long you have a graph. Sometimes that graph shares a node from two places, and sometimes it points back at itself in a cycle. This post is about writing such
23 July 2026
Soil: identity and the heap
Soil: identity and the heap The last post ended with two loose ends. A reference from one cluster to another is stored as an "id", but I never said what an id is. And I kept talking about clusters "on disk" without saying where on disk, or how you get from an id to the actual bytes. Both ends meet in the same place, so let's tie them off together. Why a reference can't just be a position The tempt
27 July 2026
Soil: transactions and how a commit happens
Soil: transactions and how a commit happens The last post gave every object a stable id and a way to find its current bytes through the index. That's enough to read and write single objects. What's still missing is the thing that makes a database out of that: a way to change several objects at once and have it count as one atomic step. That's a transaction, and this post is about what one actually
30 July 2026
Soil: MVCC and snapshot isolation
Soil: MVCC and snapshot isolation The last post walked through what a commit actually does: serialize outside any lock, then take one global lock just long enough to bump the database version, check for conflicts, write the journal, and apply it. Only one writer gets to be in that critical section at a time - the same trade-off SQLite makes. Readers, though, were said to bypass that lock entirely
11 August 2026