Soil v6 released Soil v6 is out. As with v5, that means a new branch v6 you can load, not a tag. No migration is needed if you are coming from v5: the on-disk format is unchanged, and journal fragments written by v5 are still read. This release is the first one that came with a blog series running alongside it. Several of the bigger topics have a post of their own, so this one only says what is in
Soil: Resident Eval - making persistence undead Over the years I have said and written a bit about databases: why we need them at all, how they work inside, and why objects fit so badly into most of them. An object has identity, behavior and references to other objects. Relational databases are the best-known example of a different model: they know tables with rows, columns and foreign keys. Every
Soil: indexes under MVCC The last post ended with a file of pages that knew nothing about transactions: as described there, a page is just a page, shared by whoever touches the file. That cannot be the whole story. The MVCC post promised every transaction a world frozen at its start, and the heap keeps that promise by never overwriting anything: old versions stay where they are and a chain leads b
Soil: indexes on disk, B-tree and skip list The identity post ended with an object that can be found by its id: two bytes of segment, six bytes of index, one lookup in the object index file, and the position in the heap is known. That answers "where is object 42". It does not answer the question every application asks first: "which object belongs to this name". Something has to map a key to an obj
Canopy - tags, labels, and reaching outside the image Canopy's mechanism, from the first part of this pair, in short: a singleton that is itself a tree, domains hanging off it by name, and at every leaf either a raw CanopyCell or a CanopyAccessor binding that position to a live object through two independently optional selectors - one for reading, one for writing. Config and metrics turn out to be
Canopy - I have global values Every config value starts the same way: you just need it, right now, and the fastest path is to write it straight into the method that uses it. A credential, a limit, a base URL - hardcoded, because reaching for a whole configuration mechanism for one value feels like overkill, and you don't want to be slowed down by ceremony you don't need yet. That's fine, right up
Soil: checkpoints and recovering from a crash The transactions post left this dangling: "if any of this actually changed the database, Soil checkpoints afterward - flushing everything to disk in a way that can survive a crash. What a checkpoint really guarantees is its own post." The journal post added its own loose end on top: every fragment file has to start with a checkpoint entry, for reasons
Soil: the journal, and what makes it a WAL The transactions post left this one dangling: "Every new or changed object, every index update, and the new database version becomes an entry in a journal - an in-memory list first, not yet on disk [...] The journal is central enough to deserve a full post of its own later." This is that post. From transaction to journal entries The thing a transaction as
What's hitting my static blog, and where it's coming from Anyone running a publicly reachable webserver gets visitors fast. Not necessarily human ones. This blog is produced by a static site generator and served by nginx: a handful of HTML files, some CSS, a few images, nothing else. No WordPress, no PHP, no CMS, no application behind the webserver at all - there's no login form to brute-force, no
Soil: knowing what changed The transactions post left this on the table with a one-line mention: "'Dirty' itself comes from one of two tracking strategies - either you mark an object dirty yourself, or, as of the current version, Soil detects changes on its own by hashing an object's content and noticing when that hash no longer matches." That's the whole subject of this post: what those two strat