Soil: catching write conflicts, optimistically
Every transaction in Soil gets to act as if it has the whole database to itself - read whatever it wants, take its time deciding what to change, and work all of that out without ever announcing itself to anyone else running at the same time. That's a comfortable illusion right up until two transactions both decide to change the same thing. Something has to notice, or the second commit just quietly erases the first one's work as if it never happened - the kind of bug that doesn't announce itself either, it just shows up later as data that's wrong for no reason anyone can find.
Put the last two posts side by side and this problem stops being hypothetical and starts being inevitable. The transactions post showed that changing something is cheap and quiet: a transaction changes an object in place, Soil notices that on its own by comparing hashes, and nothing about that reaches out to lock anything or tell any other transaction it happened - the actual writing only happens later, all at once, when `commit` is called. The MVCC post then showed that reading is just as solitary: a transaction gets handed one fixed `readVersion` the moment it starts, and it keeps reading through that same frozen view for as long as it lives, however long that turns out to be.
Put together, those two facts describe two transactions that can both start from the exact same snapshot, both read the exact same object, both spend an arbitrary amount of time deciding what to change about it, and both arrive at `commit` still convinced they're the only one working on it - because, up to that point, nothing has ever told either of them otherwise.
Two posts back, committing was broken into six locked steps, and step two - "lock every record and check for conflicts" - got a one-line mention with a promise to come back to it. Neither of the last two posts actually showed what happens once both of these transactions really do reach `commit`. This is that post.
What a conflict actually looks like
Soil doesn't stop a transaction from reading something and quietly working on a change to it for a while. What it won't do is let two transactions both get away with having changed the same object, one of them silently overwriting the other's work. Here's that exact situation, in a real test:
obj := SoilTestGraphRoot new nested: (SoilTestNestedObject new label: 'first').
tx1 := soil newTransaction.
tx1 root: obj.
tx1 commit.
tx2 := soil newTransaction.
tx2 root nested label: 'second'.
tx3 := soil newTransaction.
tx3 root nested label: 'third'.
tx3 commit.
self should: [ tx2 commit ] raise: SoilObjectHasConcurrentChange`tx2` opens, changes the label to `'second'` locally, and just sits there uncommitted - nothing about that is a problem by itself. `tx3` then opens, makes its own change to the very same object, and commits first. Only now, when `tx2` finally tries to commit its own change, does anything go wrong: `SoilObjectHasConcurrentChange`. The conflict wasn't in reading, or in changing something locally - it's specifically in trying to commit a change to something that's already moved on without you.
The check itself
The mechanism behind this is almost embarrassingly small. Every record a transaction is about to write remembers the position it originally read that object's previous version at. At commit time, that remembered position gets compared against whatever position is actually current on disk right now:
validateReadVersion
"a new object record cannot conflict as the record is not on disk"
(previousVersionPosition == 0) ifTrue: [ ^ self ].
currentPosition := transaction objectRepository heapPositionOf: objectId.
(previousVersionPosition = currentPosition) ifFalse: [
SoilObjectHasConcurrentChange new
objectId: objectId;
signal ]If they still match, nothing else committed a newer version in the meantime, and this transaction is free to proceed. If they don't match, somebody else's commit already moved that object's chain forward, and `previousVersionPosition` - the version this transaction thought it was building on top of - is stale. There's one deliberate exception: a genuinely new object, one that never existed on disk before this transaction created it, has nothing to conflict with yet, so the check is skipped outright.
`heapPositionOf:` doesn't scan anything to answer "what's actually current" - it's the same object index lookup from the very first post in this series, the one that maps an id straight to a byte position. Checking for a conflict costs exactly one index lookup per touched object, not a scan of anything.
There are two separate places that know where an object's current bytes are: the index file, which is updated the moment a new version lands, and each transaction's own record of the position it built its change on top of. Normally those agree. A concurrent commit in between is exactly what makes them disagree - the index now points past what any transaction still holding the old position knows about.
Why the lock is short enough to not matter
This check runs inside the single global critical section from the commit post - `Soil>>critical:`, which is a real `Semaphore forMutualExclusion` underneath, so only one transaction's commit is ever inside it at a time, database-wide - and each record it's checking gets its own per-object-id lock first, taken on that same index entry. That's a deliberately narrow lock: it protects the moment of comparing-and-stamping a new position, not the object's bytes, and definitely not any reading. Once every touched record has passed its check, the new versions get written and the locks release. A transaction that never writes anything never takes one of these locks at all - which is exactly the asymmetry the last two posts kept pointing at: readers are free because there's nothing here for them to wait on.
What this doesn't catch
This is a write-write check on individual objects, nothing more. Take two on-call engineers, A and B, with one rule: at least one of them has to stay on call. Both are on call right now. One transaction reads that state, sees B is covering, and takes A off call. Another transaction reads the same state concurrently, sees A is covering, and takes B off call. Neither transaction ever writes to the object the other one is reading - A's transaction only touches A, B's only touches B - so `validateReadVersion` has nothing to flag for either of them. Both commit cleanly. The rule is now broken: nobody is on call, and nothing in this chapter noticed.
That's write skew, the classic gap in snapshot isolation: it stops lost updates cold, but it was never designed to stop two transactions from jointly violating an invariant that spans more than one object. Soil doesn't try to close that gap - it gives you snapshot isolation, not full serializability, and it's honest about which one that is.
Where this leaves us
A transaction can read and change whatever it likes without ever taking a lock for it. Only at commit time does it find out whether that was still safe: one short-lived lock and one index lookup per touched object, comparing the position it thought it was building on against whatever is actually there now. Match, and the commit proceeds; mismatch, and it aborts loudly instead of quietly overwriting someone else's work. What it can't see is a conflict that never touches the same object twice - that's a different, harder guarantee Soil doesn't make.
The code is on github if you want to read ahead.