While working on the new immutable engine of chaos today, I wrote myself into a corner and in the process re-discovered the classic coordinated transaction problem:

If you have two separate transactions that both need to commit together or roll back together, how can you ensure that happens as an atomic operation?

If I commit the in-memory transaction first but the database commit fails, I'm out of sync.

If I commit the database transaction first, then the in-memory commit fails I'm out of sync.

The context here is that the in-memory data is handled with immutable snapshots and opportunistic commits. If no one has committed since your request started processing, then your commit should succeed. If someone else has changed things in-between then your commit should fail.

I ended up solving it using a fairly standard coordinated transaction pattern:

The in-memory transaction has a lock. Inside the database transaction, just before committing it, I call PreCommit() on the in-memory transaction. That obtains the lock to ensure no one else can attempt to commit, then it runs all the validation rules for committing but stops just before doing an actual Interlocked.CompareExchange to update the global snapshot reference. At that point, we know the in-memory transaction will succeed if we commit it.

After that, we simply commit to the database. If it succeeds, we finish off by committing the in-memory transaction. If it fails, we roll back the in-memory transaction. We don't need to involve a Distributed Transaction Coordinator or any of that other messy stuff, and since I'm using immutable data structures all wrapped up in one overall container object, committing my in-memory transaction is guaranteed to be atomic via the magic of Interlocked.CompareExchange.

The key is that all but one of the transactions in the entire scheme must have a pre-commit phase that runs the validation rules and an exclusive lock that isn't released until you either Commit() or Rollback(). If anyone's pre-commit fails, you bail on the whole thing. Otherwise, they've all made promises that Commit() will be successful. Then you can attempt the one transaction that doesn't participate in pre-commit and respond accordingly.

Part 1: Russell's Rules For Highly Concurrent Software
Part 2: Immutability Part 2
Part 3: ContextLocal
Part 5: Interlocked.CompareExchange
Part 6: Immutability and Thread Safety