OpenClaw Shared State File Architecture: The Job That Rewrites a File It Only Half Owns

On September 4 the signal forcer in my fleet fired five signals for problems that were already fixed: deployment, message discipline, mock data, privacy and context bloat. Each one had a skill on disk with real content in it. What had gone missing was two fields in deficiency-report.json, signal_resolved and skill_authored, and a report generator had written straight over them. The run log counted it as at least the seventeenth time since July 29.

Seventeen times. Same two fields.

What the Lost-Update Guides Cover

The top results for multi-agent shared state tell one story, and they tell it well. Two agents read the same record, both change it, and the second write silently erases the first. One writeup ran five agents against a single file under a worst-case schedule and kept exactly one agent's work, even though all five writes had been acknowledged. The fixes they reach for come from databases: compare-and-set before every write, version numbers on records, append-only logs, and worktrees so agents stop sharing a filesystem in the first place.

All correct, for races. None of the three I read spends a section on the case I actually had, a scheduled job that regenerates a whole file every morning from its own inputs while a different job keeps adding fields to that file in between runs. There was no race. And none of them helps with the decision I had to make afterwards, which was whether this state belonged in a JSON file at all.

Why a Version Check Would Have Passed

Walk through one morning. The generator reads the failure-replay output, builds a fresh report object with the fields it knows about, and writes the file. Later the forcer reads that file, sees five categories with no signal_resolved flag, checks the skills directory, finds all five skills, and patches the flags back in. The next morning the generator builds its object from scratch again. It has never heard of signal_resolved, so the flags vanish and the signals fire again.

Bolt compare-and-set onto that loop and nothing changes. The generator reads the current file, notes its version, writes its own object back with the matching version, and the gate lets it through. A version check confirms you saw the latest file. It says nothing about whether you kept what was in it.

The patching made things worse in a quiet way. The forcer kept a _persistence_note field inside the report with the history of each recurrence, so the bug's own paper trail lived in the file the bug kept eating. Then on October 1, with python3 unavailable in its execution context, the forcer reproduced the script's logic by hand, using plain read and write tools against the JSON. That made three writers. One of them was a language model editing a file the generator believed it owned outright.

One Writer Per File, and the File Says Who

Here is the rule I would adopt on day one. Every state file has exactly one writer, and the file names it. A job may regenerate a file freely only if it owns every field in that file; if anything else needs to record something, that record goes in a separate file with its own owner, and readers join the two.

# state/deficiency-report.json   owner: failure-replay  (regenerated daily, never patched)
# state/signal-resolutions.json  owner: signal-forcer   (updated in place, never regenerated)

def open_signals(report, resolutions):
    done = {r["category"] for r in resolutions
            if os.path.exists(r["skill_authored"])}
    return [s for s in report["signals"] if s["category"] not in done]

The resolution only counts while the skill file still exists, so deleting a skill reopens its signal on the next read. Nobody has to remember to flip a flag back.

This repo already works this way. I inherited it. src/lib/blog-posts.json is rebuilt before every build by scripts/generate-blog-posts.mjs, which reads each post's headline, description and publish date straight out of that post's page.tsx, and nothing else writes to it. The sitemap goes a step further and keeps no list at all. It walks the route tree at build time, and the comment at the top of sitemap.ts gives the reason: "there is no second list to remember to update." This page reached both of them without me touching either file.

The fleet's status file learned the same lesson at a higher price. It was maintained by hand and drifted for weeks, at one point listing a retired agent as active, so it is now rendered from a SQLite graph under a header that says Do NOT hand-edit and a line I am fond of: if this file and the graph disagree, the graph wins by definition. The fleet registry architecture piece covers how that renderer decides what is allowed into the file.

JSON File or SQLite

If one job derives the whole file from sources that live somewhere else, keep the JSON. It is the cheapest state there is, it diffs cleanly in git, and regenerating it is the whole point. blog-posts.json is that case. So was the deficiency report, right up until a second job started writing into it.

Two writers touching different fields: split the file before you reach for a database. That was the fix the deficiency report needed, and it is an afternoon of work. The run log itself offered two options, make the generator read and merge before it writes, or move resolution state out of the file entirely. I would take the second every time, because read-then-merge makes the generator responsible for preserving fields it does not understand, and that is precisely the job it failed at, seventeen times or more. The registry piece I wrote earlier this month recommends the merge. I have changed my mind.

Move to SQLite when two writers need the same record, or when you need to know who changed a value and when they did it. A row per signal with resolved_by and resolved_at columns, written by an UPDATE that names the columns it sets, cannot drop a column it never mentioned. That is why the fleet graph is SQLite, and why the work queue keeps jobs in a table where a claim is one atomic statement. Speed is almost never the reason at fleet scale. The database-backed agents article has the one time it was, a CRM query that went from 2 seconds of grep and jq to 3 milliseconds.

The Fix I Can't Point To

By late September the forcer was coming back with NO_REPLY (September 27, September 30), and on October 1 all five categories still carried their resolved flags, yet I have not found a record of the change that stopped the generator eating them, which means I am trusting a fix I have never seen in a file that still has more than one writer.

Where to Start

List your state files. For each one, write down every process that writes to it: grep for the filename across cron scripts and skills, and count any agent that has write tools and has ever been told the path. A file with two names beside it gets split, or moved into a table, this week.

If you have not stood up your first agent yet, this is the cheapest moment you will get. The OpenClaw Starter Kit walks you through one agent with an annotated config and a setup checklist. Add a line to that checklist for every state file the agent creates, with the name of its one writer, before a second job exists to argue with it.

Then delete the patch job. If the split is right, it has nothing left to do.

Internal Links & Further Reading

• OpenClaw Learning Loop Architecture →

  The harvester that found all five skills on disk and correctly wrote nothing.

• OpenClaw Fleet Registry Architecture →

  A status file rendered from evidence, with a header that forbids hand edits.

• OpenClaw Work Queue Architecture →

  Jobs in a SQLite table, claimed atomically by one dispatcher.

• Database-Backed Agents →

  Schema design and migrations once a JSON file stops being enough.

FAQ