AgentMemorySync
A shared memory layer for Claude Code and Codex CLI. Both agents write into one searchable timeline per repo, and each new session starts with what the other one already did.
Two weeks of public history, most of the work before it
The repo's history is short because I pushed it once most of the system already worked: the first commit is 14,584 lines. The entry worth reading is the last one, where I rebuilt the retrieval layer after my own audit showed the benchmark was too easy.
-
Jul 6, 2026Audited my own claims
Before publishing I wrote up an audit of the project. All 106 tests passed, but the retrieval benchmark couldn't mean much: every query shared a distinctive word with its answer, so a plain SQL
LIKEscan scored a perfect 1.000. The audit is still in the repo, now marked superseded. -
Jul 12, 2026First public commit
The whole system landed at once: the SQLite timeline store, Claude Code hooks, the Codex MCP server, history backfill from both agents' transcripts, the FastAPI dashboard with headless dispatch, per-repo roles, and the hash-chained audit log.
-
Jul 16, 2026Settings for agent connections and local models
Added a Settings screen you can reach from anywhere, for checking whether each CLI is installed and connected and for pointing dispatch at local models, with two new test files to cover it. Later that day I added setup instructions written for the agents themselves.
-
Jul 24, 2026Rebuilt retrieval around a real embedding model
Replaced the hashed n-gram vectors with
bge-small-en-v1.5running locally through ONNX. Replaced the old 180-query benchmark with a 40-query one where every paraphrase query shares zero content words with its answer, enforced by a test. Re-tuned the fusion constant from 60 to 5. On the new benchmark, paraphrase recall went from 0.05 with the old vectors to 0.60.
Two agents on one repo, and neither remembers anything
Run Claude Code and Codex CLI against the same codebase and they behave like strangers. Each session re-reads files the last one already read and re-derives an architecture the last one already worked out. The two agents can edit the same file five minutes apart without knowing it. At best that wastes work; at worst it produces conflicting changes nobody notices.
What's missing is continuity: a shared record of what already happened that either agent can query when a session starts. I use both tools on the same projects, so I built that record.
One store, two ways in
Claude Code connects through session hooks. Codex connects through an MCP server, or through hooks on Codex builds that support them. Both paths call the same write and recall code, so everything either agent records lands in one SQLite database keyed by the git repo root. Two agents working from different subfolders of the same repo still share one timeline.
· MCP
write · recall · graph
Both clients write to and read from the same store. There's no per-agent copy.
Sessions, decisions, and file touches get written as timeline events. Secrets are redacted on the way in. Existing Claude transcripts and Codex rollouts can be imported without duplicates.
Each event goes into SQLite's full-text index and gets embedded by a local model. Vectors are cached in SQLite, keyed by content hash and model, so unchanged text never gets re-embedded.
When a session starts, the agent gets a compact briefing of recent and pinned entries. Anything older stays searchable on demand instead of being stuffed into every prompt.
On top of the store is a local dashboard. From it you can send a task to either agent, which runs headless with the shared context already loaded, and watch its log stream live: messages, shell commands with their exit codes, and file edits. A finished run can become a handoff file the other agent picks up. I built it for one loop in particular: Codex researches or plans, then Claude Code checks that plan against the codebase and carries it out.
Every file edit also goes into a file-touch log. If the two agents edit the same file within 30 minutes of each other, the dashboard raises a conflict. It only checks timing and can't tell whether the edits clash, but it catches the case that started this project: two agents in the same file with no idea the other was there.
Keyword search plus embeddings, merged by rank
Keyword search is great at exact identifiers like a function name, an error string, or a file path. It's useless when the question and the answer share no words. Embeddings handle paraphrase but blur exact lookups. So both run on every query and the system merges their rankings.
FTS5's built-in BM25 ranking scores the event text directly. It's fast, exact, and needs no model. On the benchmark it gets exact-identifier lookups right every time (1.00).
bge-small-en-v1.5 runs on CPU through ONNX via fastembed. It produces real 384-dimension embeddings with no API key and no network after a one-time 64 MB download. Matches need a cosine similarity of at least 0.60. If the model can't load, search falls back to lexical ranking and the dashboard says so with an amber "lexical fallback" chip.
RRF combines the two lists by rank position instead of raw score, so I never have to calibrate BM25 scores against cosine similarities. A result that either method ranks highly rises to the top.
The first version used hashed character n-grams as its "semantic" leg. That's keyword matching dressed up as a vector: two paraphrases with no words in common hash into different buckets, so it barely beat BM25. My old benchmark didn't catch it because its paraphrase queries reused their answers' words. I switched to a trained embedding model and rebuilt the benchmark so paraphrase queries share zero content words with their answers (a test fails the build if one does), and every case gets a hard negative that shares the query's keywords. All the numbers below come from that harder test.
Separately, an incremental code index covers the repo itself. For Python files it builds a symbol graph from the AST (calls, references, inheritance, and links to tests), so a lookup can expand from one hit to everything structurally connected to it. The MCP server exposes 11 tools in total, including search_project_context, search_code, find_callers, and explain_symbol.
The fusion function and how k was chosen
Each retriever returns an ordered list of event ids. RRF scores every id by 1 / (k + rank) and adds the scores up across lists. The interesting part is the value of k, and the comment above it explains the choice.
# The usual RRF constant is 60, which comes from fusing many rankers over # TREC-scale corpora. Fusing only two legs, it flattens the top ranks (rank 1 # scores 1/61 and rank 10 scores 1/70), so a document only one leg finds gets # buried by documents both legs merely tolerate. # # Swept on benchmarks/retrieval_v2, paraphrase recall@5 by k: # k=60: 0.550 k=20: 0.550 k=10: 0.550 # k=5: 0.600 k=3: 0.600 k=1: 0.600 RRF_K = 5 def _reciprocal_rank_fusion( rank_lists: list[list[int]], k: int | None = None ) -> dict[int, float]: # Read RRF_K at call time. Binding it as a default froze the value at import # and made the constant impossible to tune or test. if k is None: k = RRF_K scores: dict[int, float] = {} for ranks in rank_lists: for position, event_id in enumerate(ranks): scores[event_id] = scores.get(event_id, 0.0) + 1.0 / (k + position + 1) return scores
Excerpt from store.py, comment trimmed.
The full comment also records why the gain is smaller than it first looked. An earlier sweep showed k=60 scoring 0.10, but only because the similarity cutoff was still tuned for the old hashed vectors. Fixing the cutoff removed most of what k had been compensating for.
What the benchmark shows
Every change gets scored against a labeled set of 60 documents and 40 queries, split evenly between exact identifier lookups and zero-overlap paraphrases. It compares three systems: the old SQL LIKE scan, BM25 alone, and the full hybrid path.
| System | R@5 | nDCG@10 | MRR | exact | paraphrase | p50 |
|---|---|---|---|---|---|---|
| like · naive scan | 0.525 | 0.524 | 0.516 | 1.00 | 0.05 | 2.4 ms |
| lexical · BM25 only | 0.500 | 0.500 | 0.500 | 1.00 | 0.00 | 8.5 ms |
| hybrid · production | 0.800 | 0.670 | 0.599 | 1.00 | 0.60 | 21.4 ms |
Adding the embedding leg takes zero-overlap paraphrase recall from 0.00 to 0.60, and exact lookups stay at 1.00.
It's a synthetic suite built to catch regressions, and calling it proof of real-world task completion would oversell it. The first search after startup is also slower, about 2.9 seconds on this corpus, because it loads the model and embeds everything once. After that, vectors come out of the SQLite cache.
That cache is also where the benchmark caught a real bug. Hybrid search was taking 1,568 ms, and when I looked, the embedding cache had zero rows. Searches open the database read-only so they never block the hooks that are writing to it, which meant every attempt to save a vector failed silently and each query re-embedded the whole history. The fix opens a short-lived writable connection only when something new was computed. Search dropped to 21 ms, and re-running the benchmark with the cache disabled gave the same accuracy.
Security controls and current limits
- Per-repo roles: viewers can read, editors can curate context, operators can dispatch agents.
- Secret redaction is on by default for context, prompts, results, and logs.
- Every admin or dashboard change writes a SHA-256 hash-chained audit record, exportable as JSONL or CSV.
- A test walks the app's route table and fails if any route other than the five auth endpoints works without a login.
- Only Claude Code and Codex. No Cursor, Windsurf, or Aider.
- The symbol graph understands Python only.
- Claude history discovery is only verified end to end on Windows.
- SQLite isn't encrypted at rest, and dispatch jobs don't survive a restart.
AgentMemorySync is open source.