Temporal Validity on Real GitHub Fixes: Paper 2 Is on arXiv
arXiv:2608.20685 validates deterministic supersession on 130 marker-free atomic transitions from 707 SWE-bench GitHub issues. MemStrata reaches 0.91 accuracy versus RAG 0.57-0.59, with stale-fact error near 0 versus 36-38%.
Synthetic results are not enough
Paper 1 showed that retrieval-augmented generation has no model of time. When a value changes, cosine similarity retrieves both the old and new fact. A deterministic (subject, relation, object) supersession rule retires the stale value. Reviewers still ask: does that hold on real software history?
arXiv:2608.20685 answers that question. From 707 SWE-bench Lite and Verified GitHub issues, we extract 130 clean atomic state transitions and render them marker-free so the only currency signal is order.
What the published numbers say
On that set, MemStrata reaches 0.91 answer accuracy versus RAG at 0.57-0.59. When models are forced to answer, RAG serves the superseded value 36-38% of the time. An LLM reranker does not help. MemStrata drives that stale-fact error to about 0, at RAG retrieval latency (about 2.1 seconds versus about 18 seconds for the reranker).
These are research measurements on a locked, local, deterministic run. They are not a product SLA.
Scope is part of the result
Only about 18% of the 707 real fixes are clean atomic transitions. Paper 2 isolates the memory mechanism on that class. Extraction coverage of the remaining fixes is follow-on work (Papers 3 and 4), not hidden in the headline accuracy.
Read the paper: arXiv:2608.20685. Paper 1 remains arXiv:2606.26511.