The case
Synthetic inquiries with known ground truth pass through hypothesis choice, accumulation, misleading evidence, discovery of a defective source, and possible reopening. Agents are randomized to six memories: current conclusions and confidence; a narrative summary; raw history with retrieval; a justification graph; a graph with evidence attributes and revisions; a task-optimized compression.
Two arms. In one every memory has access to the same facts, which tests access and format. In the other each gets a fixed compression budget, which tests what it keeps.
Outcomes, in order: decisions on revision tasks scored in the generating environment, not reproduction of the record’s graph (CLAIM-190); whether the agent can reconstruct its earlier grounds; and rigidity, whether anchoring keeps a conclusion past when it should have gone (CLAIM-184).
What each outcome would show
A6: it could tell apart “Structured memory retains more revision-relevant information under compression”, “makes retained information more accessible to bounded models” and “improves the execution of revision procedures given equivalent information”, or show that “the apparent advantages of structured memory disappear when information content and computation are adequately controlled.”
Main threat
A6: “Success in synthetic environments would not automatically establish that the same representations are sufficient for open-ended scientific inquiry, where the state space, relevant variables, available interventions, and even the appropriate questions can change over time.” That is QUESTION-112.