Update
Real SimpleFold-100M smoke completed on 2026-05-13. Three engineering targets produced real predictions, lDDT, and TM-score records. See the real smoke result packet.
Question
Can a single researcher using an M3 Max workstation plus an auditable agentic pipeline produce reproducible, inspectable structure-prediction benchmark results on open protein models?
The narrow question for this smoke run was whether the orchestration substrate can produce a complete, auditable record before real inference is introduced.
Run details
| Field | Value |
|---|---|
| Paper identifier | 10.48550/arXiv.2509.18480 |
| Paper title | SimpleFold: Folding Proteins is Simpler than You Think |
| Recipe source | pilot/fixtures/simplefold_smoke_recipe_v0.json |
| Test set source | pilot/data/test_set_v0.csv |
| Targets | 1L2Y, 1CRN, 1UBQ |
| Mode | Deterministic dry run, no structure inference |
Result
successful_prediction
The dry-run fixture completed end to end and produced local artifacts plus Durable Object audit entries.
What worked
- The local Workers stack ran the orchestrator plus five MCP services.
cite-verifyresolved the SimpleFold DOI.pdf-parsefetched and cached the SimpleFold PDF.- The runner accepted a deterministic recipe fixture and emitted reproducible output files.
- The Durable Object persisted recipe, predictions, scores, verdict, and audit entries.
What broke and was fixed
- The examples pointed to the wrong arXiv ID; they now point to
2509.18480. - DOI routing broke on
/;/paper-audit/:doinow avoids ambiguous routing. - The fixture path originally depended on the Claude Code Validator; deterministic fixture validation now exists.
pilot/.envno longer overrides explicit local environment variables.- Wrangler 3 local runtime was unstable; the local test used
wrangler@4.
What remains unproven
- Real SimpleFold inference through the public
science.eigenatlas.comtunnel. - Reader extraction from the PDF without the fixture.
- SimpleFold-3B inference on Apple Silicon.
- Authentication between public Workers and the M3 Max tunnel.
- Publication-grade test set curation.
Next experiment
Deploy the updated orchestrator, run the same SimpleFold-100M smoke through the public tunnel, and compare a no-fixture Reader extraction against the fixture recipe. Phase 1 should not start until that no-fixture public path completes.
Source document: docs/12_phase0_findings.md