Run
How to run DrugTargetWorld v1.0
Install
DrugTargetWorld runs in Harbor.
uv tool install harborRun the benchmark
Harbor pulls the task and the image, downloads and checksum-verifies the world data, starts the experiment service, runs your agent, then scores it in a separate verifier container.
harbor run \
-d drugtargetbench/drugtargetbench@v1.0 \
-a claude-code \
-m claude-opus-5Options
Appended to harbor run.
--limit 1 # one task instead of all 60
-n 2 # concurrency, default 4; each trial needs ~17 GB of disk
-k 3 # 3 attempts per task
# a single specific task
harbor run -p drugtargetbench/hard-02-full-program -a claude-code -m <model>harbor agent list shows the available agents: claude-code, codex, aider, swe-agent, terminus, oracle and others.
What to expect
- 60 — 20 worlds x 3 budget regimes
- Downloads ~17 GB
- ~345 GB unique, each world once rather than once per task
- 345 GB plus Docker overhead
- Rubric v0.9, 0-100, normalised to 0-1 for Harbor
Results land in jobs/. Each trial writes reward.txt and score.json with the full component breakdown: target identification, causal confidence, discrimination, direction of effect, phenotype construction, safety penalty.
The task
The agent gets a biobank of 54,000 synthetic participants in /app/data: proteomics, transcriptomics, metabolomics, genotypes, EHR, ECG, coronary CT, survival and raw cine-MRI. No phenotype column.
It must derive a cardiac phenotype from the imaging, identify which proteins causally drive disease against ten planted trap mechanisms, and say which direction a drug should move each. It can buy experiments within its budget.
request-experiment --kind knockdown --protein PROT_0123 # $400k
request-experiment --kind cell_perturbation --protein PROT_0123 # $150k
request-experiment --balanceOutput goes to /app/results/submission.json and /app/results/phenotype.csv.
Links
Citation
Margolis S, Schmiedmayer P, Huang A, et al. DrugTargetWorld: A Synthetic Biobank for Training and Benchmarking AI Scientists. arXiv:2610.09558 (2026).
If you use the imaging, also cite ACDC.
O. Bernard, A. Lalande, C. Zotti, F. Cervenansky, et al. “Deep Learning Techniques for Automatic MRI Cardiac Multi-structures Segmentation and Diagnosis: Is the Problem Solved?” IEEE TMI 37(11):2514–2525, 2018.
No full 17 GB trial has run end to end. Every component is verified: the pinned revision resolves, checksums match, the experiment service loads, the verifier scores 43.33 to reward 0.4333, and all 60 tasks download clean from the registry. Bulk materialisation inside a Harbor-managed build has not been exercised. Run one complete task before pointing anyone else at it.