• Purpose: Evaluate published hypotheses using the Biomni agent, summarize results, filter for validity via expert review, and score final evidence.
• Scope (manuscript): Local runs only, stored under Biomni/agent_replicate/.
• Supplemental: Additional web runs exist and can be summarized separately (see Additional Web Runs).

• Source: replication_sheet_scoring.csv (IDs, category, disease/context, target gene(s), core claim, data-driven testing prompt).
• Local run workspace: Biomni/agent_replicate/ (one subfolder per target/hypothesis).

• Agent: Biomni A1 (see Biomni/README.md for environment setup and usage).
• Inputs: The hypothesis-specific data-driven testing prompt (from the replication sheet) is passed to the agent.
• Outputs: A validation notebook per target folder:
  • Biomni/agent_replicate/<TARGET_FOLDER>/biomni_validation.ipynb

Local run procedure used:

• Biomni/agent_replicate/USAGE.md (how notebooks were generated and opened/executed)

Local runs used for the manuscript are located at:

• /Users/kuan-lin.huang/Library/CloudStorage/OneDrive-TheMountSinaiHospital/Huang_lab/manuscripts/AgentReplication/Biomni/agent_replicate

• Script: Biomni/agent_replicate/summarize_local_notebooks.py
  • Full path: /Users/kuan-lin.huang/Library/CloudStorage/OneDrive-TheMountSinaiHospital/Huang_lab/manuscripts/AgentReplication/Biomni/agent_replicate/summarize_local_notebooks.py
• What it does: Converts each local notebook (including outputs) to text and uses an LLM to extract structured fields from the final analysis section (statistical evidence, outcome, quality, key findings).
• Outputs (written under Biomni/agent_replicate/):
  • notebook_summary.json
  • notebook_summary.csv

Local summarization procedure used:

• Biomni/agent_replicate/SUMMARIZE_README.md

• Human expert review is applied to local notebooks used in the manuscript.
• Criteria: methodological correctness, appropriate use of data, correct statistical testing/interpretation, reproducibility of steps.
• Result: Only expert-validated notebooks are used for evidence evaluation and scoring.

• Final evidence for each hypothesis is evaluated solely from the set of expert-validated notebooks.
• Scoring is recorded in the first four columns of replication_sheet_scoring.csv:
  • Local_lineage_dependency: local context-specific dependency evidence (manuscript terminology).
  • Local_other_evidence: additional local association evidence (e.g., expression, survival, signatures).
  • Web_lineage_dependency: web context-specific dependency evidence (supplemental).
  • Web_other_evidence: additional web association evidence (supplemental).

• Environment: Follow Biomni/README.md to set up and activate biomni_e1 and install biomni.
• API keys: Biomni/agent_replicate/summarize_local_notebooks.py requires OPENROUTER_API (and optionally OPENROUTER_MODEL) to run the LLM-based summarizer.

• Web session exports: biomni_web_outputs/
• Web summarization: scripts/summarize_biomni_web.py → biomni_web_summary.csv and biomni_web_summary.json
• Web notebooks were also expert-reviewed for validity; however, they are not included in the manuscript’s primary results because the initial free Biomni web-portal credits available to test users were insufficient to run the full target set.