Agentic Physics Analysis — Running JFC on NRP LLMs

Teaching: 10 min · Exercises: 35 min · Total: 45 min

In Build a Simple Agent you built a ~30-line agent with two tools. This lesson jumps to the other end of the scale: JFC ("Just Furnish Context"), a framework from Eric Moreno, Sam Bright-Thonney, Andre Novak, Daniel Garcia and Phil Harris that runs a complete HEP analysis — strategy, event selection, statistical inference, and a 50–100 page analysis note — from a single physics prompt.

It is the same loop you just built, scaled up: an orchestrator that writes no code itself, spawning executor and reviewer subagents across seven phases, with a human gate before unblinding.

┌──────────────────────────────────────────────────────────────┐
│                       ORCHESTRATOR                            │
│   Never writes code. Holds: prompt, summaries, verdicts only  │
└─────┬────────────────────────────────────────────────────────┘
      ▼
  Phase 1 ──▶ Phase 2 ──▶ Phase 3 ──▶ Phase 4a ──▶ Phase 4b ──▶ Phase 4c ──▶ Phase 5
  Strategy    Explore     Selection    Expected     10% valid.   Full data    Document
  (2-bot)     (self)      (1-bot)      (1bot+bib)   (+HUMAN)     (1-bot)      (2-bot)

Each phase runs execute → review → check → commit, and a reviewer finding a physics problem traceable to an earlier phase triggers a formal regression back to that phase.

The exercise here follows Phil Harris's h4l_agent_test tutorial: a H→4ℓ mass measurement on CMS Open Data, reproducing the spirit of JHEP 11 (2017) 047.

Running it on NRP

JFC drives Claude Code, which speaks the Anthropic API. NRP exposes an Anthropic-compatible endpoint alongside the OpenAI-compatible one you've used all day:

So Claude Code can be pointed at NRP's open-weights models with the same LLM token you've been using, and the entire JFC framework runs on NRP GPUs.

Time budget. The setup below runs ~10–15 minutes, most of it the data download. The agent run itself is open-ended — Phil budgets ~20–30 minutes for the fast path to produce something worth looking at. A complete JFC analysis runs for hours and is deliberately out of scope; see Take it further at the end.

Resources. Pick a hub profile with at least:

No GPU — inference happens on NRP's GPUs, not yours.

Why 4 cores. The agent itself is almost entirely network-bound, sitting idle waiting on NRP inference; it uses negligible CPU. Cores matter for the analysis code the agent writes — decompressing ROOT files with uproot is CPU-bound, and the fits at the end lean on threaded BLAS.

Returns diminish quickly past ~8 cores, for a specific reason: the natural way to parallelise this is one worker per sample file, but the dataset is extremely lopsided — ZZTo4L.root is 572 MiB of the 865 MiB total, so two thirds of the work sits in a single file that per-file parallelism cannot split. Extra workers finish the small samples and then idle.

Careful: cores and RAM multiply. JFC's scale-out rules tell agents to reach for ProcessPoolExecutor on anything taking 2–15 minutes, and each worker holds its own arrays. On an 8 GB machine keep the pool at ~4; asking for 12 workers on 12 files is the fastest route to an OOM kill.

Disk breaks down as ~865 MiB of extracted samples, ~30 MiB of repositories, a few hundred MiB for the claude and pixi binaries, and whatever the agent writes. The take-home path adds a pixi environment carrying the full scientific-Python stack plus pandoc and LaTeX, which is the multi-gigabyte part.

RAM is driven by that same ZZTo4L.root: ROOT files are internally compressed, so materialising all of its branches at once lands in the multi-gigabyte range. Note the irony — JFC's own coding rules say "Prototype on a slice. ~1000 events first, full data only for production", but the fast path deliberately strips those rules out, so a naive agent is more likely to exhaust memory here than under the full specification. If a subagent gets OOM-killed, that is the reason, and telling the agent to read a slice or specific branches fixes it.

Part 1: Install Claude Code and Pixi

Two tools: the claude CLI (the agent runtime) and Pixi (the environment manager JFC uses — it is non-negotiable in the spec; agents are forbidden from using bare pip or conda).

curl -fsSL https://claude.ai/install.sh | bash
curl -fsSL https://pixi.sh/install.sh | sh

export PATH="$HOME/.local/bin:$HOME/.pixi/bin:$PATH"
claude --version
pixi --version

As in Agentic Workflows, a JupyterLab terminal is a separate process from the notebook — exports there do not reach it. Persist the PATH and your token so every new terminal picks them up:

# Paste your personal token from https://nrp.ai/llmtoken if it isn't already set.
: "${OPENAI_API_BASE:=https://ellm.nrp-nautilus.io/v1}"
: "${OPENAI_API_KEY:=<paste-your-token-here>}"

for RC in ~/.bashrc ~/.bash_profile; do
    touch "$RC"
    grep -v -E 'OPENAI_API_BASE=|OPENAI_API_KEY=|\.opencode/bin|\.pixi/bin|NRP managed LLM \(cms-hats-llm training\)' "$RC" > "$RC.tmp" && mv "$RC.tmp" "$RC"
    cat >> "$RC" <<EOF

# --- NRP managed LLM (cms-hats-llm training) ---
export OPENAI_API_BASE="$OPENAI_API_BASE"
export OPENAI_API_KEY="$OPENAI_API_KEY"
export PATH="\$HOME/.local/bin:\$HOME/.pixi/bin:\$HOME/.opencode/bin:\$PATH"
EOF
done

echo "Persisted. OPENAI_API_KEY = ${OPENAI_API_KEY:0:8}..."

Part 2: Point Claude Code at NRP

Claude Code reads an env block from ~/.claude/settings.json. This is where the Anthropic-compatible endpoint and your NRP token go.

Model choice matters more than usual here. Claude Code speaks the Anthropic protocol, and NRP's /anthropic endpoint is a translation layer over an OpenAI-style backend. That translation is where things break: reasoning models emit "thinking" blocks, and every agent turn emits tool-use blocks, and a bridge that mislabels either one will crash Claude Code's SDK with API Error: Content block is not a text block. We default to gpt-oss because it is the model vLLM uses as its own worked example in the vLLM ↔ Claude Code guide, so that path is the one actually exercised upstream. glm-5 and minimax-m2 are the next candidates if it misbehaves.

WebSearch is denied on purpose. It is a server-side Anthropic tool that only Claude models can emit — open-weights models on NRP cannot produce it, so leaving it enabled just burns turns on a tool that can never succeed. This matters for JFC, whose methodology insists every numeric constant be cited ("any uncited numeric constant is Category A"): with web search unavailable, the reference PDFs in docs/ are the citable source. WebFetch is a different, client-side tool and still works if the agent has a specific URL.

NRP_MODEL="gpt-oss"        # vLLM's documented Claude Code example — see note above
NRP_CONTEXT="131072"

mkdir -p ~/.claude
[ -f ~/.claude/settings.json ] && cp ~/.claude/settings.json ~/.claude/settings.json.bak && echo "Backed up existing settings to ~/.claude/settings.json.bak"

cat > ~/.claude/settings.json <<EOF
{
  "env": {
    "ANTHROPIC_BASE_URL": "https://ellm.nrp-nautilus.io/anthropic",
    "ANTHROPIC_AUTH_TOKEN": "$OPENAI_API_KEY",
    "ANTHROPIC_MODEL": "$NRP_MODEL",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "$NRP_MODEL",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "$NRP_MODEL",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "$NRP_MODEL",
    "CLAUDE_CODE_SUBAGENT_MODEL": "$NRP_MODEL",
    "ENABLE_TOOL_SEARCH": "false",
    "CLAUDE_CODE_AUTO_COMPACT_WINDOW": "$NRP_CONTEXT",
    "CLAUDE_CODE_EFFORT_LEVEL": "max",
    "CLAUDE_STREAM_IDLE_TIMEOUT_MS": "3000000",
    "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
    "CLAUDE_CODE_ENABLE_TELEMETRY": "0",
    "DISABLE_TELEMETRY": "1",
    "API_TIMEOUT_MS": "3000000",
    "CLAUDE_CODE_MAX_RETRIES": "10"
  },
  "permissions": {
    "deny": ["WebSearch"]
  }
}
EOF

python3 -m json.tool ~/.claude/settings.json

Confirm the Anthropic-compatible endpoint answers with your token before handing it a multi-hour job. HTTP 200 plus a non-empty reply means Claude Code will work.

Note the generous max_tokens. NRP's models are reasoning models — they spend part of the output budget thinking privately before emitting any visible text (the same behaviour you saw in Chat with LLMs). Ask for 64 tokens and the model will burn all 64 on reasoning and hand back "content": null with "stop_reason": "max_tokens" — which looks like a broken endpoint but is just an under-funded request.

curl -s -o /tmp/anthropic_check.json -w 'HTTP %{http_code}\n' \
  -X POST "https://ellm.nrp-nautilus.io/anthropic/v1/messages" \
  -H "x-api-key: $OPENAI_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d "{\"model\":\"${NRP_MODEL:-gpt-oss}\",\"max_tokens\":1000,\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: NRP OK\"}]}"

python3 -c '
import json
d = json.load(open("/tmp/anthropic_check.json"))
if "error" in d:
    print("ERROR:", d["error"]); raise SystemExit
c = d.get("content")
text = "".join(b.get("text", "") for b in c) if isinstance(c, list) else (c or "")
print("model:      ", d.get("model"))
print("stop_reason:", d.get("stop_reason"))
print("reply:      ", repr(text.strip()))
'

Reading the result:

This check only proves the endpoint answers. The bridge can still fail later, once Claude Code starts making tool calls — see below.

The LLM status dashboard shows what's currently up.

Part 3: Get the data and the framework

We use one working directory and absolute paths throughout. Phil's README navigates with relative ../../../ hops, which are easy to get wrong once you cd into the analysis directory — using $WORK avoids that entirely.

export WORK="$HOME/jfc-exercise"
mkdir -p "$WORK" && cd "$WORK"
echo "WORK=$WORK"

The CMS Open Data samples

Flat ntuples (~857 MiB) produced from 2017 NANOAOD with h4l_ntuplize.py: 10/fb of data plus Higgs signal (ggH, VBF, VH), ZZ, ggZZ, Drell-Yan and tt̄ backgrounds, hosted on NRP S3.

The download takes a few minutes. The cell is safe to re-run — it skips the download if data/ already exists.

DATA_URL="https://s3-west.nrp-nautilus.io/transfer-bucket/h4l-data.tgz"

cd "$WORK"
if [ -d data ]; then
    echo "data/ already present — skipping download."
else
    curl -L -o data.tgz "$DATA_URL"
    tar xzf data.tgz && rm -f data.tgz    # drop the 857 MiB tarball once extracted
fi
du -sh data 2>/dev/null; ls data | head

The framework and the tutorial context

Two repositories: jfc on the jfc_lite branch (the framework and its specification), and h4l_agent_test (this analysis's prompt, reference papers and ntuplizer).

cd "$WORK"
[ -d h4l_agent_test ] || git clone -q https://github.com/violatingcp/h4l_agent_test.git
[ -d jfc ]            || git clone -q -b jfc_lite https://github.com/violatingcp/jfc.git
ls -d h4l_agent_test jfc

Part 4: The fast path — prompt and context only

Phil's tutorial offers two routes, and this is the deliberate trade:

The fast path hands the model the physics problem and the papers, but none of JFC's guardrails. It gets you to a running agent inside a tutorial slot, and it makes the value of the full specification obvious by contrast — the JFC authors ship analysis notes from both configurations in h4l_agent_test/analysis_notes/ if you want to compare outcomes.

export ROGUE="$WORK/jfc/analyses/h4l_rogue"
mkdir -p "$ROGUE" && cd "$ROGUE"

ln -sfn "$WORK/data" data                              # symlink, don't copy 857 MiB
cp "$WORK/h4l_agent_test/h4l_ntuplize.py" .
cp -r "$WORK/h4l_agent_test/docs" .
cp -r "$WORK/h4l_agent_test/.claude" .
cp "$WORK/h4l_agent_test/prompt.md" .

ls -a

What you just staged:

Skim the prompt before launching — it is the analysis's founding document, and everything the agent does traces back to it:

head -5 "$ROGUE/prompt.md"

Part 5: Launch the agent

🖥️ Terminal step — claude is an interactive terminal UI, so launch it from a real JupyterLab terminal (File → New → Terminal), not the notebook. If that terminal was open before you ran the persistence cell in Part 1, run source ~/.bashrc first, or just open a fresh one.

cd ~/jfc-exercise/jfc/analyses/h4l_rogue
export PATH="$HOME/.local/bin:$HOME/.pixi/bin:$PATH"
cat prompt.md | claude --permission-mode auto

--permission-mode auto lets the agent write files and run commands without confirming each one — appropriate here because it is working in a scratch directory it created, and it is about to run hundreds of steps. Everything it touches lives under h4l_rogue/.

While it runs, inspect what it is producing from the notebook — the agent writes to disk continuously, so you can watch artifacts appear without interrupting it:

cd "$ROGUE"
echo "=== files produced so far ==="
find . -maxdepth 2 -newer prompt.md -type f \
     -not -path './.git/*' -not -path './data/*' -not -path './docs/*' 2>/dev/null | head -30

echo
echo "=== figures ==="
find . -name '*.png' -o -name '*.pdf' 2>/dev/null | grep -v '^./docs/' | head -10

Part 6: What "good" looks like

For this analysis the physics target is concrete, which makes grading the agent easy: a four-lepton invariant mass spectrum with a Higgs peak near 125 GeV sitting on a ZZ continuum, and a signal-strength fit returning μ ≈ 1 within uncertainties.

Judge the run on:

1. Did it produce a mass plot at all? The single most common failure is never getting past data loading.
2. Is the peak in the right place? A peak at 125 GeV means the four-lepton kinematics were reconstructed correctly. A peak somewhere else means a bug worth finding.
3. Are the backgrounds normalized? Cross-sections are in prompt.md; each sample must be scaled to 10/fb.
4. Is μ credible? A μ of 1.0 ± 0.3 is a real result. A μ of 40, or a fit with χ² identically zero, is not — JFC's spec calls χ² = 0 "an alarm, not a result".
5. Could someone else reproduce it? The prompt explicitly asks that the mass and μ extraction be easy to rerun.

Compare against the reference PDFs the JFC authors produced with Claude Opus and full JFC context, which are checked into the tutorial repo:

ls -la "$WORK/h4l_agent_test/analysis_notes/"

Take it further: the full JFC specification

The fast path removed the framework. Putting it back is the actual point of JFC — and is the natural take-home from this session.

Run this after the workshop (the pixi install alone pulls a full scientific-Python stack, and the analysis runs for hours):

cd ~/jfc-exercise/jfc
pixi run scaffold analyses/h4l_analysis --type measurement
cd analyses/h4l_analysis
pixi install

# stage the same physics context, plus the isolation config the spec needs
ln -sfn ~/jfc-exercise/data data
cp  ~/jfc-exercise/h4l_agent_test/h4l_ntuplize.py .
cp -r ~/jfc-exercise/h4l_agent_test/docs .
cp -r ~/jfc-exercise/h4l_agent_test/.claude .
cp  ~/jfc-exercise/h4l_agent_test/.analysis_config .
cp  ~/jfc-exercise/h4l_agent_test/prompt.md .

cat prompt.md | claude --permission-mode auto

Scaffolding creates the phase directories, per-phase CLAUDE.md files, a pixi.toml, and symlinks to agents/, conventions/ and methodology/ — the full specification. Check .analysis_config if your data lives somewhere other than $PWD/data.

Then read what the spec actually enforces — it is the most transferable part of this lesson even if you never run a full analysis:

• jfc/src/methodology/03-review.md — the review protocol: Category A/B/C findings, iteration caps, and the rule that a result more than 3σ from a well-measured reference is automatically blocking.
• jfc/src/agents/executor.md — how a subagent is briefed. Note that prompts are files under version control, not ad-hoc strings.
• jfc/analyses/h4l_analysis/CLAUDE.md — the orchestrator contract, including the regression checklist it must run after every review.

Those three ideas — typed findings, bounded iteration, and prompts as versioned artifacts — are what separate this from the loop you wrote in Lesson 4, and they transfer to any agentic system you build.

Discussion

• The framework is the product. JFC ships almost no analysis code. Its value is the encoded process: what a phase must produce, who reviews it, what blocks advancement. That is the same insight from Lesson 4 — tool descriptions and error strings are prompts — applied to an entire research workflow.
• Portability, again. JFC was written against Anthropic's API. It ran here on open-weights models on NRP GPUs, unchanged, because the endpoint speaks the same protocol.
• Human gates are load-bearing. JFC pauses for human approval before unblinding — the agent does not decide on its own when to look at the full dataset.
• Honest reporting. If your run stalled or produced a wrong peak, that is a legitimate result about open-weights models on long agentic tasks. Bring it to the discussion — the failure modes are more useful to this community right now than a clean success.

Where this came from: JFC is by Eric Moreno, Sam Bright-Thonney, Andre Novak, Daniel Garcia and Phil Harris — AI Agents Can Already Autonomously Perform Experimental High Energy Physics. The H→4ℓ exercise is Phil Harris's USCMS tutorial, adapted here to run on NRP.

References

• JFC framework · jfc_lite branch used here
• h4l_agent_test tutorial
• NRP client configurations — the Claude Code settings used in Part 2
• NRP available models · LLM status dashboard
• CMS H→4ℓ, JHEP 11 (2017) 047 — the reference analysis
• Pixi · Claude Code