What a submission is

A submission is the folder Reactor Bench's package() writes for a run scored on every case of a track: entry.json, result.json, the icon, light rollouts and the scorer's frozen input. A results-only submission is that folder; a full submission also pins the model's Reactor workspace, weights and adapter, so a reviewer can reproduce it, and so it can later be deployed to Reactor. The submitting skill in reactor-bench (skills/submitting) produces it.

Submit a Hugging Face link

Publish the folder with publish(), then send the link it returns. A private dataset needs a read-only token; it is stored encrypted and deleted once the review ends.

curl -fsS -X POST https://reactorbench.com/api/submissions \
  -H 'content-type: application/json' \
  -d '{
    "type": "results-only",
    "contact": {"email": "[email protected]", "github": "my-handle"},
    "source": {"kind": "hf", "link": "hf://datasets/my-lab/my-model-wbench-navi@<commit>"}
  }'

The answer is {"id": "<uuid>", "status": "received", "uploads": []}. Keep the ID.

Upload the folder

Zip the folder as submission.zip, and send its entry.json and result.json as text with every file's size and SHA-256. A full submission's code and weights are the pins in its entry.json by default ("ref"); to upload them instead, set model.workspace or model.weights to "upload" and add model/workspace.tar.gz (the workspace, with its reactor.yaml) or model/weights.tar.

curl -fsS -X POST https://reactorbench.com/api/submissions -H 'content-type: application/json' -d @- <<EOF
{
  "type": "full",
  "contact": {"email": "[email protected]"},
  "model": {"workspace": "ref", "weights": "upload"},
  "source": {
    "kind": "upload",
    "entry_json": $(jq -Rs . < my-model/entry.json),
    "result_json": $(jq -Rs . < my-model/result.json),
    "files": [
      {"name": "submission.zip", "size": $(stat -c %s submission.zip), "sha256": "$(sha256sum submission.zip | cut -d' ' -f1)"},
      {"name": "model/weights.tar", "size": $(stat -c %s weights.tar), "sha256": "$(sha256sum weights.tar | cut -d' ' -f1)"}
    ]
  }
}
EOF

The answer lists each file with its part size and part count. For each file:

1. POST /api/submissions/<id>/parts with {"name": "submission.zip", "from": 1, "to": 1000} returns a presigned URL per part (at most 1,000 at once).
2. PUT each part's bytes (the file split every part_size bytes) to its URL, and keep the ETag header of each answer.
3. POST /api/submissions/<id>/files with {"name": "submission.zip", "parts": [{"part": 1, "etag": "..."}, ...]}.

Then POST /api/submissions/<id>/complete queues it for review.

Private code and weights

A full submission's code and weights can be private. Name each repository its entry.json pins, with a read-only token; the repository must be the pinned one. Tokens are stored encrypted and deleted once the review ends. Uploaded code or weights work with either source.

"model": {
  "workspace": {"kind": "ref", "repo": "https://github.com/my-lab/my-model", "token": "<read-only token>"},
  "weights": {"kind": "ref", "repo": "https://huggingface.co/my-lab/my-model", "token": "<read-only token>"}
}

What happens next

The submission is checked as it arrives, against Reactor Bench's formats and the benchmarks this site lists; anything wrong comes back as a 400 with the problems. Once it is reviewed, it appears on its leaderboard.

Review API

Reviewers' agents verify submissions through /api/review, with a reviewer API key minted on /admin (Authorization: Bearer rbk_...). The reviewing-a-submission skill is the procedure.

Publishing and rejecting stay with a person, on the submission's page.

Limits