An OpenAI-compatible chat API that runs several candidate answers, checks them, and returns the model's best attempt, verified by running it, with the scores reported per call. Same swdo_ key, same prepaid balance as everything else on this platform.
from $0.01 per verified request
Base URL https://assay.api.swarmdo.com/v1 · any OpenAI SDK, base-URL swap only · stream: true works · GET /v1/models lists the aliases. The aliases are third-party hosted models selected by SwarmDo, not SwarmDo-trained weights.
On our held-out chart set (n=64, best-of-4 through the hosted endpoint, two runs each): assay-chart went from 0.50 to 0.67 and 0.65 (CI excludes zero; none worse), median 16–18 s, p95 28–51 s; assay-fast from 0.42 to 0.68 and 0.66 (CI excludes zero; none worse), median 32–34 s, p95 66–71 s, so for charts either alias is a fair pick at the same price; assay-pro from 0.75 to 0.78 (CI excludes zero; none worse), median 12.5 s, p95 21 s — the interactive tier; its lift is small because the strongest backend is rarely wrong on the first try. Zero of 320 task-runs across the five runs scored below its first candidate. Treat both as background calls, or use the async flow below. For code, we measured the apply-and-parse check on its own and it does not move the resolve rate (+1 of 107); running the project's tests, which needs Assay beside the repo, is where the code result comes from.
assay-chart | With an image in the request, each candidate's code is rendered and the result compared to the chart (mode: "visual"); without one, each candidate must run and draw a figure (mode: "render"). Without an image, Assay can only check that the code runs and draws, so it rescues prompts whose first attempt fails and does nothing for charts that already run (measured: +0.05 on assay-chart, +0.14 on assay-fast); for text-to-chart without an image, assay-fast is the better pick. selected_score minus first_score is what selection bought on that call. |
assay-code | Send the file in the body as "assay_files": {"path": "contents"}: the edit is applied to the file you supply and parsed (mode: "code"), with apply and parse scores per candidate; the field is stripped before the model sees it. Running a repository's tests needs Assay beside the repo. |
assay-fast · assay-pro | General answers, checked by the verifier before one is returned; assay-pro is the strongest alias. |
| Nothing to verify | The request is passed through (mode: "passthrough") and billed per token instead. |
| Model | Prompt chars / max_tokens / N | Price |
|---|---|---|
assay-fast | 60,000 / 4,096 / 4 | $0.01 per verified request |
assay-chart | 32,000 / 4,096 / 4 | $0.01 per verified request |
assay-code | 120,000 / 4,096 / 4 | $0.02 per verified request |
assay-pro | 96,000 / 2,048 / 4 | $0.60 per verified request |
| Pass-through | — | the backend's list price × 1.5 per token, rounded up per call (a small call is 1c) |
selected_index is 0 when the reply is the answer you would have received without Assay; anything else means selection changed it. An identical request within 90 s replays the earlier reply (marked x_verifier.replayed: true); send x-assay-idempotency: off to generate afresh.
x-verifier-n is optional (default 4, capped per alias). A prompt over its cap is refused with 413 before any model call; max_tokens above the cap is clamped and reported in x_verifier.limits. No credit: 402 before any token is spent. Raw upstream model slugs: 400 — use the aliases. Authorization: Bearer, api-key and x-api-key are all accepted.
The hosted service can apply and parse an edit; it cannot run your tests. The local launcher can: it applies each candidate edit in a scratch copy of the repository named by the agent's working directory: line (confined to ASSAY_REPO_ROOT, default $HOME) and runs the project's tests (set ASSAY_TEST_CMD to a fast, scoped command); the candidate whose tests pass wins. Measured in this mode: blind single-shot django patches 32/107 → best-of-5 43/107 when tests can run (exact McNemar p=9.8e-04; the model must be shown the file).
curl -H "Authorization: Bearer $SWARMDO_API_KEY" -o assay-local.tar.gz https://assay.api.swarmdo.com/v1/launcher # ~53 KB: run.sh, README, proxy modules, local catalogue tar xzf assay-local.tar.gz && cd assay-local SWDO_KEY=swdo_… BACKEND_KEY=<your OpenRouter or any OpenAI-compatible key> ./run.sh # → an OpenAI-compatible endpoint at http://127.0.0.1:8188/v1 (the local API key is printed); needs python 3.10–3.13 or uv, and curl
Metering: each verified request is charged to your own SwarmDo account at the fee in the bundle — assay-code 3c, the other aliases 1c — through POST /api/assay/meter; pass-through calls are not charged by us (your BACKEND_KEY pays the model provider). With no credit on the account the launcher refuses verified requests (402) and keeps relaying pass-throughs.
Send x-assay-async: 1 and the call returns at once with 202 and a job to poll; results are kept for 15 minutes, per key.
202 {"id":"job_…","status":"queued","poll":"/v1/jobs/job_…","expires_after_s":900}
GET https://assay.api.swarmdo.com/v1/jobs/job_… (same key)
→ {"status":"queued|running|done|failed","result":{ …the normal completion, with x_verifier… }}
curl https://assay.api.swarmdo.com/v1/chat/completions \
-H "Authorization: Bearer $SWARMDO_API_KEY" -H "content-type: application/json" -H "x-verifier-n: 4" \
-d '{"model":"assay-chart","max_tokens":2000,"messages":[{"role":"user","content":[
{"type":"text","text":"Write matplotlib code that reproduces this chart. Reply with one python code block."},
{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}]}]}'
"choices": [ { "message": { "role": "assistant", "content": "```python …" } } ], ← exactly one choice: the verified winner
"x_verifier": {"n":4,"mode":"visual","scores":[0.0,0.71,0.84,0.66],"first_score":0.0,"selected_score":0.84,"selected_index":2,
"upstream_calls":4,"charged_cents":1,"latency_s":18.2}
Charges appear in your spend explorer under Assay. All hosted models: /models · reference: /docs#assay