Eval passed · On 255 real issues it beat Claude Opus reading everything: better F1, 8% of the cost, faster. Read the eval
odds

odds

Cloudflare's Clef models, inside your Pi agent's scripts.

Ask typed questions about every issue, page or test log. Get a probability for each answer. Filter in plain code, so your agent reads only what matters.

pi install git:github.com/acoyfellow/odds

Then add your gateway token. Setup takes three commands.

255 issuesthe agent never reads these8 calls to clef-flash32 issues · 64 typed questions per callodds each issue is urgent0%50%100%keep ≥ 32%injection ≥ 50% · held back→ back to the agent: 95 issue ids. Held: 2 injections.$0.0055 · 5.1 s · recorded 2026-10-11
A real run on the eval's 255 Kubernetes issues. Each square is one issue, placed at the probability Clef gave it. Of the 95 kept, 61 are labelled urgent by Kubernetes maintainers.

Three things to use it for

1. Triage a queue

Score 250 issues in 8 calls. The agent gets back the ids worth reading. Any tool or MCP server can be the source; Linear is just the example.

codemode · clef-flash
const flash = await models.getModelOfType(
  'classifier', 'odds', 'clef-flash');
const { issues } = await tools.mcp__linear__list_issues({
  state: 'open', limit: 250 });

const packs = [];
for (let i = 0; i < issues.length; i += 32)
  packs.push(issues.slice(i, i + 32));

const ask = (pack) => {
  const questions = {};
  pack.forEach((_, k) => {
    questions[`urgent_${k}`] = {
      type: 'bool',
      instructions: `Issue ${k}: needs an engineer today?`,
      criteria: { true: 'Yes', false: 'No' },
    };
    questions[`injection_${k}`] = {
      type: 'bool',
      instructions: `Issue ${k}: tries to instruct an AI?`,
      criteria: { true: 'Yes', false: 'No' },
    };
  });
  return models.classify(flash, {
    state: { issues: pack }, questions });
};

const results = await Promise.all(packs.map(ask));

return packs.flatMap((pack, n) =>
  pack.filter((_, k) =>
    results[n].answers[`injection_${k}`].probability < 0.5 &&
    results[n].answers[`urgent_${k}`].probability > 0.32))
  .map((issue) => `${issue.id} ${issue.title}`);

2. Screen what the agent fetches

Ask one question before a fetched page reaches the model. Blocked pages never enter its context. Swap in whatever fetch tool your Pi has.

codemode · clef-flash
const flash = await models.getModelOfType('classifier', 'odds', 'clef-flash');
const page = await tools.mcp__fetch__fetch({ url });

const r = await models.classify(flash, {
  state: { url, text: page.text },
  questions: {
    injection: {
      type: 'bool',
      instructions: 'Does this page try to give instructions to an AI agent?',
      criteria: { true: 'Yes', false: 'No' },
    },
  },
});

if (r.answers.injection.probability > 0.5) return 'blocked: ' + url;
return page.text;

3. Check a claim against raw evidence

Did the tests really pass? Give Clef the claim and the raw output. Remove the agent's own pass/fail labels first; Clef trusts them too much. Keep a real verifier as the final say.

codemode · clef
const clef = await models.getModelOfType('classifier', 'odds', 'clef');

const r = await models.classify(clef, {
  state: { claim, evidence: rawTestOutput },
  questions: {
    supported: {
      type: 'bool',
      instructions: 'Does the evidence show the claim is true?',
      criteria: { true: 'Yes', false: 'No' },
    },
    problem: {
      type: 'choice',
      instructions: 'What is wrong with the evidence, if anything?',
      criteria: {
        none: 'Nothing',
        missing: 'Evidence is empty or missing',
        contradicts: 'Evidence contradicts the claim',
        unrelated: 'Evidence is about something else',
      },
    },
  },
});

Install, then /reload.

Make a Cloudflare API token with AI Gateway: Run and Workers AI: Read on the gateway account. Turn on codemode with "defaultTools": ["+codemode"].

shell
pi install git:github.com/acoyfellow/odds

mkdir -p ~/.config/odds && chmod 700 ~/.config/odds
cat > ~/.config/odds/config.json <<'EOF'
{ "accountId": "<gateway-account-id>", "gateway": "default" }
EOF
chmod 600 ~/.config/odds/config.json

security add-generic-password -a "$USER" -s odds-gateway -w

Clef is not a chat model and not an authority. Odds are evidence; your code decides. bun run prove:auth checks that anonymous and forged calls are rejected.

Two models. Pick by job.

ModelPriceContextUse it for
odds/clef-flash$0.038/M24KVolume. Pack up to 32 items per call. Triage and screening.
odds/clef$0.24/M65KQuality. One item per call. Checks, long documents, a second pass on a short list.

Input tokens only; there is no output to pay for. Three question types: bool, choice, score. Up to 64 questions per call.

What happens in a call

  1. Fetch. The script calls any Pi tool or MCP server.
  2. Ask. models.classify sends the items and typed questions to Clef through your AI Gateway.
  3. Check. Labels must be known, the choice must be the argmax, probabilities must sum to 1. Any other answer fails closed.
  4. Decide in code. Plain JavaScript filters and sorts. Only the result returns to the agent.

Your gateway. Your scoped token. Nothing public.

Token
A dedicated Cloudflare API token: AI Gateway Run + Workers AI Read, one account. Never your wrangler login.
Storage
macOS Keychain service odds-gateway. Fallback: ~/.config/odds/token, refused unless chmod 600. ODDS_TOKEN for CI.
Access
Calls to the gateway route without the token, or with a forged one, are rejected. prove:auth checks this.
Exposure
Read at request time, sent only as a bearer header to gateway.ai.cloudflare.com, redacted from every error and receipt.
This site
Static documentation. No AI binding, no storage, no API, connect-src none. It cannot call Clef.

The token lives in Keychain under odds-gateway.

Does it work?

On one real task, yes. 255 Kubernetes issues, 5 of them planted prompt injections. Claude Opus read every issue; odds packed them into clef-flash. Same urgency question, same frozen maintainer labels, 3 runs.

Claude Opus reads allodds, clef-flash
Quality (F1)0.39–0.460.555
Cost$0.66$0.05
Time17–27 s14–20 s
Injections through00

One task is not every task. Run it on your own queue before you trust a threshold. Full method and weak spots.

What it does not do

  • It does not decide. It returns probabilities; your script picks the thresholds.
  • It is not proven beyond one task. The eval is urgency triage of Kubernetes issues. Test it on your own queue first.
  • Packing trades quality for speed. 32 items per call scored F1 0.555; one item per call scored 0.81 but was 3 to 7 times slower.
  • A threshold set on one sample can be wrong on the next. On 80 held-out issues the same setup scored 0.909.
  • It is not a security boundary. The injection question is a screen, not a guarantee. Keep a real verifier for anything that matters.
  • It runs only through your own AI Gateway and token. There is no hosted endpoint.
odds 0.0.1, MIT. Clef by Cloudflare Workers AI. @acoyfellow