Methodology
The number opens a gate. It does not prove the label.
Jev returns a confidence value with its choice. JevLab uses that value as a cutoff on one routing policy. This page is a reading guide for that cutoff. It is not a calibration study, and it does not name a threshold to ship.
Three uses of the same number
The same confidence value gets used in three different ways. A cutoff that fits one of them can be meaningless for the others.
| Use | What you are trusting | What JevLab does |
|---|---|---|
| Ranking | Which label is ahead of the others. The size of the gap is secondary. | Each case shows its label probabilities. There is no separate ranking policy. |
| Gating | The magnitude. At or above the cutoff, a valid suggestion is auto-adopted. Below it, or on a failed response, the task hands back to the caller. | The lab slider. 0.80 is the interaction starting point. |
| Margin | The gap between the first and second label. A high top score with a close second is a weak decision. | No separate control. The probability row stays visible so a close call can be seen. |
What this value is
- A score attached to one answer, on the question you asked. In this lab the question is which of 5 routing labels fits one task.
- The input to the gate above. Moving the slider changes how many suggestions the policy auto-adopts.
noneis the label for a task outside the five capabilities. Handoff means the policy refused to auto-adopt, including when the response failed.
It is not an accuracy rate. A confidence of 0.90 does not mean the label is right on 90% of your tasks. It is also not a check that the task contained enough evidence. Published notes have described cases where removing source material raised the score. JevLab has not reproduced that measurement. The working assumption here is that a high score can sit on a thin or wrong task.
No reviewed benchmark is published. The lab fixture is an unverified candidate set, so the slider does not establish a cutoff you should copy. No agreement rate or model accuracy is claimed.
What moving the slider shows
On the development lab, the slider recalculates in the browser. It does not call Jev. Three ratios move together. N is every sample, V is valid responses, A is valid responses at or above the cutoff, and C is the correct ones inside A.
- Auto-adopt rate, A / N.
- Accepted error rate, (A − C) / A. When A is 0, the rate is N/A.
- Handoff rate, (N − A) / N. Failed responses stay in this rate.
Those ratios describe the fixture on screen. They become a published measurement only with a reviewed report. Denominators and the rest of the policy are on the methodology page.
Before you copy a cutoff into code
Fit a cutoff on labels from your own queue. Shared model weights are not a calibration of your traffic. This lab's first case is English routing among code_search, test_runner, docs_lookup, code_review, none. The TypeScript export uses the same comparison as the slider and returns a suggestion or a handoff. It does not execute the tool named by the label.
The tasks this number is not being asked to solve are listed on the limitations page.