Evaluation report

Do jev's probabilities mean anything?

A measured answer, from nine experiments and twenty-eight predictions fixed in writing before any data was collected.

The findings are at the end. If you want the results before the method, go to Conclusions, which collects every prediction, every surprise, and what the results change. The sections between here and there define the terms those findings use.

What this document is

This reports an evaluation of jev, a decision model sold by TypeSafe. The evaluation followed a plan written before any request was sent. That plan fixed the nine experiments, the sample size of each, and twenty-eight predictions, each stated with the result that would prove it wrong. Nothing here was chosen after seeing an outcome.

The purpose was not to grade the model. It was to find out which of several proposed ways of using it would work. Each experiment therefore ends with what it settles, and the Conclusions gather those into three decisions a reader can act on.

One run produced everything below: 123,805 requests over 138 minutes, at a cost of $12.69. Five requests failed and were excluded; they are counted, never replaced by a default value.

The model and how you ask it things

jev answers questions about a piece of text. One request carries a single state — the text, or a structured object — and one or more questions about it. Every question in a request is answered from that same state, and a request may carry many questions at once. Whether asking many at once costs anything is the subject of the third experiment below.

Questions take three forms. This document uses the vendor's names for them.

noul
A yes-or-no question. It returns one probability between 0 and 1, and nothing else.
choice
A question supplying up to 255 named options. It returns the option selected, a probability for every option, and a separate number the vendor calls confidence.
score
A question supplying an ordered list of rubric points, lowest to highest. It returns a position on that scale, a probability for each point, and a confidence.

A noul returns no confidence. Only a choice and a score do. That absence matters in the ninth experiment, which asks whether confidence drops when the model cannot know the answer; there, a substitute had to be used for yes-or-no questions, and the substitute is named where it appears.

Every request in this run was answered by version jev-1.13.0.

What the evaluation asks

A model that reports a probability is making a claim that can be checked. If it says 0.9 on a thousand questions, about nine hundred should turn out true. A model whose probabilities behave that way is called calibrated.

Calibration is the stated reason to choose jev over a language model, and a vendor's calibration on the vendor's data does not establish calibration on yours. So the evaluation asks three questions:

  • Do the probabilities mean what they say on data the vendor did not calibrate against?
  • Does answer quality hold when many questions are asked in one request, which is the arrangement that makes the model cheap enough to use at scale?
  • Where does the model stop working, and does it signal that it has stopped?

How the experiments were run

Four rules applied throughout.

Ground truth came from solvers, never from the model
Every problem was generated together with its correct answer: satisfiability from a SAT solver, reachability from a graph search, Sudoku from a solver that also verified the puzzle had exactly one solution. The model was never asked to grade itself or another model.
Every accuracy is reported beside a baseline
Three baselines were computed where each applies: the random rate, one divided by the number of options; the majority-class rate, the score from always giving the commonest answer; and a cheap heuristic, a short program using an obvious surface feature, such as reading clause density off a formula's header. An accuracy without a baseline beside it is not a result, because a set with lopsided answers makes any accuracy look good.
Problems were generated fresh, at a fixed seed
No standard benchmark was used, because a published benchmark may sit in training data. Every problem is reproducible from its generator, difficulty, seed, and index.
A failed request is never a data point
Failures are counted and excluded. None is replaced by 0.5, by false, or by the commonest answer.

The measurements

Accuracy
The fraction of answers that match the known answer.
Expected calibration error
Sort the answers into ten bins by the probability given. Within each bin, compare the average probability against the fraction that actually turned out true. The expected calibration error is the average of those gaps, weighted by how many answers fall in each bin. Zero means the probabilities are exactly honest. It is the single most important number in this evaluation, and it is abbreviated ECE in the tables.
Brier score
The average squared distance between the probability given and the outcome. It penalises a model that hedges everything near 0.5, which the calibration error alone would not.
AUROC
How well the probabilities rank true cases above false ones, ignoring whether the numbers themselves are right. A model can be badly calibrated and still rank perfectly, so both numbers are needed to tell those two failures apart.
Reliability diagram
The picture of the calibration bins described above: predicted probability on one axis, observed frequency on the other, with a diagonal showing where a perfectly calibrated model would sit. Several appear below. A curve that runs the wrong way is a worse result than one that is merely offset.

The grades

The plan set five grades and applied them to every condition separately, not to an experiment as a whole. A model that does well on easy problems and fails on hard ones is useful if you know where the change happens, so the tables below report a grade for each difficulty rather than one grade overall.

PerfectSaturates the measurement. Nothing left to test.
SuperhumanMatches or beats a knowledgeable person given unlimited time, at a rate that makes new work possible.
HumanMatches an unhurried competent person. Useful, but the speed unlocks nothing new.
BadAbove chance, below a short program. The grade most easily mistaken for success.
Doesn't workAt or below chance, or confidently wrong in a way that tracks nothing.

The two checks that make the rest readable

Every formal subject here — satisfiability, reachability, Sudoku — lies outside what a model trained on decisions was built for. A poor result on one could mean the subject is hard, the model is weak, or the measuring apparatus is broken. Two controls separate those.

The positive control is a set of support-ticket routing problems with unambiguous answers, run alongside every experiment. It scored 0.980 throughout this run. A weak result on formulas can therefore be read as a fact about formulas.

The smallest difference worth reporting comes from the first experiment below, which measures how much the model's own answers vary between identical requests. That variation is 0.0112. Differences smaller than about twice that figure are not findings, and no claim below rests on one.

Terms the experiments use

The sections below quote the experiments' own wording: the reasons the grading rubric generates, the outcome text each experiment writes, and the predictions the plan fixed in advance. Most of the terms there are standard usage in some field — monotone and total variation from mathematics, witness from complexity theory, noise floor from signal processing, phase transition from the satisfiability literature, ceiling from experimental psychology, and the named statistical tests from statistics. Two are particular to this plan and would not be recognised elsewhere: enrollment and signature. All are defined here because a term being standard in its own field is no help to a reader who works in another one.

monotone
A curve that only ever moves one way. A reliability curve that rises where it should fall is not monotone, and the rubric treats that as the worst result whatever the other numbers say.
knee
The point at which a curve that has been flat starts to fall. Several experiments sweep a difficulty looking for one and report when there is none in the range swept.
ceiling
A task the model already answers almost perfectly. A decline cannot be measured there, because no accuracy is left to lose.
noise floor
The model's own variation between identical requests, measured at 0.011. The smallest difference that can be read as a finding.
signature
This plan's term for a pattern it named in advance as evidence of a particular failure. The density-heuristic signature, for instance, is accuracy near chance on a balanced set while an unbalanced sweep still looks good.
witness
A specific example that settles a question. A satisfying assignment is a witness that a formula is satisfiable. Unsatisfiability has no witness, which is part of why it is the harder direction.
phase transition
The clause ratio near 4.26 at which random formulas stop being mostly satisfiable and start being mostly unsatisfiable. The change is abrupt, which is what makes it a useful target curve.
enrollment
This plan's term for putting several related outcomes into one choice over their combinations instead of asking about each separately. It is not general usage.
quantised
Reported to a fixed precision. Every probability this model returned lies on a two-decimal grid.
total variation
A distance between two probability distributions: half the sum of the absolute differences. Used to measure how far putting questions in one request moved an answer.
DIMACS
The standard plain-text format for writing a satisfiability formula.
Dyck words
Strings of brackets used to test counting. A Dyck word is one whose brackets are correctly matched and nested.
McNemar's test
A significance test for two methods measured on the same items. It looks only at the cases where they disagree, which is what makes it the right test for a paired comparison.
Wilson interval
A confidence interval for a proportion that stays sensible near 0 and 1, where the ordinary interval does not.
bootstrap
Estimating an interval by resampling the measured data many times, rather than assuming a distribution for it.

How to read the tables

Each experiment reports one row per difficulty. Deciding metric names the measurement that set the grade for that row, value is what it came to, baseline is what it had to beat and where that number came from, and why gives the rule that produced the grade. Where a grade was set on a smaller sample than the row's own, both counts are shown.

The nine experiments

In the order the plan ran them, which is the order in which an early result could invalidate a later one. Each gives the predictions made in advance, the method, the result, what was unexpected, and what it settles.

The plan numbered these E1 to E9 in a different order, and those labels survive in the file names of the recorded results. Both numbers appear on each heading below: the position in the run, and the plan's own label.

A prompting guide

Twelve rules that follow from the results above, in four groups. Each gives the requests to write, the mistakes to avoid, and the measurement behind the advice.

Figures come from one of two places. Those attributed to an experiment are from the full run against jev-1.13.0, and the sample size is given with each. Those described as measured live were run while this page was written, on sixty problems per condition. Both are reproducible from the seeds in the repository.

A difference is treated as real here only if it exceeds the model's own variation between identical requests, which the first experiment puts at 0.011. Anything smaller than roughly twice that is not a finding, and is not used as one below.

Conclusions

Every prediction, scored against the condition the plan set for it in advance; then the behaviours no experiment was designed to find; then the three decisions this run settles.

The plan noted that a high proportion correct would be mildly suspicious, since it would suggest the experiments had been set too easily. Predictions that turned out wrong are the most useful part of this section and are listed first.

The two predictions the plan singled out

Before any data, the plan named one result it expected to be the biggest surprise and one it expected to matter most. They came out differently.

Expected surprise — did not happen

That asking many questions at once would turn out to degrade the later ones, showing the parallel evaluation to be less complete than advertised.

It does not degrade them. Accuracy at question 200 matched accuracy at question 1 to within the measurement. The architecture behaves as described.

Expected to matter most — and it does

That if confidence fails to track ignorance, the recommended way of deploying the model — act when confident, escalate when not — stops working, however good the accuracy figures look.

By the measure the plan named this holds: the mean separation is 0.199, below the 0.30 set for a usable signal. But a mean is the wrong statistic for a quantity that sits at exactly 1.00 on 86% of answerable states. Ranked instead, confidence on a choice question separates every kind of unanswerable state, at 0.893 to 0.992. The deployment pattern works — at two gates rather than one, and only if you ask as a choice. The abstention section above has the detail.

Behaviour no experiment was designed to find

Three patterns appeared across every experiment that measured them.

Probabilities are quantised to two decimal places. All 3,194,813 probabilities seen in the batching experiment fall exactly on a hundredths grid. No calibration measurement here can resolve anything finer than 0.01.

Probabilities cluster rather than spread. The value carrying the decision was exactly 1.0 on 52 percent of 836,700 answers in that same experiment.

Confidence is not the largest probability. The two differ on 47.1 percent of choice and score answers, by 0.09 on average and by as much as 0.37. The vendor documents neither quantity's definition, and the difference is large enough to be a usable signal in its own right.

Every recorded anomaly

    What this changes