Does the AI believe the witness?
Hidden credibility judgments inside a language model reading legal statements. What we looked for, how we looked, and what we found, in plain English.
When an AI system reads a witness statement, does it form a view about whether the witness is believable, a view it never states out loud?
This matters because AI tools are moving into legal work: document review, disclosure exercises, case summarisation, chronologies. If the software carrying out those tasks holds an unstated opinion about who is credible, and that opinion is influenced by how people speak rather than what they say, then a discrimination risk has entered the workflow, and it is invisible to everyone using the tool.
You cannot answer this question by asking the AI. You can only answer it by looking inside. That is possible with “open-weights” models, where the internal workings are inspectable; it is impossible with closed products. So we looked inside one.
The caveats are below. One model, synthetic statements — but the effect is real and the controls held. Every number and quotation on this page comes from the experiment logs.
Three ideas are enough to follow everything
We can watch the model think
As a language model reads text, it builds an internal pattern of activity: millions of numbers representing what it currently “makes of” the text. With an open model, we can record that pattern at any point.
We found the model’s “credibility dial”
We wrote 200 pairs of witness statements describing exactly the same events, in the same confident tone, but in one version the facts don’t add up. The average difference in internal activity between the two groups acts like a dial: a single internal setting tracking “credible ↔ not credible.”
We can turn the dial by hand
While the model read a fresh, neutral statement, we reached into its internal activity mid-thought and nudged it along the dial, then asked it to rate the witness’s reliability from 1 to 10. If the score moves when we move the dial, the dial is doing work. It’s part of how the model judges.
Both versions describe the same events in the same confident tone. The only difference is whether the account adds up. The highlighted phrases are the parts that change.
The model detected, internally, whether a witness’s account was self-consistent, and it did so perfectly, separating the two versions on every held-out test statement.
Findings like these are easy to fake by accident
So the experiment was built with tripwires. Each one had the power to kill the result, and a “no result” was designed to be a publishable result.
Placebo nudges
Alongside the real dial, we pushed the model equally hard in eight randomly chosen internal directions. If a random shove moves the score as much as the credibility dial does, the dial means nothing.
Two independent readings
We scored the model’s written answers, and separately measured, before it wrote anything, how strongly it leaned toward “Yes” vs “No” on “should this account be relied upon?” Both had to agree.
Broken answers discarded
Push any model too hard and it stops writing sense. An incoherent answer is not evidence of anything, so every answer was screened and the discard count is reported.
Everything is logged
Every answer, at every dial setting, is saved and published. The reports are generated automatically and must show the placebo results next to the real ones.
The credibility dial is real, it steers the model’s judgment, and it isn’t about tone
With no interference, the model rated our neutral test statements at 5.5 out of 10 on average: a sensible “middling” answer with balanced reasoning. Then we turned the dial.
View the data as a table
| Steering strength | Credibility dial | Placebo (avg of 8) |
|---|
The score slid smoothly with the size of the nudge, in the right direction, at every step, and at these settings every one of the model’s answers stayed coherent. The model’s reasoning changed to match: nudged down, it wrote things like “the statement is vague… susceptible to memory error” about text it had rated as balanced moments before. It wasn’t blurting a different number; its assessment of the same words changed.
The placebo test separates signal from noise. The credibility dial swung the score by about 7 points; the strongest of the eight placebo directions managed about 3, and most managed 1 or less. The independent “Yes/No leaning” measurement agreed: the real dial moved it 2.3× more than the strongest placebo, and more than every placebo without exception, on both ways of reading the answer.
An earlier version of this experiment used statements that differed in tone (hedging, “I’m not sure”) rather than fact-consistency, and found the same steering effect. The result held across both definitions of credibility, and the fact-based dial steered more strongly, not less.
When we pushed the dial hard, the model’s written answers stayed measured on the surface right up until they broke, but its scores had already swung. The written text and the internal opinion can come apart.
Hesitant speech is internally marked down, even when the facts are identical
This is the finding with direct discrimination consequences — and the final run changed the picture. We took statements with identical facts and rewrote only the voice.
View the data as a table
| Voice | Mean position on dial | 95% CI |
|---|
Summary
Hesitant speech was marked down against both dials: the tone-based one from the earlier run and the fact-based one from the final run. However the model construes credibility, “um, like, you know” pushes a statement toward “not credible” with the facts held constant.
Regional dialect was marked down against the tone-based dial, but not significantly against the fact-based one. The dialect penalty is attached to the model’s tone-reading of credibility. That weakens the dialect finding from our earlier draft.
Translated-sounding English was never penalised. Against the fact-based dial it scored slightly above formal English, on both runs.
Terse statements were marked down against the fact-based dial. Short, bare accounts offer little that can be cross-checked, and the consistency dial appears to read that thinness as a credibility deficit.
Hesitancy is not evenly distributed across society. It tracks nerves, age, trauma, neurodivergence, and unfamiliarity with formal settings: precisely the witnesses a fair process is supposed to protect. If a legal AI tool summarises, ranks or triages witness evidence, this internal markdown can shape outcomes without anyone having decided it should.
The “voice” of a statement is stored largely separately in the model from the credibility dial, but a component of it leaks onto the dial. The model doesn’t file “hesitant = liar”; features of hesitant speech overlap with features it treats as marks of unreliability. The discriminatory effect is the same either way.
A bad first impression sticks to the person
The final experiment mimics a real litigation file: multiple documents about the same person, read in one sitting. Document A is a prior assessment of a named person. Document B, word-for-word identical in every test, is a neutral statement by that person. We asked the model to rate Document B alone.
Choose what the model reads before the identical statement:
Same words; a score of 1.1 versus 4.6, depending entirely on what the model read about the person beforehand. And notice: only the bad reference had power; the good reference didn’t lift the score at all. This is a taint effect, not a symmetrical halo. Prior doubt sticks; prior praise doesn’t.
The final run answered the question our earlier draft left open: is the taint attached to the person, or does a bad document simply sour the mood of the whole file? We re-ran the experiment with Document A describing a different, unrelated person. The taint did not transfer: a damning assessment of someone else left the witness’s statement barely changed (3.9, against 4.6 with no Document A at all).
A glowing reference about someone else slightly lowered the witness’s score (3.05), a contrast effect, as if the witness suffered by comparison. Consistent with the model judging people relative to one another rather than each on their own terms.
Does it hold on a bigger model? A mixed answer
We repeated all three experiments, unchanged, on a model twice the size: 14 billion parameters, same statements, same dials and controls. The published expectation was that the effects would get cleaner at larger scale. That is not what we found.
The steering results didn’t transfer, and we can see why. The larger model is more confident by default (baseline 7.5/10, not 5.5, leaving no headroom to push scores up), and it separates credible from non-credible statements perfectly at every layer, so our rule for choosing where to intervene had nothing to distinguish them and picked a layer that steers messily. The evidence that this is a method problem, not a real collapse: reading the model’s Yes/No leaning directly and steering toward “not credible” produced a strong, orderly, correct effect. The dial is present and causal in the larger model; our procedure for pushing it was calibrated on the smaller one.
The persistence finding changed. The “bad first impression” effect was much weaker (about half a point, against nearly three-and-a-half), and the larger model showed a contrast effect the smaller one didn’t: a bad reference about a different person made our witness look more credible. The larger model appears to grade people against each other rather than each on their own terms. Not obviously safer: a witness’s assessed reliability now depends partly on who else appears in the bundle.
| 7-billion model | 14-billion model | |
|---|---|---|
| Dial separates credible from non-credible internally | Perfectly | Perfectly |
| Steering moves the judgment cleanly and specifically | Yes · 2.3× the strongest placebo | Inconclusive · direction is causal, but the steering procedure didn’t transfer; re-run needed |
| Hesitant speech marked down | Yes | Not established · rode on the same untuned layer |
| Bad first impression sticks to the person | Strongly · ≈3.4 points | Weakly · ≈0.5 pt, replaced by a comparative/contrast effect |
Results on one model size cannot be assumed to hold on another. A vendor’s assurance based on testing one model tells you little about the model actually in production.
Caveats
- Small wording clues may remain. Expressing a contradiction requires some words to differ (“described the same events” vs “described different events”). The model separated the pairs perfectly, so we can’t tell how close to its limits we pushed it. A future version can make the only difference numeric, though the steering and cross-person results don’t depend on this.
- One model, synthetic statements, compressed weights. Everything was measured on open models run in a memory-saving compressed mode, using statements we wrote ourselves. No real case data was used at any point, deliberately.
- The placebo comparison, while passed, has limited resolution. The real dial beat all eight placebo directions on both ways of reading the answer, but “beat all eight” is a coarser statistical guarantee than we’d like. More placebo runs would tighten it; nothing we saw suggests the conclusion would change.
- Synthetic voices are not real sociolects. Our dialect and hesitancy rewrites are stylised. A study using transcripts of real (consented, anonymised) testimony is the proper next step before drawing quantitative conclusions about any actual community.
What this means for legal practice, today
Stated reasoning is not the assessment
The two can come apart. Procurement questions like “does the tool make credibility judgments?” cannot be answered by inspecting outputs, and a vendor cannot honestly answer “no” without inspecting internals.
Hesitant speech is a discrimination risk
Tools that summarise or triage testimonial material should be assumed, until shown otherwise, to internally discount hesitant speech: the speech of nervous, young, traumatised or neurodivergent witnesses. Relevant to Equality Act 2010 duties and procedural fairness.
The model keeps a file on the person
What an AI reads about a named individual persistently changes its assessment of that individual’s other statements, and negative material is far stickier than positive. Bundle composition and ordering are now fairness-relevant decisions.
Only inspectable models can be audited this way
Every measurement here required access to the model’s internals. Closed commercial models cannot currently be examined for any of these properties by their customers.