
Abstract
A retrieval-augmented system succeeds only when the document it supplies beats what the model already believes. We measure which one wins, at the level of the token the model would actually emit, over 864 forward passes: six open models from 124M to 1.5B parameters, 24 counterfactual probes, three phrasings, with and without a contradicting document. Restricting to the 304 cases where the model demonstrably held the fact beforehand, the document is adopted 66.8% of the time and the model keeps its own answer 27.3%. Which model defers is not predicted by size. Holding the fact rises monotonically with parameter count, from 25.0% to 98.6%, but adoption does not: TinyLlama-1.1B holds the fact 97.2% of the time and still adopts the document least of the six, at 38.6%, while Qwen2.5-0.5B, less than half its size, adopts 91.8%. Wording moves the result nearly as much as the model does: over all probes the rate spreads by a median of 47.9 percentage points across three paraphrases, and restricted to conflicts, where the spread measures deference rather than recall, it still reaches 57.6. We also show why this has to be read at the rank: the in-context answer carries more than 0.10 of the probability mass in 332 conditions and is still not what the model says in 69 of them, 20.8%. That gap is not hypothetical. It is how an earlier version of this work reached a published conclusion that the measurement did not support, which we correct here. All code, probes and measurements are public.
Introduction
A retrieval-augmented system puts a document in front of a language model and expects the model to use it. When the document contradicts what the model learned in training, one of the two has to give, and which one gives is the question this paper measures. It is not hypothetical: a clinical assistant reading a withdrawal notice, or a coding assistant reading a deprecation note, is useful only if the document wins.

Figure 1. Parametric memory and retrieved context are two replicas of one fact, with no protocol to settle which is current.
The measurement is easy to get wrong, and we got it wrong. An earlier version of this work concluded that models resist a corrective document, on the strength of the probability the model assigned to the expected answer. That quantity can be small while the model would nonetheless answer correctly, because the mass is spread over a vocabulary of tens of thousands of tokens and the expected answer needs only to be ranked first, not to be large. Re-running the same probes with the rank recorded showed the conclusion did not hold. This paper reports the measurement done properly, and the size of the gap we fell into is itself one of the results.
Two changes make it proper. First, we record what the model would actually say, the top-ranked continuation, alongside the probability. Second, we use counterfactual probes rather than real-world updates. A real-world update creates a conflict only for a model whose training data predates it, so across models of different vintages the conflict is present for some and absent for others, and a probability-only readout cannot tell those two cases apart. A counterfactual document asserts something every model is confident is false, so the conflict is present by construction for all of them. This is the standard device in the knowledge-conflict literature [1], [2].
Contributions
- A rank-level measurement of parametric and context conflict over 864 forward passes, reported on the 304 probe-phrasing pairs where the model demonstrably held the fact and a conflict therefore existed.
- The finding that deference to a document is not a function of scale. Holding the fact rises monotonically with parameter count across the six models; adopting the document does not, and the 1.1B model that holds the fact most reliably defers least.
- Evidence that the question’s wording moves the outcome nearly as much as the choice of model, by up to 57.6 percentage points of deference alone, and a separation of that effect from the larger one the wording has on whether the model holds the fact at all.
- Evidence that a probability-only readout cannot support a claim about what a model answers, and a correction of our own earlier report on exactly that ground.
Method
Probes. Each probe is a country whose capital no model of this size should get wrong, paired with a counterfactual that is a real, prominent city of the same country: France and Lyon, Japan and Osaka, Egypt and Alexandria, and 21 others. Using a real city keeps the substitution plausible rather than nonsensical. Holding the fact type fixed across all 24 probes means that a difference between probes is not a difference between kinds of question. The document asserts the substitution as a constitutional relocation and names the new capital twice.
Phrasings. Each probe is asked three ways, differing in wording but not in what is asked. A quantity that moves across the three is a property of the sentence rather than of the model’s knowledge, and reporting the spread is what lets a reader tell those apart.
Conditions. Every probe and phrasing is run twice: once with the question alone, which measures what the model believes, and once with the contradicting document prepended, which measures what it does about the conflict. The first condition is not a control in name only. It is what identifies the subset in which a conflict exists at all, and every headline number in this paper is restricted to that subset.

Figure 2. Every probe runs twice. The first run is what makes the second interpretable.
Models. Six open models spanning an order of magnitude: GPT-2 at 124M, 355M and 774M, Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct, and TinyLlama-1.1B-Chat. All run in float32 on CPU, so the numbers reproduce on ordinary hardware rather than depending on a particular accelerator’s arithmetic.
What is recorded. For a prompt the model defines a distribution over the vocabulary. Writing t for the first token of the true answer and c for the first token of the counterfactual, we record the probability of each, the rank of each, and the arg-max over the vocabulary. Rank determines the answer under greedy decoding; probability is what the earlier report used alone.
That distribution is the softmax over the final layer’s output, the standard construction for a transformer decoder [3]. One consequence of training by maximum likelihood matters here: such a model spreads mass over everything the context makes plausible rather than committing to a single continuation. That is precisely why a modest probability does not mean the token is not the answer, and it is the mechanism behind the gap reported below.
Answers are compared at the first token. Tokeniser boundaries were checked for every probe, phrasing and model: all 144 country and answer pairs tokenise cleanly across the three tokeniser families used here, so no probe silently measures a merged token.
Results
Holding a fact scales; deferring to a document does not
Whether a model holds the fact at all rises monotonically with parameter count, from 25.0% at 124M to 98.6% at 1.5B, with no inversion anywhere in the ordering. This is the expected result and serves mainly as a check that the probes measure what they are meant to.

Holding a fact scales. Deferring to a document does not.
What the model then does with a contradicting document follows no such pattern. Across the 304 genuine conflicts the document is adopted 66.8% of the time and the model keeps its own answer 27.3%, but the per-model rates run from 38.6% to 91.8% in an order that has little to do with size. The clearest case is TinyLlama-1.1B: it holds the fact 97.2% of the time, the second-highest of the six, and yet adopts the document in only 38.6% of its 70 conflicts, keeping its own answer in 57.1%. Qwen2.5-0.5B, less than half its size, adopts 91.8%. The two Qwen models, at 494M and 1540M, are the two most deferential; the three GPT-2 models and TinyLlama are not ordered by size among themselves.
The natural reading is that deference is a property of how a model was trained rather than of how large it is. We cannot separate those factors here: the six models differ in family, in instruction tuning and in scale at once, and the design does not let us hold two fixed while varying the third. What the data do establish is the negative claim, which is the one that matters for practice: scale alone does not buy a model that uses the document in front of it. Qwen2.5-1.5B is the only model in the set that never keeps its own answer once contradicted, at 0.0%.
Wording moves the answer, and moves two things at once
How the question is worded changes the outcome substantially, and separating what it changes is worth doing, because the two mechanisms carry different advice.

Three paraphrases of one question, and the spread between them.
The coarse quantity is the rate at which the counterfactual is top-ranked over all 24 probes of a wording, with no restriction to conflicts. Across the three wordings it has a median spread of 47.9 percentage points and a maximum of 87.5, and that maximum is Qwen2.5-0.5B, which ranks the counterfactual first 100.0% of the time under its most favourable wording and 12.5% under its least.
That figure is not a measure of deference, and reading it as one is a mistake of the same shape as the one corrected below. Under its third wording Qwen2.5-0.5B holds the fact in 1 of its 24 probes rather than in all 24, so there is almost no conflict left for it to resolve; in the single conflict that remains it does adopt the document. Restricted to conflicts, its adoption rate across the three wordings is 16.7 points wide, not 87.5. Most of what the wording moved was whether the model knew the fact, not whether it deferred.
Deference does move, and by less. The adoption spread is defined only for the 4 models whose every wording leaves at least one conflict, and over those it has a median of 35.1 percentage points and a maximum of 57.6. The maximum is TinyLlama-1.1B, which adopts the document 17.4% of the time under one wording and 75.0% under another, on conflicts in every case.
Either way the practical conclusion stands, and the second version states it more precisely: a spread of 57.6 points in deference alone is larger than the gap between most pairs of models. A single-phrasing result about knowledge conflict, including any single-phrasing result in the earlier version of this work, reports a property of the sentence at least as much as a property of the model. It also has to say which rate it is reporting, since the two differ here by a factor of five on the same model.
Probability is not an answer

More than 0.10 of the mass, and still not what the model says.
The in-context answer carries more than 0.10 of the probability mass in 332 of the 432 conditions with the document present, and in 69 of those, 20.8%, it is still not the token the model ranks first. A reader given only the probability would conclude the document had landed; a reader given the rank would see that the model would say something else.
The converse also holds and is the trap we fell into. A small probability for the expected answer is compatible with that answer being ranked first, because the mass is distributed over a large vocabulary. Neither direction of inference from probability to answer is safe. Recording the rank costs one comparison against the maximum, and it is the only quantity that corresponds to what a system deployed under greedy decoding will actually produce.
A correction to our earlier report
An earlier version of this work reported that across six models and three documented fact changes the conflict did not resolve, and that no model answered a drug-withdrawal probe correctly under any of three phrasings. That claim was wrong, and the manner of its wrongness is the reason this paper exists.

Figure 3. Two defects, one conclusion that did not hold.
The script that produced it recorded, for each model and phrasing, the surprisal of the expected answer with the document present. It recorded neither the rank of that answer nor the identity of the top-ranked token. A claim about what a model answers cannot be derived from that quantity, and when we re-ran the same probes with the rank recorded, three of the six models ranked the corrected answer first under two of the three phrasings.
A second defect was structural. The probes were real-world updates, so whether a model held the superseded fact depended on its training cutoff, and the six cutoffs differ. For the drug probe the event predates every model’s training data, so no model held the superseded fact and there was no conflict to resist. The large surprisal we reported was the question frame declining a participle, not a model defending a stale belief. Both defects are removed by the design above: counterfactual probes make the conflict independent of cutoff, and the no-document condition identifies the cases where the model actually held the fact.
We report this at length because the correction generalises. A probability-only readout is cheap to compute and easy to tabulate, and it will return a number for any probe whether or not that probe tests anything at all. Both defects would have been caught immediately by recording the rank, which is one comparison.
Related work
What the readout should be. Factual probing of language models has used the top-ranked continuation as its readout since Petroni et al. posed the question [4], and the quantity reported there is accuracy at rank one rather than a probability. This paper’s methodological claim is therefore not a new proposal but a return to the existing convention, and the correction above is a case of having departed from it. The reason the convention is the right one is that probability mass and rank come apart: the mass is divided over tens of thousands of tokens and over the several surface forms of one answer, so a small probability is compatible with rank one and a large one with rank two. Min et al. make the general form of the point for in-context learning [5], where the quantity a prompt moves is not the quantity the model acts on.
What the wording does. That a paraphrase of a probe changes what a model appears to know is established. Jiang et al. show that the prompt used to query a model changes how much knowledge is extracted from it [6], and build a prompt archive precisely because no single phrasing is representative. Elazar et al. measure consistency directly over 328 paraphrases of 38 relations and find pretrained models frequently inconsistent under paraphrase [7]. Our contribution there is narrower and, we think, the part that matters for a retrieval system: the wording moves two things at once, whether the model holds the fact and whether it defers once it does, and the first effect is much the larger.
What scale is expected to buy. The expectation that a capability improves with parameter count is the default reading of the scaling literature [8], and it holds here for one of the two quantities we measure. It does not hold for the other. The natural alternative explanation, which our design cannot isolate, is that deference is set by post-training rather than by size: instruction tuning optimises a model to follow what it is given [9], and the two most deferential models are the two instruction-tuned ones. We state this as the hypothesis it is.
Whether a model uses the context it is given. That supplying a document does not guarantee the model uses it is itself a measured phenomenon. Liu et al. show that retrieved evidence is used unevenly depending on where it sits in the context [10]. Both that work and benchmark suites for retrieval-augmented generation treat use of the context as something to be verified rather than assumed, which is the position this paper takes and the reason it conditions on a conflict existing.
Knowledge conflict. Conflict between parametric memory and supplied context has been studied directly. Longpre et al. construct conflicts by substituting entities in the retrieved passage, the device used here, and find models often prefer their parametric answer [1]. Xie et al. elicit a model’s parametric belief first and then present conflicting evidence, reporting that receptiveness depends on how coherent that evidence is [2]; our finding that deference is not ordered by scale is consistent with their account of receptiveness as a trained disposition. The conflict matters because retrieval is standard practice, and because editing the parametric memory directly does not remove the possibility that weights and context disagree at inference.
Framing the failure as a synchronisation problem is deliberate. Parametric memory and retrieved context are two replicas of one fact, and the failure is that they disagree with no protocol to settle which is current, the question a distributed system answers with a synchronisation protocol. A natural response is to gate a representation on an auxiliary scalar carrying recency or provenance. We do not evaluate such a gate here; an earlier version of this work proposed one, and we have removed it because nothing in our measurements tests it.
Limitations
The models are small. The largest is 1.5B parameters, and behaviour at 7B and above may differ. The non-monotonicity we report is evidence about this range.
The design confounds family, instruction tuning and scale. We can show that adoption is not ordered by parameter count, which is a negative claim and is supported, but we cannot attribute the variation to any one factor.
The probes are homogeneous by construction. Every one is a national capital, which controls for question type at the cost of telling us nothing about numeric, temporal or causal facts.
Counterfactual probes buy freedom from the cutoff confound at a price: the document asserts something false, and a model trained to resist false premises may refuse it for reasons unrelated to the recency of its knowledge. A deployed retrieval system meets the opposite case, where the document is true and the parametric memory is stale. The two share a mechanism but are not identical, and a model that looks admirably deferential here may be deferential to misinformation as well.
Answers are compared at the first token, sufficient here because the candidate cities differ there, but not general. The measurement is of a single-turn next-token distribution under greedy reading; systems that sample, reason over several steps, or are prompted to weigh sources explicitly may behave differently.
Conclusion
Across 304 genuine conflicts in six open models, a contradicting document is adopted 66.8% of the time. Whether a given model adopts it is not predicted by size: holding the fact rises monotonically with parameter count while deference does not, and the model that holds facts second-most reliably defers least of all. The wording of the question moves the rate by up to 87.5 percentage points, comparable to the spread between models.

One document, two answers. Which one wins is not a function of size.
The methodological result is the one we would most like carried forward. The probability a model assigns to an answer and the answer it gives are different quantities, they disagree in 20.8% of the conditions measured here, and a published conclusion of ours rested on the difference. Rank is one comparison more expensive and is what a deployed system actually emits.
Future work. The natural extension is multimodal. When the contradicting evidence is an image rather than a passage the same question arises with no token rank to read the answer from, and the systems where this matters, image translation and zero-shot visual detection, where a model must lean on parametric knowledge because no corrective document exists, are exactly those for which the diagnostic is missing. A second direction is deployment: the accompanying demonstration runs a model in the browser, the same client-side setting as other real-time in-browser inference work, and a conflict diagnostic cheap enough to run there would put the measurement where the retrieval happens.
The poster

The poster, as presented. The full-resolution PDF is linked below.
Where the work is
Citation
Please cite this work as:
Thakur, Amey, and Sarvesh Talele. "The Knowledge Lifecycle of Large Language Models" (Nov 2026). https://vixra.org/abs/2611.0038.Or use the BibTex citation:
@article{thakur2026lifecycle,
title = "The Knowledge Lifecycle of Large Language Models",
author = "Thakur, Amey and Talele, Sarvesh",
year = "2026",
month = "Nov",
url = "https://vixra.org/abs/2611.0038"
}