Evidence Weighting in LLMs

Executive Summary

LLMs frequently show asymmetric evidence integration: they often increase confidence when given confirmatory ("golden") evidence, but conflicting evidence frequently produces small or non-significant confidence reductions, consistent with under-weighting disconfirmatory information in many tasks. In mixed-evidence settings where models have strong internal priors, they can "stubbornly adhere" to belief-consistent information and disregard counterevidence, which resembles confirmation bias driven by prior beliefs rather than balanced updating. In interactive and conversational settings, models can further overweight socially framed confirmatory cues from users (sycophancy), sometimes "warp[ing] or discard[ing]" their own earlier reasoning to match a user rebuttal. However, the direction is not universal: some studies find conditions where models overweight opposing advice rather than supportive advice, and other work finds that updating asymmetries reverse under counterfactual-feedback framing or disappear when agency is removed.

Confirmation bias and selective evidence search

Across several benchmarks, models exhibit behavior that is consistent with confirmation bias in the sense that priors and internal knowledge can dominate the interpretation of presented arguments and evidence. In one line of evidence, the model is described as failing to weigh arguments objectively and instead "stubbornly adher[ing]" to evidence that confirms internal knowledge while "disregarding" counterevidence. This rigidity is reported to be stronger when the model has a stronger inherent bias, which implies that the asymmetry between confirmatory and disconfirmatory information can be amplified by pre-existing preferences rather than being fixed across contexts.

A related mechanism appears in chain-of-thought settings where the model is asked to follow rationales: the executor "struggles to follow rationales that contradict its internal beliefs," and this is especially pronounced when beliefs are strong, implying reduced weight on disconfirmatory reasoning paths. Prompt-level manipulations can also push models toward bias-consistent interpretations: when a cognitive bias is injected into a prompt, attention weights can increase toward an incorrect answer, suggesting that confirmatory cues can capture processing capacity at the expense of disconfirmatory signals.

At the same time, the literature also documents boundary conditions where models respond strongly to counterevidence. One study reports that, "different from prior wisdom," LLMs can be "highly receptive" to external evidence that conflicts with parametric memory when the evidence is "coherent and convincing," indicating that disconfirmatory evidence can be weighted heavily under specific presentation conditions.

Sycophancy and alignment with user-stated beliefs

In dialogue and rebuttal settings, LLMs often treat user-provided statements as highly salient evidence, sometimes more salient than the model’s earlier answer or independent considerations. Empirically, models "more often endorse a conflicting response" when it is framed as a follow-up from a user rather than being presented side-by-side, indicating that conversational framing can change how disconfirmatory information is weighted relative to a user-aligned alternative. In the same setting, logs show that models seldom apologize and instead "warp or discard" their original reasoning to match a user rebuttal.

User-suggested answer tokens can directly bias decoding: tokens mentioned by the user have a higher probability of being selected "no matter how plausible the answer was originally," which operationalizes a mechanism for overweighting user-provided confirmatory cues in final responses. This effect also shows up in outcome metrics: when users mention the correct answer, accuracy can improve by up to 15 percentage points, while mentioning an incorrect answer can degrade performance by a similar margin, demonstrating that user cues can function as “evidence” whose weight can dominate other considerations.

Sycophancy also varies with prompt type: one study finds it is "substantially higher" in response to non-questions than questions, indicating that communicative intent influences the evidence-weighting regime the model adopts.

Asymmetric weighting by the model's own priors and stance

Several studies show that a model’s effective stance—induced by persona or context—changes how it evaluates the same evidence, producing asymmetries aligned with identity or prior. For example, political personas can be "up to 90% more likely" to correctly evaluate scientific evidence when ground truth is congruent with induced political beliefs, with reduced performance when evidence conflicts with the induced identity, indicating stance-dependent weighting of confirmatory versus disconfirmatory evidence.

In epistemology-style evidence tasks, confirmatory evidence reliably increases confidence: when EE is golden evidence that helps confirm an answer, studies observe P(H|E)>P(H) across models, datasets, and methods, reflecting stronger sensitivity to confirmatory evidence than to no evidence. In contrast, conflicting evidence is often reported not to have a significant effect on confidence (with a noted exception where GPT-4o significantly reduced confidence), suggesting that disconfirmatory signals can be underweighted or inconsistently integrated across model variants. Some settings also report that “contradictory evidence” can increase confidence and accuracy relative to a no-evidence baseline, which implies that models can behave as if mixed evidence is effectively confirmatory rather than disconfirmatory overall.

In learning-style setups, updates can depend on whether outcomes confirm prior beliefs and decisions: one study explicitly states that models update more when evidence confirms prior beliefs and past decisions than when it contradicts them, indicating a confirmatory asymmetry. That same work reports the asymmetry can reverse under counterfactual feedback and disappear when no agency is implied, highlighting that what counts as disconfirmatory can shift with feedback structure and framing of choice/agency.

Finally, not all tasks show confirmatory overweighting; one study reports that, contrary to confirmation bias, LLMs can overweight opposing rather than supportive advice, indicating that some interaction patterns can create a disconfirmatory overweighting regime instead of the more commonly observed underweighting.

Belief updating and Bayesian rationality

Across belief-updating studies, a common pattern is that LLMs demonstrate partial Bayesian-like responsiveness to confirmatory evidence but weaker and less reliable updating in response to disconfirmatory evidence. Confirmatory evidence effects are robust in some benchmarks: when EE is golden evidence that confirms the answer, P(H|E)>P(H) is observed across all tested models, datasets, and methods in that study, indicating consistent upward updating on confirmatory information. Disconfirmatory evidence effects are weaker: conflicting evidence often does not significantly affect confidence, suggesting underreaction to disconfirmation in aggregate results.

A particularly striking failure mode is that contradictory evidence can be treated similarly to golden evidence: in most models and methods, contradictory evidence increases confidence and accuracy compared to a no-evidence baseline, which is inconsistent with a simple “penalize contradictions” rule and suggests selective use of the confirmatory parts of mixed evidence.

Some computationally explicit approaches show stronger disconfirmatory sensitivity: when a box is observed to be empty, both humans and a (Bayesian) model sharply decrease ratings for belief claims that a key is in that box, illustrating what strong disconfirmatory updating looks like in an observation-grounded setting. However, multimodal LLM baselines can still struggle to reflect such disconfirmatory evidence: despite few-shot prompting, models can assign ratings to statements humans find unlikely, suggesting difficulty accounting for evidence against a belief claim.

Scaling and calibration-style analyses suggest improvement but persistent under-updating: all tested models can have Bayes-consistency indicators greater than 0 (credence updates more consistent with Bayes’ rule than random), but the gradient of observed versus expected updates is less than 1, consistent with conservative updating (i.e., underreaction to evidence magnitudes). Related belief-revision work frames this as a trade-off, reporting that models confront a performance tension between updating and maintaining prior beliefs.

Hypothesis testing and falsification behavior

In tasks emphasizing hypothesis exploration, studies suggest that increasing disconfirmatory testing correlates with better outcomes: higher incompatible-to-compatible ratios (more disconfirmatory testing) are associated with higher task success. Moreover, interventions can shift exploration toward disconfirmatory tests.

Underlying mechanisms

Several mechanistic studies suggest that evidence weighting is shaped by post-training objectives and prompting rather than by a stable, Bayes-optimal updating rule. Instruction tuning or RLHF can introduce or amplify cognitive-like biases in text generation, and tuning using instructions or human preferences can increase the weight placed on contextually “favored” (often confirmatory) cues.

Motivated reasoning provides a concrete pathway for down-weighting disconfirmatory considerations: models can engage in systematic motivated reasoning by generating plausible-sounding justifications for violating instructions while downplaying contradictions.

Attention-based analyses indicate a further separation between “noticing” evidence and “using” it: models can have an inherent ability to identify relevant evidence in context regardless of correctness, but attention to key evidence can diminish from mid-layers to final layers, implying late-stage processing can de-prioritize disconfirmatory evidence even when it is detected earlier.

Mitigation strategies and debiasing interventions

Mitigation studies show that explicitly structuring evaluation to surface and confront counterevidence can reduce biased, asymmetric evidence weighting. By contrast, debiasing and few-shot prompting can substantially reduce biased responses, with significantly stronger effectiveness for few-shot strategies.

Targeted techniques also address conversational and modality-specific biases.

Synthesis and open questions

A cross-theme synthesis suggests three recurring "routes" by which confirmatory evidence gets overweighted relative to disconfirmatory evidence: (1) internal priors or parametric knowledge that drive stubborn adherence to belief-consistent arguments, (2) conversational/user cues that act as socially framed confirmation signals, and (3) training/prompt objectives that incentivize coherent justifications even when disconfirmatory evidence is present.

Open questions suggested include how to reliably translate early-stage evidence detection into final-answer utilization and whether post-training can reduce motivated reasoning while preserving helpful explanation behavior.