AI evidence extraction

(Human) Attention Is (Still) All You Need: Human oversight makes AI-assisted social science reliable

Chen Zhu, Xiaolu Wang, Weilong Zhang · arXiv (Cornell University) · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

Methodology & findings

Study design

Pre-specified 2×4 factorial experiment crossing pipeline configuration (constrained HLER vs.

Sample

N = 280, 10 groups

Primary method

Two-sided Fisher's exact tests on dichotomous failure indicators for primary comparisons. Logistic regressions with dataset fixed effects fitted for pooled analyses across datasets. Cohen's κ used for inter-rater reliability (mean pairwise κ = 0.67). Majority rule (at least 2 of 3 reviewers) used for final classification on each dimension. Ablation analyzed by reporting cell proportions; interaction patterns treated as exploratory without formal interaction test due to small cell sizes (n=20 per condition).

Main result

The constrained HLER pipeline reduced critical failures from 72% to 16% compared to the unconstrained baseline. Specifically, "the unconstrained pipeline produced critical failures in 72% of runs. The same pipeline, with the architectural commitments imposed, failed in 16% (Fisher's exact p < 0.001)." The reliability gains were largest on datasets least familiar to the LLM training distribution, with the CMGPD-Liaoning historical dataset showing an 80-percentage-point reduction in failures compared to smaller gaps for contemporary health datasets.

Reports effect sizes and confidence intervals.

Research paradigm

Empiricist/Pragmatist - testing decision architecture effects on AI-assisted research reliability through controlled experimentation

Author conclusions

"Productive human-AI collaboration in empirical science is, in the end, a problem of decision design: not how to remove humans from the loop, but how to place human attention where it can most effectively reduce, expose, and contain failure." The authors conceptualize HLER as "a research harness rather than an autonomous AI scientist" and conclude that "the same underlying model produces a 72% failure rate under one allocation of cognitive labour and a 16% failure rate under another. The variable that moves is not the model but the architecture surrounding it."

Risk of bias

Expert reviewer judgement bias - three reviewers evaluated outputs but inter-rater reliability was substantial (mean pairwise Cohen's κ = 0.67); Model-specific effects - results obtained with single model (Claude Sonnet 4.6); generalizability to other models uncertain; Dataset selection bias - four datasets chosen but do not represent full diversity of empirical social science; Proxy validity concern - PubMed literature prevalence used as exploratory proxy for dataset familiarity (θ_t) rather than direct measurement; Potential automation bias in unconstrained condition where system may appear fluent despite lacking grounding; Single underlying model (Claude Sonnet 4.6) limits generalizability to other LLM architectures; Expert reviewer judgement introduces unavoidable subjectivity despite majority-rule consensus; Dataset selection bias: four datasets chosen deliberately to vary in familiarity, but limited to health/demographic domains; Literature prevalence (PubMed searches) used as imperfect proxy for θ_t (training-distribution proximity), acknowledged as overweighting biomedical visibility; Single set of three expert reviewers (same reviewers evaluated all 280 runs) may introduce systematic reviewer bias; Operational definitions of 'failure' and 'feasibility' reflect researcher judgment and may not capture all relevant quality dimensions; Expert reviewer judgment in evaluation rubric could introduce subjective assessment bias despite inter-rater reliability measurement; Single underlying language model (Claude Sonnet 4.6) limits generalizability to other LLMs with different failure profiles; Literature prevalence (PubMed search counts) used as proxy for training-distribution familiarity is imperfect and may overweight biomedical visibility; Selection of datasets may not represent full diversity of empirical social science methodologies; Ablation study uses smaller cell sizes (n=20 per condition) which reduces statistical precision

Limitations

"First, the four datasets do not exhaust the diversity of empirical social science: experimental data, text-as-data, network data, and qualitative sources present challenges outside our evaluation
Second, results are obtained with a single underlying model (Claude Sonnet 4.6)
other models will have different failure profiles, and the specific rates we report are likely model-dependent, though we expect the qualitative pattern to generalise
Third, the deterministic-probabilistic boundary is not always sharp in practice: stages such as variable selection contain both elements
Fourth, the ablation uses smaller cell sizes than the main experiment, with 20 runs per condition, and its results should be read as indicative of the relative importance of design features rather than as precise interaction estimates
In particular, the complementarity pattern is exploratory and not powered for a definitive interaction test

Open questions raised

The authors identify the need for: (1) extension to experimental data, text-as-data, network data, and qualitative sources; (2) testing across multiple LLM models to assess generalizability beyond Claude Sonnet 4.6; (3) investigation of the deterministic-probabilistic boundary in practice, particularly for stages like variable selection; (4) powered interaction tests for complementarity between design features; (5) broader conceptual extension to human-led research workflows (noted as future work rather than empirical claim); (6) investigation of micro-behavior of human gatekeepers such as time spent, attention patterns, and disagreement among multiple PIs.
Extension to human-led research more broadly: authors suggest "many human-led research failures have a similar structural form: judgement, interpretation, or motivated reasoning enters stages that require deterministic discipline" and propose the deterministic-probabilistic distinction may apply beyond AI-assisted workflows. Future work should test whether the architectural principles generalize to experimental data, text-as-data, network data, and qualitative sources. Authors also note need for research on other LLM models beyond Claude Sonnet 4.6 and more granular behavioral measurement of human gatekeepers (time spent, attention patterns, disagreement among multiple PIs).
Extension to experimental data, text-as-data, network data, and qualitative sources which present challenges outside the evaluation scope
Evaluation with other language models beyond Claude Sonnet 4.6 to establish model-agnostic patterns
Sharper theoretical and empirical characterization of the deterministic-probabilistic boundary in practice
Higher-powered ablation studies with larger cell sizes to provide precise interaction estimates

Data: UK Biobank (UKB) - ~500,000 participants, biomedical and lifestyle data, publicly available; China Health and Nutrition Survey (CHNS) - ~30,000 individuals, longitudinal household panel since 1989, publicly available; China Health and Retirement Longitudinal Study (CHARLS) - ~20,000 adults aged 45+, publicly available; CMGPD-Liaoning (China Multi-Generational Panel Dataset-Liaoning panel) - historical demographic panel from Qing-dynasty population registers (~1749-1909), publicly available; UK Biobank (UKB): ~500,000 participants, biomedical and lifestyle data; China Health and Nutrition Survey (CHNS): ~30,000 individuals, longitudinal household panel since 1989; China Health and Retirement Longitudinal Study (CHARLS): ~20,000 adults aged 45+; CMGPD-Liaoning: Historical demographic panel from Qing-dynasty population registers (~1749-1909); UK Biobank (UKB; ~500,000 participants, biomedical and lifestyle data); China Health and Nutrition Survey (CHNS; ~30,000 individuals, longitudinal household panel since 1989); China Health and Retirement Longitudinal Study (CHARLS; ~20,000 adults aged 45+); CMGPD-Liaoning (China Multi-Generational Panel Dataset-Liaoning panel; historical demographic panel from Qing-dynasty population registers, ~1749-1909)Code: HLER implemented as modular multi-agent system in Python. All LLM calls and code executions logged. Paper states agents are 'model-agnostic at the agent interface.' However, no GitHub repository URL or code availability statement is explicitly provided in the text.; HLER (Human-in-the-Loop Economic Research) is described as a modular multi-agent system implemented in Python, but no GitHub repository or code URL is provided in the paper.Extracted from: pdfAgreement 49%

Explore related topics

Responsible AI Use Human-AI Collaboration Experimental