AI evidence extraction

The Verification Gap: Artificial Intelligence Adoption, Hallucination Awareness, and Verification Practices Among Early Career Medical Researchers in Pakistan

Mobeen Sajjad · medRxiv · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.64898/2026.05.28.26354373

Methodology & findings

Study design

Cross-sectional, self-administered anonymous online survey with 24 items across four domains: demographic and professional characteristics, AI adoption patterns, verification practices and hallucination awareness (including two behavioral vignettes), and attitudes and disclosure practices.

Sample

N = 373, 6 groups

Primary method

Python (pandas and scipy libraries) for data analysis. Frequencies and percentages calculated for categorical variables. Chi-square tests of independence assessed associations between demographic/training variables and key outcomes. P-value less than 0.05 considered statistically significant. No corrections for multiple comparisons applied given exploratory nature; results interpreted as hypothesis-generating rather than confirmatory.

Main result

The study found that despite near-universal AI adoption (99.7% reported using at least one AI tool, with 60.3% reporting daily use), "the majority (59.2%, n=221) reported typically not verifying outputs; only 1.3% (n=5) verified nearly all AI-generated content before use." Additionally, "40.2% (n=150) reported regularly using AI but having never heard that AI may generate fabricated scientific references." A statistically significant association was found between formal research training and consistent disclosure: "51.7% of trained researchers reported always disclosing, compared with 17.1% of untrained researchers (chi-square=48.43, df=1, p<0.001)."

Reports effect sizes.

Research paradigm

Positivist/empiricist with descriptive-analytical orientation

Author conclusions

"This cross-sectional survey suggests that early career medical researchers in Pakistan have adopted AI tools at a rate that substantially outpaces the development of verification habits, hallucination awareness, and disclosure practices. Formal research methodology training was significantly associated with consistent AI disclosure, though not with verification behavior -a dissociation that may inform the design of future educational interventions. These findings provide a quantitative basis for advocating the integration of AI literacy, including explicit training on hallucination risk and verification practice, into Pakistani medical education curricula and institutional research governance frameworks."

Risk of bias

Selection bias: Convenience sampling via WhatsApp and LinkedIn; Social desirability bias: Self-reported verification and disclosure practices; Temporal bias: Cross-sectional design prevents causal inference; Participation bias: Respondents likely more research-active and digitally engaged; Language/accessibility bias: Sample restricted to English-speakers and digitally-engaged professionals; Network bias: Tool rankings (Claude, ChatGPT, Perplexity) reflect investigator network characteristics; Procedural bias: Three survey questions added after initial piloting affecting data completeness; Selection bias from convenience sampling via WhatsApp and LinkedIn; Social desirability bias in self-reported verification and disclosure practices; Potential overrepresentation of research-active or digitally engaged professionals; Exclusion of non-English-speaking healthcare professionals; Investigator network characteristics affecting tool ranking (Claude prevalence); Non-representativeness of non-English-speaking healthcare professionals; Investigator network bias in AI tool ranking (Claude prevalence); Timing bias from three questions added after initial data collection

Limitations

The authors acknowledge several limitations: "Convenience sampling via WhatsApp and LinkedIn introduces selection bias
respondents may be more research-active or digitally engaged than the broader population of early career Pakistani medical professionals." Additionally, "Self-reported data are subject to social desirability bias, potentially overestimating verification and disclosure practices
The cross-sectional design precludes causal inference." Furthermore, "this study measured self-reported perceptions and behaviors, not directly observed research conduct
actual rates of verification and fabricated reference use may differ from those reported here."

Open questions raised

Systematic documentation of verification practices and hallucination awareness in LMIC contexts remains limited
Formal AI governance frameworks lacking in Pakistani medical institutions
Structured AI literacy training not established in most medical institutions in Pakistan
Need for prospective or observational methods to confirm relationship between awareness and actual verification behavior
Replication using probability-based sampling across broader LMIC contexts warranted
Dissociation between normative awareness (knowing disclosure required) and habitual verification practice requires further investigation

Extracted from: pdfAgreement 69%

Explore related topics

Hallucination Control Research Integrity Survey