AI evidence extraction

Comparative analysis of text readability and writing styles in AI-generated vs. Human-written academic abstracts

Yumei Zou, Florence Kuek, Kwan Hoong Ng, Xiaoli Cheng · PLoS ONE · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1371/journal.pone.0343163

Methodology & findings

Study design

Comparative empirical study using computational readability analysis and expert evaluation.

Sample

N = 300, 2 groups

Primary method

Non-parametric statistical methods using SPSS 27 software for quantitative data analysis. Readability and writing style metrics were computed using the Readability Scoring System, a computational tool.

Main result

The study revealed that "AI-generated abstracts exhibited significantly lower readability across eight metrics, indicating greater complexity and lower readability" and that "Interdisciplinary comparisons revealed non-significant differences across nine readability metrics, highlighting AI's potential to mimic natural writing. However, it still faces challenges in generating lexically diverse content."

Reports effect sizes.

Research paradigm

Positivist/empiricist (quantitative measurement-based)

Author conclusions

The authors conclude that "These results underscored the current limitations of AI in generating readable and human-like abstracts, especially in technical fields." The study demonstrates that while AI shows some potential to mimic natural writing patterns, it struggles with lexical diversity and overall readability compared to human-written academic abstracts.

Risk of bias

Selection bias (abstracts limited to high-impact journals in linguistics and computer science only); potential algorithmic bias in the Readability Scoring System; limited discipline representation; no information on inter-rater reliability for expert evaluation; Selection bias: Abstracts limited to high-impact journals in linguistics and computer science may not represent broader academic disciplines; Source bias: AI abstracts generated from the same corpus as human abstracts may introduce systematic differences in quality; Evaluator bias: Expert evaluation methodology and inter-rater reliability not specified in abstract; Tool bias: Single computational tool (Readability Scoring System) used for assessment without validation details; Selection bias: abstracts sourced only from high-impact journals in two specific disciplines (linguistics and computer science), may not represent broader academic writing; Potential confounding: different AI tools (Kimi, ChatGPT, DeepSeek) used without apparent stratification or control; Expert evaluation methodology not detailed, risking subjective assessment bias

Limitations

The abstract states that "These results underscored the current limitations of AI in generating readable and human-like abstracts, especially in technical fields." However, specific methodological limitations are not explicitly detailed in the provided abstract text.

Open questions raised

The authors implicitly identify the need for further research on improving AI-generated abstract readability and lexical diversity, particularly for technical and discipline-specific fields. The non-significant differences across nine readability metrics suggest that interdisciplinary variations warrant further investigation.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 62%

Explore related topics

AI Manuscript Writing Academic Writing Empirical Study