AI evidence extraction

“ChatGPT seems too good to be true”: College students’ use and perceptions of generative AI

Clare Baek, Tamara Tate, Mark Warschauer · Computers and Education Artificial Intelligence · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

8/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

9.99

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1016/j.caeai.2024.100294

Methodology & findings

Study design

Mixed methods study combining regression analysis with thematic analysis and natural language processing of open-ended survey responses from N=1001 U.S. college students

Sample

N = 1001

Primary method

Regression analysis (for predictors of ChatGPT use), thematic analysis, and natural language processing of open-ended responses

Main result

The study found that "gender, age, major, institution type, and institutional policy significantly influenced ChatGPT use for general, writing, and programming tasks." Additionally, "Students in their 30s–40s were more likely to use ChatGPT frequently than younger students" and "Non-native English speakers were more likely than native speakers to use ChatGPT frequently for writing, suggesting its potential as a support tool for language learners." Furthermore, "Higher-income students generally viewed ChatGPT more positively than their lower-income counterparts."

Reports effect sizes.

Research paradigm

Mixed methods (quantitative survey with regression analysis and qualitative thematic analysis with natural language processing)

Author conclusions

The authors conclude that "Our research underscores how technology can both empower and marginalize within educational settings; we advocate for equitable integration of AI in academic environments for diverse students."

Risk of bias

Selection bias: sample drawn from U.S. college students only; Self-report bias: survey-based data on ChatGPT use and perceptions; Potential response bias from students in institutions with different ChatGPT policies; Cross-sectional design limits causal inference; Self-selection bias in survey participation; Social desirability bias in self-reported ChatGPT use; Selection bias from college student population only; Potential institutional policy effects on honest reporting; Self-selection bias: Survey participants may be systematically different from non-respondents in their attitudes toward ChatGPT; Social desirability bias: Students may misreport their ChatGPT use if they perceive institutional disapproval; Institutional variation: Results may not generalize across diverse institutional types and policies; Temporal specificity: Cross-sectional design captures only one point in time; attitudes may have shifted since data collection

Limitations

The paper notes that "Thematic analysis and natural language processing of open-ended responses revealed varied attitudes towards ChatGPT, with some fearing institutional punishment for using ChatGPT and others confident in their appropriate use of ChatGPT," indicating potential selection bias in how students reported their experiences and perceptions.

Open questions raised

The paper identifies the need for equitable integration of AI in academic environments and highlights the importance of examining how technology affects diverse student populations differently, particularly regarding income, language background, and major field of study.
The study identifies the need for further research on equitable integration of AI across diverse student populations and the importance of institutional policies that support appropriate AI use across different demographic groups.
The abstract does not explicitly identify specific research gaps or future research directions beyond the general advocacy for equitable AI integration.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 64%

Explore related topics

Productivity Effects Research Productivity Survey