AI evidence extraction

Characteristics, motivations and attitudes of students using ChatGPT and other language model-based chatbots in higher education

Vicente Morell‐Mengual, Olga Fernández‐García, Carmen Berenguer, Jéssica Ortega-Barón, María Dolores Gil‐Llario, Verónica Estruch‐García · Education and Information Technologies · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

7/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

22.51

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s10639-025-13650-1

Methodology & findings

Study design

Cross-sectional survey research using an ad hoc questionnaire distributed online via LimeSurvey platform (January-May 2024) to university students.

Sample

N = 974, 6 groups

Primary method

IBM SPSS Statistics 28.0.1.1 was used for all analyses. Frequencies and percentages were calculated for categorical variables. Comparisons between men and women were made using the chi-square test (χ²) with effect size estimated using Cramer's V index (>0.25=very strong association; >0.15=strong association; >0.10=moderate association; >0.05=weak association). Student's t-test was used to compare means between men and women for continuous variables (attitudes and perceptions) with effect size estimated using Cohen's d (values below 0.20=small effect; around 0.50=moderate effect; above 0.80=large effect). Statistical significance level was set at p<.05.

Main result

A total of 61.4% of participants reported using generative AI chatbots in their university studies, with "the majority of students used generated text as an initial reference to develop entirely new content or make significant modifications." The primary purposes for using chatbots were "to obtain information (79.3%) and generate ideas (61.6%)," and "the most commonly cited motivation was to improve understanding of complex concepts (63.6%)." Regarding attitudes, "many students acknowledge that chatbots facilitate understanding academic concepts," though "a major concern is the potential loss of the ability to differentiate between reliable and unreliable information."

Reports effect sizes and confidence intervals.

Research paradigm

Positivist/Quantitative empiricism

Author conclusions

"The widespread use of generative AI chatbots among university students suggests that governing bodies should reconsider the formal integration of these technologies into academic programs. These tools can be incorporated as complementary learning resources, provided that clear institutional policies and ethical guidelines are established." Furthermore, "a critical aspect is the digital literacy of students. Before the widespread implementation of these technologies, it is necessary to design and implement training programs that help students develop skills to critically evaluate information generated by chatbots, particularly in terms of originality and academic integrity."

Risk of bias

Selection bias: non-probabilistic convenience sample recruited through courses and social media; Social desirability bias: students may underreport unethical uses due to fear of academic repercussions; Sampling bias: overrepresentation of women (80.2%) and students from health sciences (50.4%) and social/legal sciences (44.8%); Lack of pre-testing: no preliminary test of the ad hoc questionnaire before formal data collection; Selection bias: Non-probabilistic convenience sampling through courses taught by authors and social media recruitment; Social desirability bias: Self-reported data on potentially sensitive behaviors (plagiarism, academic misconduct); Gender imbalance: 80.2% female participants, potentially masking gender trends observed in other studies; Disciplinary bias: Overrepresentation of health sciences (50.4%) and social/legal sciences (44.8%), underrepresentation of STEM fields where chatbot adoption may differ; No pilot testing or reliability measures reported for survey instrument; Selection bias: Non-probabilistic convenience sample recruited through professors' courses and social media, limiting representativeness; Gender imbalance: 80.2% female participants, though authors note this reflects Spanish higher education demographics; Social desirability bias: Self-report method vulnerable to underreporting of unethical chatbot use due to fear of academic repercussions; Sampling location bias: 99.3% public universities, 91.6% in-person modality, primarily health sciences (50.4%) and social/legal sciences (44.8%), limiting generalizability; Recall bias: Self-reported frequency and use patterns subject to inaccuracy; Disciplinary underrepresentation: Limited STEM representation may mask gender-based trends found in other studies

Limitations

The authors note that "the analyses were conducted based on a sample composed primarily of students from public universities in face-to-face learning modalities
This limitation could restrict the generalizability of the findings." Additionally, "the use of self-reports as an evaluation method introduces a potential source of bias due to social desirability
This bias might be particularly relevant in contexts where students may fear that admitting to the use of generative chatbots could have academic repercussions." Furthermore, "the absence of these methodological steps may introduce limitations in terms of measurement validity and reliability," referring to the lack of a preliminary test of the instrument.

Open questions raised

Need for more diverse samples from private universities and online learning modalities to improve generalizability
Further refinement of the survey instrument as research in this field evolves
Incorporation of qualitative methods such as focus groups or individual interviews to provide broader perspective
Longitudinal research to understand long-term implications of chatbot use on development of academic skills
Deeper investigation into gender differences in attitudes and behaviors regarding chatbot use, as these remain insufficiently explored
Need for more diverse samples including private universities and online learning modalities

Extracted from: pdfAgreement 55%

Explore related topics

Responsible AI Use Human-AI Collaboration Survey