AI evidence extraction

Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey

Abdulhadi Shoufan · IEEE Access · 2023

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

6/10

Relevance

1/4

Quality (LMQS)

Evidence

464

Citations

16.71

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1109/access.2023.3268224

Methodology & findings

Study design

Two-stage mixed methods study: Stage 1 involved qualitative thematic analysis of open-ended responses from students (n=56) after using ChatGPT for a learning activity, generating 36 codes and 15 themes from 3136 words of text.

Sample

N = 56, 1 group

Primary method

Thematic analysis using coding methodology for qualitative data (Stage 1). Questionnaire-based survey for quantitative validation (Stage 2). Specific statistical tests or software not mentioned in the abstract.

Main result

The study found that "the students admire the capabilities of ChatGPT and find it interesting, motivating, and helpful for study and work. They find it easy to use and appreciate its human-like interface that provides well-structured responses and good explanations." However, "many students feel that ChatGPT's answers are not always accurate and most of them believe that it requires good background knowledge to work with since it does not replace human intelligence."

Reports effect sizes.

Research paradigm

Mixed methods (qualitative thematic analysis with quantitative survey validation)

Author conclusions

The authors conclude that "ChatGPT can and should be used for learning. However, students should be aware of its limitations. Educators should try using ChatGPT and guide students on effective prompting techniques and how to assess generated responses. The developers should improve their models to enhance the accuracy of given answers."

Risk of bias

Selection bias: Sample limited to senior students in computer engineering program, not representative of broader student population; Potential social desirability bias in self-reported perceptions; Attrition risk: Three-week gap between Stage 1 and Stage 2 could result in participant loss; Recall bias from temporal separation between qualitative and quantitative phases; Single institution sample limits external validity; Selection bias: Sample limited to senior students in a computer engineering program; Self-selection bias: Students who used ChatGPT may have different perceptions than those who did not; Temporal bias: Three-week gap between stages could affect recall and response consistency; Lack of control group: No comparison with students who did not use ChatGPT; Selection bias: participants limited to senior students in computer engineering program; Self-selection bias: students who chose to use ChatGPT may differ from those who did not; Temporal confounding: attitudes may change over three weeks between survey stages; Lack of control group comparison

Open questions raised

The study identifies the need for: (1) educator training on ChatGPT use and prompting techniques, (2) student guidance on assessing AI-generated responses, (3) future development to enhance accuracy of AI models, (4) future research into ChatGPT's impacts on learning, academic integrity, employment, and life.
The study provides insights into student perceptions of ChatGPT and suggests future research directions related to educator adoption, prompting techniques, assessment of AI-generated responses, and model improvement for enhanced accuracy.
The authors identify that while the study provides insights into ChatGPT capabilities and limitations in education, further research is needed to understand implementation strategies for educators, development of assessment techniques for student-generated responses, and systematic evaluation across different educational disciplines and student populations.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 63%

Explore related topics

AI Literacy Training Doctoral Training Mixed Methods