AI evidence extraction

Can ChatGPT be an author? Generative AI creative writing assistance and perceptions of authorship, creatorship, responsibility, and disclosure

Paul Formosa, Sarah Bankins, Rita Matulionytė, Omid Ghasemi · AI & Society · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

4/4

Quality (LMQS)

Evidence

Citations

3.75

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s00146-024-02081-0

Methodology & findings

Study design

Mixed methods experimental survey study using factorial vignette design (N=602).

Sample

N = 602, 6 groups

Primary method

Analysis of Variance (ANOVA) models with categorical predictors (assistant type and assistance level) and ordered dependent variables. Between-subjects factorial design. Repeated ANOVA analyses conducted with ChatGPT experience as covariate. Additional analyses with age as covariate. Ordinal Bayesian regression models conducted for robustness. Thematic analysis of qualitative data using inductive approach with investigator triangulation (two researchers coding independently).

Main result

The study found that "for a human author, the degree of assistance matters for our assessments of their level of authorship, creatorship, and responsibility, but not what or who rendered that assistance, although it was more important to disclose human rather than AI assistance." Additionally, "when people evaluated the assisting agent, human assistants were viewed as warranting higher rates of authorship, creatorship, and responsibility, compared to AI assistants rendering the same level of support."

Reports effect sizes.

Research paradigm

Empiricist; positivist quantitative + interpretivist qualitative mixed methods

Author conclusions

"The increasing importance of Generative AI raises a range of ethical, philosophical, and legal issues." The authors conclude that "the degree of assistance matters for our assessments of their level of authorship, creatorship, and responsibility, but not what or who rendered that assistance except that it was more important to disclose human rather than AI assistance. However, regarding types of assistants, human assistants were viewed as having higher rates of authorship, creatorship, and responsibility compared to AI assistants rendering the same level of support. These results help us to better understand emerging norms around combined human-AI generated content, which has significance for a range of important practical and legal debates in the use of increasingly sophisticated GenAI technologies."

Risk of bias

Selection bias: Sample from Prolific platform (convenience sample of online respondents); Hypothetical vignette design: Participants evaluated scenarios rather than actual authorship experiences; Exclusion of participants who failed attention check: 21 participants removed from original 623; Potential social desirability bias in responses regarding disclosure and ethics; Limited sample diversity: 366 of 602 participants had Bachelor degree as highest education; Selection bias from use of Prolific platform; Hypothetical vignette context may not reflect real-world decision-making; Sample education skewed toward Bachelor's degree holders (366/602); Potential social desirability bias in responses regarding disclosure; Hypothetical nature of vignette study may not reflect real-world decision-making; Sampling from Prolific platform may introduce selection bias; Attention check excluded 21 participants (3.4%), but no analysis of whether excluded participants differed systematically; Between-subjects design prevents within-subject comparisons

Limitations

"Experimental vignette studies have established limitations, given their hypothetical nature." The authors note that "sampling choices can also impact the generalizability of results, and it would therefore be helpful to replicate our study with other samples." Additionally, the study was limited to "fictional novel case" contexts and did not examine "GenAI producing non-textual outputs, such as images, music, or video, and when producing non-fiction content, such as news articles, academic papers, or business reports."

Open questions raised

Authors identify need to: (1) replicate study with other samples including specific groups such as academic writing experts or working novelists; (2) examine effects for non-textual GenAI outputs (images, music, video); (3) extend study to non-fiction contexts (news articles, academic papers, business reports) where veracity concerns matter more; (4) specifically examine GenAI use in academic writing contexts given the focus of literature review on this area
Replication with other samples beyond Prolific participants
Extension to non-fictional content (academic papers, news articles, business reports) where veracity concerns differ
Examination of GenAI producing non-textual outputs (images, music, video)
Specific examination of GenAI use in academic writing contexts
Investigation of contractual arrangements and legal implications of human-AI co-authorship

Data: Anonymous quantitative data and analysis scripts available via OSF at https://osf.io/wgd7y/; "The anonymous data and analysis scripts are accessible through OSF (https://osf.io/wgd7y/)"; Anonymous quantitative data and analysis scripts available via OSF (https://osf.io/wgd7y/)Code: Analysis scripts available on Open Science Framework (OSF): https://osf.io/wgd7y/; OSF repository at https://osf.io/wgd7y/; Open Science Framework (OSF): https://osf.io/wgd7y/ (data and analysis scripts)Extracted from: pdfAgreement 47%

Explore related topics

AI Manuscript Writing Academic Writing Experimental