Everyone prefers human writers, including AI
2510.08831v1
cs.AI, cs.CL, cs.HC
2025-10-14
Авторы:
Wouter Haverals, Meredith Martin
Abstract
As AI writing tools become widespread, we need to understand how both humans
and machines evaluate literary style, a domain where objective standards are
elusive and judgments are inherently subjective. We conducted controlled
experiments using Raymond Queneau's Exercises in Style (1947) to measure
attribution bias across evaluators. Study 1 compared human participants (N=556)
and AI models (N=13) evaluating literary passages from Queneau versus
GPT-4-generated versions under three conditions: blind, accurately labeled, and
counterfactually labeled. Study 2 tested bias generalization across a
14$\times$14 matrix of AI evaluators and creators. Both studies revealed
systematic pro-human attribution bias. Humans showed +13.7 percentage point
(pp) bias (Cohen's h = 0.28, 95% CI: 0.21-0.34), while AI models showed +34.3
percentage point bias (h = 0.70, 95% CI: 0.65-0.76), a 2.5-fold stronger effect
(P$<$0.001). Study 2 confirmed this bias operates across AI architectures
(+25.8pp, 95% CI: 24.1-27.6%), demonstrating that AI systems systematically
devalue creative content when labeled as "AI-generated" regardless of which AI
created it. We also find that attribution labels cause evaluators to invert
assessment criteria, with identical features receiving opposing evaluations
based solely on perceived authorship. This suggests AI models have absorbed
human cultural biases against artificial creativity during training. Our study
represents the first controlled comparison of attribution bias between human
and artificial evaluators in aesthetic judgment, revealing that AI systems not
only replicate but amplify this human tendency.
Ссылки и действия
Дополнительные ресурсы: