Readers Rate AI-Generated Novels Higher Than Human Works

When comparing short stories written by ChatGPT with those written by humans, many people cannot distinguish between them. A new study has confirmed this. In an experiment with more than 2,500 participants, the accuracy rate of correctly identifying which work was written by a human was nearly at the level of chance probability, further highlighting the difficulty of differentiating between AI and human text.

Since its public release at the end of 2022, ChatGPT has rapidly improved the accuracy of text generation. As a result, identifying whether something was written by AI has become difficult not only in everyday writing but also in creative domains such as stories and poetry. This research is noteworthy in that it directly confronted this question with a large number of participants.

The results showed that AI-generated short stories received higher ratings than works written by humans. However, the moment participants were told "this was written by AI," their evaluation of the work dropped. Despite the text itself remaining unchanged, a fascinating psychological reaction was observed: simply knowing that the author is AI changes how people perceive it.

This phenomenon, which could be called the "source effect," demonstrates that information about "who (or what) created it" has a stronger influence on evaluation than the content of the work itself. In literature and art, the existence and intentions of the author are often considered part of the viewing experience, while AI is easily regarded as lacking such human context. Therefore, a structure emerges where the same quality of text receives a lower evaluation once its source is revealed.

As generative AI becomes more widespread, discussions about the "authenticity" of content are expanding across many fields including text, images, and music. This study shows that high quality alone may not be sufficient to gain public trust and acceptance, and it provides empirical evidence for the issue of how to disclose and label AI-generated content. The research contributes to understanding how AI-generated content should be presented.

On the other hand, the fact remains that readers could not distinguish between AI and human works, even though evaluations dropped. In other words, two problems coexist: the reality that AI has achieved a quality level that makes identification nearly impossible, and the psychological reaction that occurs when people learn this fact. How to interpret these two aspects remains a difficult challenge for creators, media outlets, and platforms utilizing AI.

In the future, discussions about disclosure requirements and display methods for AI-generated content may become more active. The accumulation of research on how readers and viewers evaluate AI-generated content "after knowing its origin" could become an important insight that influences policy-making and platform design, and it continues to warrant close attention.