AI Detection Scores Weaponized as Public Criticism Tool

AI text detection tool 'Pangram' is being misused for user criticism on social media. Pangram is a tool that scores whether AI is used in text, but there has been a growing trend of using these scores as a basis for public criticism of others on social media, and it has been revealed that the company hired 'attackers' to support these activities.

AI-generated text detection technology has seen rapidly growing demand against the backdrop of the recent explosive expansion of generative AI. A need has emerged in academic and workplace settings to verify 'whether AI was relied upon,' and detection tools like Pangram have begun to attract attention. Against this backdrop, Pangram's scores have come to be treated as if they were 'evidence of AI use.'

However, the reality is more complex. What Pangram can measure is only the degree of possibility that AI was used, not definitive proof. Critics of the tool point out that even text independently researched and written over many hours can produce the same high score as text generated by AI in just 10 seconds. In other words, a high score does not prove that a person 'did not think for themselves' or 'did not write it themselves.'

The crux of the problem is a significant gap between the measurement accuracy of the tool and the criticism based on it. Pangram's score merely indicates a tendency that 'AI may have been used,' but public criticism on social media transmits this as a definitive message: 'This person relied only on AI and did not think for themselves.' This is a structural problem in which the limitations of measurement do not match the weight of criticism.

This situation highlights issues around the reliability of AI detection tools. No AI detection tool currently guarantees 100% accuracy and all carry the risk of false positives. Nevertheless, the numerical score has a tendency to take on a life of its own as 'easy-to-understand evidence,' which can easily lead to misunderstanding.

The fact that Pangram itself hired activists to conduct criticism campaigns raises questions about the company's position as a tool provider. The question of how much responsibility tool providers should bear for how detection results are used and their societal impact becomes an important issue for AI detection tools as a whole going forward. The key to maintaining trust in this field lies in how honestly convey the limitations of accuracy while designing mechanisms to prevent the misuse of scores.