Why AI Text Detectors Fail Beyond Benchmarks
Researchers found that AI-generated text detectors achieving high benchmark accuracy often fail in real-world settings because they exploit dataset-specific artifacts rather than identifying genuine signals of machine authorship. Using explainable AI techniques on two major benchmark datasets, the team demonstrated that detector performance degrades substantially when tested across different domains and generators, with the most discriminative features varying significantly between datasets. The work reveals a fundamental tension in linguistic-feature-based detection: features most useful for in-domain classification are also most vulnerable to domain shift and formatting variations. The authors released an open-source Python package providing both predictions and instance-level explanations to support more robust detector development.