Why do AI detectors give different results on the same text?

Weber-Wulff and eight colleagues from the European Network for Academic Integrity wrote 54 documents in six classes, from human text never exposed to the internet to AI text paraphrased by QuillBot on its default setting, and ran every document through 12 free detectors plus Turnitin and PlagiarismCheck between March and May 2023, 756 tests in all (Weber-Wulff et al. 2023, International Journal for Educational Integrity, read 9 November 2026). The figures below are theirs, and every one is a 2023 reading on the model each vendor served that spring.

Read across one row and the disagreement is plain. GPTZero was the only tool that leaned toward calling text AI, which gave it the lowest miss rate and the highest false accusation rate at once; Content at Scale called everything human and missed every AI document. The same text, sent to those two tools on the same day, came back with opposite answers, and both tools were behaving as designed.

Three things differ between any two detectors: the model, the data it learned from, and the line it draws between human and AI. Wu and colleagues' 2025 survey in Computational Linguistics sorts the field into watermark detectors, statistics-based detectors, neural classifiers and human-assisted methods, and lists the open problems as out-of-distribution text, attacks on the detector, messy real-world data and the lack of a shared evaluation framework (Wu et al., A survey on LLM-generated text detection, arXiv 2310.14724, read 9 November 2026). Two tools from two families are not measuring the same thing, so they have no reason to agree.

The threshold matters as much as the model. Weber-Wulff and colleagues scored the tools under four approaches, one of which treated a reading as correct only above 80 percent confidence and another above 60 percent; the scores changed and the ranking almost did not, which means the thresholds decide how often a tool is "sure", not which tools are better. A vendor that moves its threshold changes every reading without changing its model. What each family actually measures is covered in detail on the detector measures guide.

Because paraphrasing pushes text toward what each model treats as human by a different amount. In the 2023 test, overall accuracy on unedited AI text was 74 percent, on AI text a person edited with synonyms and reordering 42 percent, and on AI text paraphrased by QuillBot 26 percent, so most paraphrased AI text read as human and the tools that still caught it were not the same tools that caught the originals.

Sadasivan and colleagues stress-tested detectors the way an attacker would. Their recursive paraphrasing attack, run on passages of about 300 tokens, cut detection rates across watermark-based, neural, zero-shot and retrieval detectors while only slightly degrading text quality in many cases, and they showed that an attacker could infer a watermark's signature without access to the detector and use it to make human text read as AI (Sadasivan et al. 2023, arXiv 2303.11156, read 9 November 2026). Their theoretical result ties the best possible detector's performance to how far apart human and AI text distributions sit, which shrinks as models improve. A humanizer is a paraphraser with a purpose, so every finding here applies to humanized text directly.

Not always. Weber-Wulff and colleagues expected two sets of AI documents prepared the same way to score the same, found that the OpenAI Text Classifier and DetectGPT gave notably different results on them, re-uploaded documents to check, and got different values again; they attributed the change to the tools, not to their testing. The study did not measure replicability systematically, and the authors called that inconsistency another strong reason against using a detector as the single source of an accusation.

No detector score should be the only evidence in a misconduct case. The study's authors put it in their own words: reports from these tools cannot be used as the only basis for reporting students for cheating, and a student accused on that basis alone would have no way to mount a defence.

By reporting the spread rather than averaging it away. The published method runs every humanizer output through five detectors no humanizer vendor owns, GPTZero, Originality.ai, Copyleaks, Pangram and ZeroGPT, and records each one's own label, its published threshold and the date, with a screenshot. Every input is scanned before any tool touches it, so the change a humanizer makes is visible against a baseline instead of against another detector's opinion, and six human-written control texts show what each detector's false positive behaviour looks like on the same genres.

Pass is counted only on each detector's own human label, per detector and pooled, and the per-detector tables show where the five disagree. Detector pass rate carries 30 of 100 points in the score; meaning kept carries 35 and errors added 20, because a rewrite that drops a claim has done damage no detector will report. The guides hub holds the rest of this series, including what detectors measure and what to do if you are flagged. Anyone writing for a class should read their institution's AI policy first; a humanizer does not make undisclosed AI use acceptable.