How do AI detectors work? What they measure and what they miss

Wu and colleagues' survey of detection research sorts the field into watermarking, statistics-based detectors, neural-network detectors and human-assisted methods, and notes the shared problems of out-of-distribution text, deliberate attacks and the lack of a common evaluation framework (Wu et al. 2024, A survey on LLM-generated text detection, arXiv 2310.14724, read 9 November 2026).

The commercial detectors in this site's published method, GPTZero, Originality.ai, Copyleaks, Pangram and ZeroGPT, are trained classifiers, some with statistical features alongside. Watermarks only apply when the generating model planted one, and the writer rarely knows whether it did.

The core idea is predictability. A language model assigns a probability to each next word; text a model would have written scores high, and the summary statistic, perplexity, is low. Liang and colleagues showed what that measures in practice: the TOEFL essays all seven detectors in their study flagged as AI had significantly lower perplexity than the rest, and prompting a chatbot to enrich the vocabulary raised the perplexity and cut the average false positive rate from 61.22 percent to 11.77 percent (Liang et al. 2023, Patterns, read 9 November 2026). They warned against "using low perplexity as an indicator of AI-generated text", because limited vocabulary is also low perplexity.

DetectGPT refined the statistic. Mitchell and colleagues observed that text sampled from a model tends to sit in regions of negative curvature of the model's log probability function, so perturbing a passage and comparing scores separates model text from human text without any training data; on news articles from a 20 billion parameter model it raised detection from 0.81 to 0.95 AUROC against the strongest zero-shot baseline (Mitchell et al. 2023, DetectGPT, arXiv 2301.11305, read 9 November 2026). That is the detector Krishna and colleagues then cut from 70.3 to 4.6 percent with a paraphraser the same year.

A classifier is a language model fine-tuned to tell two piles apart. OpenAI described its 2023 classifier as "a language model fine-tuned on a dataset of pairs of human-written text and AI-written text on the same topic", with the threshold set to keep false positives low, and retired it in July 2023 at 26 percent true positives and 9 percent false positives (OpenAI, new AI classifier announcement, read 9 November 2026).

The current vendors describe the same shape with more parts. GPTZero states that it uses "an end-to-end deep learning approach" trained on web, education and AI-generated text, a sentence-by-sentence classifier that gives a probability and confidence per sentence, and a "Paraphraser Shield" against paraphrasing and homoglyph attacks, with confidence categories tuned so the average error rate is under 1 percent for high-confidence predictions (GPTZero, technology, read 9 November 2026). Turnitin states that its model is built on the transformer architecture, trained on its in-house corpus of academic data, that sentences over a threshold are labelled AI, that the document score is the share of prose sentences above the threshold, and that it does not display scores below 20 percent (Turnitin, AI writing detection FAQ, read 9 November 2026). Pangram Labs describes a transformer classifier trained with "hard negative mining with synthetic mirrors" to drive down false positives on high-data domains such as reviews (Emi and Spero 2024, arXiv 2402.14873, read 9 November 2026).

Three things follow. A classifier knows only what it was trained on, so new models and new genres are blind spots. A threshold is a vendor's choice, so the same text can be "AI" at one company and "human" at another. And the percentage is not a share of the text a person wrote; Turnitin says so in as many words.

A watermark is the one method that does not read the text's statistics after the fact; it plants a pattern at generation time. Kirchenbauer and colleagues' method selects a randomised set of "green" tokens before each word is generated and softly promotes them during sampling, so that a detector holding the key can run a statistical test with interpretable p-values on a short span of text, with what the authors call negligible impact on text quality (Kirchenbauer et al. 2023, A watermark for large language models, arXiv 2301.10226, read 9 November 2026). Google has deployed a text watermark under the SynthID name.

The catch is coverage and durability. A watermark only exists if the generating model applied one, and Krishna and colleagues reported in 2023 that paraphrasing evaded a watermarking scheme along with the other detectors they tested, while Sadasivan and colleagues showed recursive paraphrasing weakening watermark detection too. The guide on AI text watermarks covers where watermarks are deployed and what survives a rewrite.

Because they measure different things with different models and cut at different points. One scores perplexity against its own model; another classifies sentences with a network trained on its own corpus; a third looks for a watermark that may not exist. Weber-Wulff and colleagues saw the result in 2023: 14 tools on the same 54 documents, with false positive rates from 0 to 50 percent and no tool above 80 percent accuracy. The guide on whether AI detectors work sets those figures beside the vendors' current claims.

That disagreement is why this site's method records each of five detectors' own label, on the run date, rather than a single pass or fail, and checks every input before humanizing so the change a tool makes is visible against a baseline. No detector score should be the only evidence in a misconduct case. The guides hub holds the rest of the series, including what to do if you are the one flagged.