The short answer
AI detectors do not work reliably enough to be used as proof. They estimate how likely text is to be machine-written, and research shows they can flag human writing as AI, especially by non-native English speakers, while lightly edited AI text often slips through. OpenAI withdrew its own detector in 2023 because of low accuracy, and Australian regulators advise against relying on detectors alone. Treat a score as a prompt for a conversation, never as a verdict.
- Detectors give a probability, not proof of who wrote something.
- A Stanford-led study found they wrongly flagged many non-native English essays.
- OpenAI shut down its own AI detector for low accuracy.
- Use a score to start a conversation, never to punish.
How AI detectors work
Most detectors look for statistical patterns: how predictable the word choices are, how varied the sentence length is, and whether the text resembles what language models tend to write. They output a probability or percentage. That is an estimate about style, not a record of who typed the words, and nothing in the text itself proves its origin. This is why the same piece of writing can score differently in two different tools, and why a short text, a technical text or a formulaic one is more likely to be misjudged.
What the research found
A widely cited study from Stanford researchers tested several detectors on essays by native and non-native English writers. The detectors misclassified more than half of the non-native essays as AI-generated while handling native writing far better. The authors argued the tools should not be used in settings that judge people, such as education. The same study showed that asking a model to rewrite text with richer wording made the non-native essays pass, which suggests the detectors were measuring simple phrasing, not authorship. Sources: Liang et al., GPT detectors are biased against non-native English writers
Even the makers struggle
OpenAI released an AI text classifier in January 2023 and withdrew it that July, saying its accuracy was too low. At launch it caught only about a quarter of AI-written text while wrongly labelling some human writing as AI. If the company that built ChatGPT could not make a reliable detector, a score from another vendor deserves caution. OpenAI said it would keep researching better ways to tell, but a purpose-built detector from the maker of the leading model still could not meet its own bar. Sources: The Register: OpenAI quietly shuts down its AI classifier
Vendor claims versus independent testing
Vendors publish low false-positive rates from their own testing. Turnitin, for example, has said its document-level false positive rate is under 1 per cent for text with more than 20 per cent AI writing, and has said its detector should not be the sole basis for action. Independent academics have questioned how those figures hold up on real, varied writing. Sources: Turnitin: AI writing detection update from the chief product officer, iTnews: Australian universities hits and misses detecting ChatGPT in assignments
Why workarounds beat detectors
People can rewrite AI text, ask a model to vary its style, or run it through a paraphrasing tool, and detection rates fall sharply. That means a clean score proves little as well. The tools therefore miss some AI text and flag some human text, which is the worst combination for anyone who wants certainty. The practical result is an arms race that honest people lose: the person who followed the rules gets flagged, while someone willing to run text through a rewriter walks through.
What to do instead
If authenticity matters, change the process. Ask for drafts, version history, a short conversation about the work, or a task done live. In Australia, TEQSA advises universities against relying on detection alone and pushes assessment redesign, a sound model for businesses too. If you do use a detector, record the score as one input and speak to the person before drawing any conclusion. Sources: TEQSA: artificial intelligence in higher education
Questions people ask
Are AI detectors accurate?
Not reliably. They produce both false positives and false negatives, and their makers advise against treating a score as proof.
Can AI detectors flag my own writing?
Yes. Clear, plain or formulaic writing, and writing by non-native English speakers, is more likely to be flagged.
Which AI detector is the most accurate?
No detector is accurate enough to rely on alone. Published accuracy figures usually come from the vendors themselves.
Should my business use an AI detector?
Only as a weak signal alongside other evidence, never as the sole basis for a decision about a person.
No affiliate links on this page at publication. Product names link to official sites so you can check current pricing. How we work.
