Are AI detectors biased against non-native English writers? What the evidence shows

In 2023, one finding travelled further than almost any other piece of research on generative AI in education. Several detectors used to spot AI writing labelled most essays by non-native English writers as machine-generated, while essays by American schoolchildren passed almost untouched. Three years on, that result is still the most-cited reason universities give for restricting AI detection. It is also routinely stretched beyond what it showed.

The short answer is this. There is solid evidence that some detectors, under some conditions, flag second-language English writing as AI-generated far more often than first-language writing. There is also credible later evidence that the bias is not inevitable. Detectors built and tested carefully, on longer texts, have shown little or no gap. What has not changed is the underlying arithmetic. Any detector with a non-zero false positive rate will wrongly accuse some students, and the cost of those errors falls heavily on students who already write in a second language. This briefing sets out what each strand of evidence shows, why the results differ, and how institutions have responded.

How detectors decide

AI-text detectors do not find a hidden signature in machine writing. They estimate a probability that a text was generated, from statistical features of the words.

Many early detectors leaned on perplexity, a measure of how predictable a text is to a language model. Text produced by a model tends to be highly predictable to a similar model, because the model picks likely words. Human writing is usually less predictable. A low-perplexity text is therefore treated as more likely to be machine-made. Other detectors are supervised classifiers trained on labelled examples of human and AI writing. They may use perplexity alongside many other features, or none at all.

This is where second-language writing becomes a problem. A learner writing in English at B1 or B2 level typically draws on a smaller vocabulary, more common collocations and more conventional sentence patterns than an expert first-language writer. Those are exactly the features that make text predictable. Exam preparation reinforces them: candidates are taught templates for introductions, linking phrases and paragraph structure. In perplexity terms, careful, correct, limited English can look a great deal like the output of a cautious language model.

The study that started the debate

The finding behind the headlines comes from a team at Stanford led by Weixin Liang and James Zou, published in the journal Patterns in 2023. The researchers ran two sets of human-written essays through seven widely used detectors. The first set was 91 essays written for the TOEFL test and posted on a Chinese educational forum. The second was 88 essays by US eighth-grade students, taken from the Hewlett Foundation's Automated Student Assessment Prize dataset.

The contrast was stark. Across the seven detectors, the average false positive rate on the TOEFL essays was 61.22%. On the US essays it was close to zero. Eighteen of the 91 TOEFL essays (19.78%) were judged AI-generated by all seven detectors, and 89 (97.80%) were flagged by at least one.

Two follow-up experiments pointed to perplexity as the cause. When the researchers asked ChatGPT to enrich the vocabulary of the TOEFL essays, making them sound more like a native speaker, the average false positive rate fell to 11.77%. When they asked it to simplify the language of the US essays, misclassification rose from an average of 5.19% to 56.65%. In other words, the detectors were responding to linguistic sophistication, not to whether a machine had written the text. The second experiment is perhaps the more troubling. A student who used AI only to polish vocabulary was less likely to be flagged than one who wrote plainly and alone.

The authors were explicit about the limits. The samples were small, and most of the detectors tested were built on GPT-2, an older model. Their method focused on perplexity-based and supervised detectors, so it did not represent every detection technique. The TOEFL essays came from a single forum and a single first-language background. None of this makes the result wrong. It does mean the 61% figure describes seven 2023-era tools on 91 essays, not "AI detectors" in general, and certainly not every product on the market in 2026.

What later evidence adds

Four other pieces of evidence matter, and they point in different directions.

Independent testing of 14 tools. In December 2023, a team led by Debora Weber-Wulff published a systematic test of 12 public detectors and two commercial systems, Turnitin and PlagiarismCheck, in the International Journal for Educational Integrity. They ran 54 documents through every tool, 756 tests in total. No tool reached 80% accuracy, and only five exceeded 70%, with Turnitin scoring highest. The authors concluded that the tools were "neither accurate nor reliable". They also found a general tendency to classify texts as human, so lightly edited or paraphrased AI writing often went undetected. One category matters directly here. Human texts written in other European languages and machine-translated into English were classified correctly about 20 percentage points less often than texts written directly in English. The authors noted that this exposes non-native writers who draft in their first language and translate to false accusations.

The vendor's own subgroup study. In October 2023 Turnitin published research on its detector's performance with English language learners. It drew up to 2,000 texts per group from three open corpora, including ICNALE, a corpus of writing by Asian learners of English. For documents of 300 words or more, Turnitin reported no statistically significant difference in false positive rates between second-language and first-language writers, with both close to its 1% target. For shorter documents of 150 to 299 words, false positives were significantly higher than the target, and the company raised its minimum length for AI detection to 300 words. This is the vendor evaluating its own product, and the published summary does not give exact rates for each group. Its main value is the concession that short texts are a weak point, which bears directly on the brief assignments common in EFL teaching.

A large-scale assessment study. In August 2024, researchers at ETS, which runs the TOEFL and GRE tests, published a study in Computers & Education asking directly whether detecting ChatGPT-generated essays in a large-scale writing assessment was biased against non-native English speakers. Using GRE writing data, Yang Jiang, Jiangang Hao, Michael Fauss and Chen Li found that carefully developed detectors did not necessarily show the bias Liang's team reported. The important word is developed. These were detectors trained on a specific assessment, with known prompts and large volumes of genuine test-taker writing, not general-purpose tools applied to whatever students submit. A 2026 overview by Hao makes the same distinction and adds cautions of its own. Accuracy falls sharply on short responses and is unreliable for writing that mixes human and AI text. Detectors trained on one family of models may not transfer to another, so they need regular re-evaluation.

A replication in another language. At the EACL conference in 2026, Adnan Al Ali, Jindřich Helcl and Jindřich Libovický revisited the perplexity explanation using Czech. They found that texts by non-native Czech writers did not have lower perplexity than texts by native writers. They also found no systematic bias against non-native writers across three families of detectors, and showed that current detectors can work well without relying on perplexity. The study is about Czech, not English, and is a student-workshop paper, so it cannot settle the English case. It does suggest the effect Liang's team observed may depend on the language, the text type and the generation of detector, rather than being built into detection itself.

A sixth data point is the one that ended a product. OpenAI launched its own AI-text classifier on 31 January 2023. The company reported that the classifier correctly identified only 26% of AI-written English text and wrongly labelled human text as AI-written 9% of the time. It also warned that the tool was unreliable on texts under 1,000 characters and in languages other than English. OpenAI withdrew the classifier on 20 July 2023, citing its low accuracy.

html

<table>
  <thead>
    <tr>
      <th>Study or source</th>
      <th>Who produced it</th>
      <th>What was tested</th>
      <th>Main finding for second-language writers</th>
      <th>Main limitation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Liang et al., <em>Patterns</em> (2023)</td>
      <td>Independent academic team (Stanford)</td>
      <td>7 detectors; 91 TOEFL essays and 88 US eighth-grade essays</td>
      <td>Average false positive rate of 61.22% on TOEFL essays, close to zero on US essays; falls to 11.77% after vocabulary enrichment</td>
      <td>Small samples; mostly GPT-2-based detectors; one source of learner essays</td>
    </tr>
    <tr>
      <td>Weber-Wulff et al., <em>International Journal for Educational Integrity</em> (2023)</td>
      <td>Independent academic team</td>
      <td>14 tools including Turnitin; 54 documents, 756 tests</td>
      <td>No tool above 80% accuracy; machine-translated human text classified correctly about 20 points less often than original English</td>
      <td>Small document set; translated texts from European languages only</td>
    </tr>
    <tr>
      <td>Turnitin subgroup study (October 2023)</td>
      <td>Vendor evaluating its own product</td>
      <td>Up to 2,000 texts per group from ASAP, PELIC and ICNALE</td>
      <td>No statistically significant gap for texts of 300+ words; significantly higher false positives below 300 words</td>
      <td>Not independent; exact group rates not published in the summary</td>
    </tr>
    <tr>
      <td>Jiang, Hao, Fauss and Li, <em>Computers &amp; Education</em> (2024)</td>
      <td>ETS researchers</td>
      <td>Purpose-built detectors on GRE writing data</td>
      <td>Carefully developed detectors did not necessarily show bias against non-native speakers</td>
      <td>Detectors built for one assessment with known prompts; not general-purpose tools</td>
    </tr>
    <tr>
      <td>Al Ali, Helcl and Libovický, EACL (2026)</td>
      <td>Independent academic team</td>
      <td>Three detector families on native and non-native Czech writing</td>
      <td>Non-native texts not lower in perplexity; no systematic bias found</td>
      <td>Czech, not English; workshop paper</td>
    </tr>
    <tr>
      <td>OpenAI AI classifier (2023)</td>
      <td>Model developer</td>
      <td>Its own classifier on an internal challenge set</td>
      <td>26% of AI text identified; 9% of human text wrongly flagged; unreliable outside English and on short texts</td>
      <td>Withdrawn in July 2023; no subgroup analysis</td>
    </tr>
  </tbody>
</table>

Why the results diverge

Read together, the studies do not contradict each other as sharply as the headlines suggest. They differ in conditions, and the conditions predict the results.

General-purpose tools against purpose-built ones. The worst results come from general tools run on writing they were not designed for. The best come from detectors trained on one assessment, with known prompts and large samples of genuine writing from the same population. Most classroom use looks like the first case.

Length. Every source that examines length finds that short texts are where detectors fail. Turnitin raised its minimum to 300 words, OpenAI warned about texts under 1,000 characters, and the ETS overview reports sharp losses on short responses. Many EFL tasks are paragraph-length or short essays, which places learners' writing in the least reliable range.

Text type and preparation. Formulaic genres and template-driven exam writing are predictable by design. The University of Melbourne's own staff guidance warns that formulaic writing and a regular personal style raise the chance of a false positive. Those are common features of writing by students trained for high-stakes English tests across Asia.

Tools that sit between "human" and "AI". Machine translation, grammar checkers and paraphrasing tools all change the statistical profile of a text. Weber-Wulff's team found that translation reduced correct classification of genuinely human text. Many second-language writers draft partly in their first language, or run their work through a grammar tool, which is often explicitly encouraged. Detectors cannot see that process. They see only the output.

Detector generations. The 2023 tools Liang's team tested are not the 2026 products. Detection methods have changed, and so have the models being detected. A result that was true of GPT-2-based detectors may not hold for a current classifier, and a current classifier's validation may not hold against next year's models.

The arithmetic institutions tend to miss

Even a detector that performs equally well for all students produces wrongful flags, and the proportion of flags that are wrong depends on how much misuse there is. The following figures are an illustration, not measurements, but the logic applies to any detector.

Suppose a department receives 1,000 essays, and 100 of them contain substantial AI-generated text. Its detector catches 90% of those 100, and wrongly flags 1% of the 900 genuine essays. It produces 90 correct flags and 9 false ones, so about one flag in eleven points at an innocent student. Now suppose the detector's false positive rate on second-language writing is 5% rather than 1%, and every student in the cohort writes in a second language. The same 900 genuine essays now produce 45 false flags alongside the 90 correct ones. One flag in three is wrong.

Vanderbilt University made the same point with its own numbers in August 2023, when it turned off Turnitin's AI detector. The university had submitted around 75,000 papers to Turnitin in 2022. At the vendor's stated 1% false positive rate, that would have meant about 750 papers wrongly labelled as partly AI-written.

For a UK or Australian university where a large share of postgraduate students write in English as a second language, or an English-medium university in Asia where almost all students do, this is not a marginal concern. It is how detection works out across a whole cohort.

How institutions have responded

Institutional responses fall into three broad groups: switching detection off, keeping it with restrictions, and treating it as one input among several. The positions below are drawn from each institution's own published guidance.

html

<table>
  <thead>
    <tr>
      <th>Institution or body</th>
      <th>Position</th>
      <th>Date</th>
      <th>Stated reasoning or safeguards</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Vanderbilt University (US)</td>
      <td>Turnitin's AI detector disabled</td>
      <td>August 2023</td>
      <td>Lack of transparency, false positives (about 750 of 75,000 papers at a 1% rate), research on bias against non-native English writers, privacy</td>
    </tr>
    <tr>
      <td>Curtin University (Australia)</td>
      <td>Turnitin's AI detection disabled; text-matching retained</td>
      <td>Announced September 2025, effective 1 January 2026</td>
      <td>Fostering trust and clarity, and making assessment secure, fair and future-ready</td>
    </tr>
    <tr>
      <td>University of Melbourne (Australia)</td>
      <td>Detection available to staff, not students</td>
      <td>Guidance current as of February 2026</td>
      <td>Consider only scores above 20%; a high score is not proof and cannot alone support an allegation; a second source of evidence is required</td>
    </tr>
    <tr>
      <td>Hong Kong Baptist University</td>
      <td>Detection tools may support but not replace academic judgement</td>
      <td>2025 guidelines</td>
      <td>No decision made solely by automated tools; staff may request AI-use records, drafts or an oral explanation; all generative AI use must be declared</td>
    </tr>
    <tr>
      <td>Turnitin (product design)</td>
      <td>Scores of 1–19% shown as an asterisk, not a number</td>
      <td>Current product documentation</td>
      <td>To reduce false positives at low scores; detection limited to submissions of 300 words or more</td>
    </tr>
    <tr>
      <td>European Union, AI Act</td>
      <td>AI systems for "monitoring and detecting prohibited behaviour of students during tests" listed as high-risk</td>
      <td>Annex III rules apply from 2 December 2027, according to the European Commission's timeline as amended by the Digital Omnibus on AI</td>
      <td>High-risk systems face requirements on risk management, data quality, human oversight and accuracy</td>
    </tr>
  </tbody>
</table>

Two points in that comparison deserve emphasis.

First, the dominant position is restriction, not prohibition. Institutions that keep detection almost all frame it the same way: a score may prompt a closer look, but it is not evidence of misconduct on its own. The second evidence these policies ask for is concrete. It includes fabricated references, a marked break from a student's previous work, leftover AI phrases, drafts, version history and a conversation with the student. These are exactly the forms of evidence that do not depend on how predictable a student's English is.

Second, regulation is beginning to reach this area, but its reach is uncertain. The EU AI Act's high-risk category for education explicitly covers systems that monitor and detect prohibited behaviour during tests. Whether a text detector used on coursework, rather than during an exam, falls within that wording is a question of interpretation that regulators and courts have not settled. The Act would in any case apply only to providers and deployers within its territorial scope. Still, the direction is clear. Where detection affects a student's access to education, legislators expect human oversight and documented accuracy. That is a standard many institutions have not yet set for themselves.

In the UK, there is no equivalent sector-wide rule. The Quality Assurance Agency's 2023 advice described detecting generative AI as "fraught with difficulty", and decisions remain with individual universities. Use of AI by students is now near-universal. HEPI's 2026 student survey of 1,054 UK undergraduates found that 95% used generative AI in some form, and that 12% directly included AI-generated text in assessed work, up from 3% in 2024. That makes the question of how institutions establish misuse, and not merely suspect it, more pressing every year.

What this means for English-language teaching

For EFL teachers and the institutions that employ them, the evidence supports a few practical conclusions.

Do not treat a detector score as a finding. Use it, if at all, as a reason to look at other evidence. Policies that require a second, independent source before any allegation are the minimum consistent with the research.

Be most cautious where detectors are weakest. Short tasks, formulaic genres, exam-style essays, and work by students who are known to use translation or grammar tools all sit in the zone where false positives are most likely.

Design assessment so that process is visible. Drafts, in-class writing, oral follow-up and short reflections on how a piece was produced give evidence about authorship that no statistical model can. They also have value for learning in their own right.

Write tool-use rules that reflect how second-language writers work. If grammar checkers and dictionaries are permitted but translation is not, or the reverse, say so explicitly. Many integrity cases begin with a mismatch between what the student assumed was allowed and what the teacher intended.

For students, especially international students, the practical advice follows from the same evidence. Keep drafts and version history. Record which tools were used and for what. Declare use where the institution requires it, and ask early if the rules are unclear. A student who can show how a text came into being is far better placed than one who can only assert that they wrote it.

What remains unresolved

Several questions remain open. No large independent study has yet compared current commercial detectors on writing by Asian learners of English across proficiency levels, with transparent group-level error rates. Vendors' own subgroup analyses remain the main source of evidence for specific products, and they are rarely published in full. It is not clear how detectors should treat the growing share of writing that is neither wholly human nor wholly machine-made, which is precisely the category second-language writers using legitimate support tools produce. And it is not yet known how the EU's high-risk rules will be interpreted for academic integrity tools when they take effect.

Until those gaps are filled, the defensible position is the one most careful institutions have already reached. The 61% figure should not be cited as a universal fact. But no detector score should be allowed to stand alone as evidence against a student writing in a second language.

Frequently asked questions

Are AI detectors biased against non-native English speakers?

Some have been. A 2023 Stanford study found that seven detectors misclassified an average of 61% of 91 TOEFL essays as AI-generated. Later research, including large-scale work by ETS, found that carefully developed detectors do not necessarily show this bias. The risk is highest with general-purpose tools, short texts and formulaic writing.

Why do detectors flag second-language writing?

Many detectors rely partly on how predictable a text is. Writing with a smaller vocabulary and more conventional phrasing is more predictable, and so can resemble machine-generated text statistically.

Is Turnitin's AI detector biased against English language learners?

Turnitin's own 2023 study reported no statistically significant difference for documents of 300 words or more, but higher false positives for shorter texts. This is vendor research and has not been independently replicated in full.

Can an AI detector score prove misconduct?

No. Universities including Melbourne and Hong Kong Baptist University state that detector results cannot be the sole basis for a decision and require other evidence.

Have universities stopped using AI detection?

Some have. Vanderbilt disabled Turnitin's AI detector in 2023, and Curtin University did so from 1 January 2026. Many others keep it with restrictions.

Does the EU AI Act cover AI detectors in education?

The Act lists AI systems used to monitor and detect prohibited student behaviour during tests as high-risk. The European Commission's timeline gives 2 December 2027 for Annex III rules. Whether text detectors used on coursework fall within that category has not yet been settled.