Sharing a specific case because I think it illustrates a problem that’s bigger than my classroom.
I had a student this term — strong ESL learner, first language Mandarin, been writing in English for about four years. Careful, rule-following writer. Correct grammar, predictable structure, clear topic sentences. Writes the way she was taught to write, which is also more or less how a lot of AI writes.
Her submitted essay came back at high AI probability on two separate tools. I knew from her in-class writing and her drafting process that she had written it herself. But if I hadn’t already known her work, I might have acted on those scores.
This isn’t an isolated case. ESL students, students from educational traditions that emphasize formal structure, students who’ve internalized academic writing conventions very thoroughly — they all write in ways that detection tools flag. The very students who most deserve support are the ones most likely to be falsely accused.
The tool can’t distinguish between “writes like AI” and “was taught to write with the same structural patterns AI was trained on.” Those are different things and the consequences of getting it wrong are serious.
I’m not saying detection tools are useless. I’m saying the false positive rate on structured non-native writing is a genuine equity problem that I haven’t seen addressed honestly anywhere. Does anyone have protocols for handling this specifically?
Worth saying that this problem has a name in the research: differential accuracy across demographic groups. Detection tools that work reasonably well on one population of writers can have dramatically higher error rates on others. Non-native writers, writers from certain educational traditions, writers with certain cognitive styles — all of these represent populations where the tools fail more.
The honest answer is that no current detection tool should be used as evidence in any academic integrity proceeding without corroborating evidence. A score is a flag for human investigation, not a finding. The equity dimension you’re describing is exactly why.
not to be dramatic but this is one of the things that keeps me up a little bit. the writing that looks “most AI” to a tool is structured, clear, grammatically correct. which is also the writing that took the most effort for someone who isn’t writing in their first language.
the irony is brutal. the students who worked hardest on clarity and correctness get flagged. the students who write loosely and colloquially don’t.
i don’t have a protocol answer. just wanted to say the framing of this post is exactly right and the tool developers should be asked about this directly and publicly.
This is a legitimate concern and the institutional response has been inadequate. Most schools have adopted detection tools without accompanying guidance on their known failure modes.
The protocol I’d suggest — not as a tech solution but as a process one — is to treat any detection result as the beginning of a conversation, never the conclusion. If a score prompts concern, the next step is a writing conference where the student explains their process and responds to questions about the content. A student who wrote the piece will be able to do that. The score alone should never be the basis for an integrity decision.
That said, pushing back on vendors to publish accuracy data broken down by writing population is overdue.
From my experience reviewing content from writers across different backgrounds, non-native English writing does trigger higher false positive rates consistently. This is well known in the content industry even if it hasn’t filtered into educational policy discussions yet.
The tools are trained on predominantly native English web content. Writing that doesn’t match those patterns — either because it’s AI or because it comes from a different linguistic tradition — gets flagged. Those are very different situations with very different implications.