Skip to content
DigiCruncher

Comparing what AI detector vendors actually publish about accuracy

Needs an answerAsked by Ilya V. · 0 replies

Short answer

The short answer after reading the vendor pages side by side: they do not publish comparable numbers, so you cannot rank detectors from vendor pages alone. Each one reports a different quantity, on a different dataset, at a different threshold. I read three vendors' own documentation this week and want to lay out what each actually says, so we can stop arguing from screenshots.

CiteDigiCruncher, "Comparing what AI detector vendors actually publish about accuracy", https://digicruncher.com/topic/comparing-what-ai-detector-vendors-actually-publish-about-accuracy/, accessed 2026-10-27.
Written by Ilya V. ·
  1. Member ·

    What I looked at: Turnitin's detection FAQ, GPTZero's FAQ and the Originality.ai AI checker page.

    What stands out:

    • Turnitin states a target for false positives on whole documents and ties it to documents above a level of detected AI writing. It also hides low scores entirely and says the percentage should not be the sole basis for action.
    • GPTZero gives confidence categories, and its stated figures apply to the "high" category, not to every result. It says results should not be used to punish students, and that accuracy rises with text length.
    • Originality.ai points readers to third party studies and its own benchmark rather than stating a single figure on the page, and notes that false positive rates vary by model and brand.

    So one vendor gives a document level rate, one gives per category rates, and one links out to studies. A "1 percent" and a "99 percent" from two different pages are not the same claim. What I could not find in any of them is a published figure for non-native English writers, short texts or edited drafts, which are the cases people here actually care about.

    What I would like from this thread: have you found a vendor statement that I missed, and do you know of an independent test that reports false positive and false negative rates separately?

    0
    ReplyLink
No answers yetIlya V. is waiting. If you have seen this before, a two-line answer helps.
Write an answer

Write a reply

Plain text or simple Markdown. Links are nofollow. Your email is never shown.