How accurate is the AI detector?

What it measures

• Sentence rhythm: chat models write sentences of very even length; people vary more.
• Stock phrases: a catalogue of habits like "delve", "it's important to note" and "in today's fast-paced world".
• Contractions and formality: AI text tends to avoid contractions.
• Hedging, list shapes and openers: repeated transitions and tidy lists of three.
• Paragraph shape: paragraphs of near-identical length.

A quick scan uses only these statistics. A deep scan adds a language-model review of every sentence and blends it with the statistics.

How we try not to over-claim

• Every result shows a confidence level. Under about 80 words, or with a middle-range score, it says low.
• Short texts are pulled toward the middle, because there is little evidence.
• A statistics-only scan is never labelled high confidence.
• You see the reasons behind the score and each sentence's score, so you can disagree.

What has and has not been tested

HumanTalky is in beta. We run a small, repeatable test (npm run bench) on about 30 labelled texts. These are the results of the quick scan, without the deep review:

• On the texts we tuned it on, it ranks AI above human about 97 times in 100. On a fresh set it had not seen, about 90 in 100.
• At the 70% line it flagged none of our human texts, but it also caught only about 4 in 10 to 7 in 10 of the AI texts. Plain, short AI writing often lands near 50, which reads as "mixed". The deep scan is stronger on those.
• In a side-by-side on 8 texts, Sapling's free detector scored plain AI text higher and more confidently than our quick scan. We do not claim to be the most accurate detector.

Limits of this test: it is small, and our human texts are classic books, not modern writing such as emails or blog posts. So we cannot yet give you a trustworthy error rate on everyday human text. We will publish one once we have a large set of real modern writing.