How DropSlop is tested
How we check the detector and the rewriter, in general terms.
We test the detector and the rewriter on writing they have never seen, and we keep a record of every test.
The detector
- We keep test sets of writing whose origin we know: pages people wrote before AI writing tools were common, and texts written by several different AI models.
- Test sets are kept apart from the writing the detector learns from. We check that no page, site or passage appears in both.
- We count both kinds of mistake: writing by a person that the detector marks as machine-made, and machine writing it misses. Marking someone's own writing as machine-made is the more harmful mistake, so we look at it closely.
- We look at results by kind of writing, such as articles, business pages and essays, so a weakness on one kind of page shows up.
The rewriter
- We test it on AI-written drafts it was not tuned on, never on customers' drafts.
- A rewrite is judged on two things: whether it reads less machine-made, and whether it kept its facts. Facts are checked by a separate judge as well as by Fact-Guard, so Fact-Guard isn't marking its own work.
What we don't claim
No detector is right every time, and DropSlop is no exception. Its weak spots are listed in How to read a score. Treat every score as an estimate.