Audited Benchmarks & Technical Whitepaper · Zero-Leakage Vaults

How DropSlop scores against paid detectors & humanizers.

Plain English at the top, full methodology and downloadable proof below. Every evaluation text was locked in a strict vault with zero overlap with training data—measuring accuracy, false alarms on real writers, pasted AI paragraphs, and factual fidelity.

AI Detector v5 Accuracy 95.3%

0.9927 AUROC across 300 held-out articles; catches 98.0% of raw AI and 90.5% of AI paragraphs pasted inside human articles.

False Alarms on Hard Human 2.0%

Only 3 of 150 tough pre-ChatGPT human texts flagged at 0.50 (0 of 25 editorial posts)—safer than GPTZero (4.0%) and Pangram 4.1 (7.3%).

Humanizer Fact Retention 530/530

100% material facts kept across 45 audited drafts (90% pass GPTZero on PR pitches), vs. WriteHuman inventing details on 96% of articles.

MCP & REST Forensics 16 Tools

Live link & stat verification, site sameness checks, SEO info gain, human-edit certificates, author voice forensics, skim heatmaps & PR diff gates.

View by role:

Complete Side-by-Side Scorecard (Detector v5 + Fact-Guard Humanizer v4.5)

Showing all 800 held-out detector benchmark texts and 45 live-verified humanizer drafts alongside GPTZero, Pangram 4.1, WriteHuman, Phrasly, and previous DropSlop releases. Select a role above to highlight the exact metrics that matter most for your workflow.

Open X-Ray

1. AI Detector v5 vs. Previous Versions & Paid Competitors

What changed in plain English: Most detectors read a page with one model and average the whole page together—so two AI paragraphs pasted into a human article disappear into the average. Detector v5 pairs two vocabulary-aligned language models side-by-side to spot unnatural word choices and grades every paragraph independently, running locally at $0 per check.

Held-Out Test Suite DropSlop v5 (Current) DropSlop v4 / v3 GPTZero (Live API) Pangram 4.1 (Live API) Plain-English Takeaway
Everyday Articles & Blogs (D1 · n=300) 150 verified human + 150 frontier AI across Business, Tech, News, Essays & Blogs 95.3% accurate (0.9927 AUROC) Catches 98.0% of raw AI (147/150) v4: 95.0% (0.9926 AUROC) v3: 94.3% (0.9867 AUROC) 100.0% Paid cloud API 100.0% Paid cloud API Beats v4 & v3 Locally 98% raw AI recall on local hardware with zero per-word API fees or third-party data sharing.
False Alarms on Hard Human Writing (D2 · n=150 + 25) 150 formal pre-ChatGPT human texts (wire news, SEC filings, technical, ESL) + 25 editorial articles 2.0% false alarms at 0.50 (3/150) 3.3% at strict 0.45 · 0/25 (0%) on editorial posts at 0.50 v4: 1.3% (2/150) v3: 3.3% (5/150) 4.0% false alarms (6/150) 2x more false accusations than v5 at 0.50 7.3%–8.7% false alarms Flagged 32% (8/25) on editorial benchmark Safer Than GPTZero & Pangram Lowest false-accusation risk among major detectors on formal, technical, and journalistic human prose.
Hybrid Articles: AI Pasted Into Human Text (D3-Hybrid · n=50) 50 human articles with 1–2 AI paragraphs quietly spliced inside (367 total paragraphs) 34.0% full-doc catch (17/50) 90.5% block accuracy (0.9369 AUROC, catches 76% of spliced AI paragraphs) v4: 20.0% (10/50) v5 catches +70% more hybrid docs 0.0% caught (0/50) Whole-page averaging hides pasted AI blocks Not scored Credit-capped per document Major Win Over GPTZero (34% vs 0%) Pinpoints the exact pasted AI paragraph inside an otherwise human-written draft.
Disguised / "Humanized" AI Pages (D3-Spun · n=100) 50 AI articles run through WriteHuman + 50 run through Phrasly 52.0% WriteHuman caught 78.0% Phrasly caught v4: 50% WH · 76% Phrasly v3: 38% WH · 70% Phrasly 32.0% WriteHuman 64.0% Phrasly 58.0% WriteHuman 90.0% Phrasly +20 pts Over GPTZero · Behind Pangram Handily beats GPTZero on spun AI; Pangram 4.1 still leads on full-page commercial spinners.
Short Single Paragraphs, 40–100 Words (D4 · n=200 + 536) Standalone quotes, email intros, blurbs & 536 individual article paragraphs 87.0% accurate (0.9534 AUROC) 86.4% across 536 D1 short paragraphs (100% blind) v3 clean: 76.5% (0.8164 AUROC) 72.6% on 536 short paragraphs 96.5% (0.9872 AUROC) Paid API Full doc credit per snippet +10.5 to +13.8 pt Zero-Leak Jump Reliable paragraph-by-paragraph grading inside X-Ray, PR quotes, and CI diff gates.
Business (n=100)
99.0%
0.9988 AUROC · 100% AI caught
Technical (n=40)
97.5%
0.9950 AUROC · 0% false alarms
Essays (n=50)
96.0%
0.9984 AUROC · 100% AI caught
News (n=50)
94.0%
0.9872 AUROC · 4% false alarms
Blogs (n=60)
90.0%
0.9978 AUROC · 100% AI caught

2. Fact-Guard Humanizer v4.5 vs. WriteHuman, Phrasly & v3

Why competing humanizers break trust: WriteHuman beats detectors by inventing fake anecdotes, unrequested first-person filler, and random days of the week in 24 out of 25 articles—while Phrasly keeps facts only by barely changing the text (failing GPTZero 88% of the time on full articles). DropSlop Fact-Guard v4.5 enforces 4 hard safety gates and varies sentence cadence across neighboring paragraphs.

Quality & Stealth Metric DropSlop Humanizer v4.5 DropSlop v3 (Previous) WriteHuman Phrasly Plain-English Takeaway
PR Pitches, Quotes & Short Copy (PR-20 · n=20) Scored live against paid GPTZero API + Pangram 4.1 API 90.0% pass GPTZero (18/20) 75.0% pass Pangram · 70.0% pass both (91.7% on blind-reader 12) 33.3% reader-approved Dropped closing point on 8 of 12 ~58% pass both Injects casual filler 57% pass GPTZero 43% pass Pangram Best-in-Class on Pitches & Quotes 9 in 10 pass GPTZero while keeping every executive name, number, and closing takeaway intact.
Material Fact & Number Retention (All-45 · 530 Claims) Independently audited claim-by-claim by Claude Haiku fact-checker 100.0% kept (530 / 530) 45 of 45 drafts had zero dropped material facts · 0% dropped endings 98.5% kept Dropped facts in 3/25 articles; dropped endings in 67% of pitches 98.6% kept Dropped facts in 3/25 articles 99.4% kept Minimal structural editing Zero Dropped Material Facts Safe for journalists, agencies, finance, legal, and researchers who cannot afford silent deletions.
Zero Invented Details / Hallucinations (Benchmark-25) Share of full-article rewrites with 0 dropped facts AND 0 invented details 40.0% strict zero-addition 51.1% across all 45 drafts · hard-blocks invented names & fake dates 16.0% (4/25) 59 added details total 4.0% (1 out of 25) Fabricates details in 96% of articles (67 invented additions) 40.0% (10/25) Only 12.0% pass GPTZero (3/25) 10x Cleaner Than WriteHuman Never invents fake Tuesday anecdotes or fictional coworkers just to game perplexity.
House Style, POV & Student Mode Lock (n=45) Enforcing "No I/we/you", "No em-dashes", "No rhetorical questions" & locked paragraphs 100.0% compliance (0/45 broken) Hard candidate veto before assembly 58.3% compliance Broke POV/rules on 5 of 12 pitches No rule lock Injects 1st-person slang No rule lock Strict Editorial Control Enforces publication style guides and third-person academic/wire voice automatically.

3. Capabilities Frontier Chatbots & Standard Detectors Don't Have

ChatGPT and Claude cannot visit live URLs inside your draft without guessing, cannot compare an article against 200 pages already on your domain, and cannot issue cryptographic proof of human editing. All 7 new forensic engines below are live in the web app, REST API, and 16-tool MCP server.

Live Fact & Link Verification

Live Citation & Statistic Verifier

Fetches every cited URL and DOI on the live web (with Wayback fallback) and verifies that the exact numbers, percentages, and quotes in your sentence actually appear on the source page.

Run inside X-Ray →
Site-Wide SEO Protection

Site Sameness (Monoculture) Check

Scans a new draft against dozens or hundreds of posts already on your site to catch recycled AI outlines, repeated intro hooks, and self-competing pages before search engines penalize your domain.

Run inside Site Audit →
Search Information Gain

Originality & Information-Gain Scorer

Compares your draft against competing articles on the topic to measure your net-new facts, numbers, and examples—and lists the unanswered reader questions that will set your piece apart.

Measure Info Gain in X-Ray →
Cryptographic Proof of Work

Tamper-Proof "Verified Human-Edited" Badge

Issues a signed HMAC-SHA256 editing-lineage certificate proving genuine human drafting and revision—including a strict zero-storage privacy mode where your text is never saved.

Mint a badge in X-Ray →
Forensic Stylometry

Author Voice Match & Ghostwriting Check

Measures unconscious writing habits—punctuation rhythm, function words, and clause pacing—against 2–10 past samples to confirm author match and flag exact paragraphs where pasted AI crept in.

Check voice match →
Reader Attention Modeling

Reader Skim & Drop-Off Heatmap

Simulates sentence-by-sentence reader survival for a busy Reporter, Executive, Blog Reader, Engineer, or Academic—pinpointing slow throat-clearing intros and jargon walls.

Simulate reader skim →
CI / CD & CMS Automation

Pull-Request & CMS "Diff Gate"

Grades only newly added prose lines in a GitHub pull request or CMS update—skipping code, config files, and untouched legacy paragraphs—to block AI slop automatically in milliseconds.

Get the GitHub Action & API →

4. Technical Whitepaper: Methodology, Firewall Rules & Limits

For engineers, editors-in-chief, and researchers reviewing how every number on this page was produced.

4.1 Zero-Leakage Vault Firewall & Out-of-Fold Calibration Protocol

Many AI detection benchmarks accidentally leak evaluation domains or tune thresholds directly on their test sets. DropSlop v5 enforces a programmatic three-tier firewall across all 800 held-out items (D1_core_300, D2_hard_human_150, D3_adversarial_ai_150, and D4_short_paragraphs_200):

  • Exact & Near-Duplicate Purge: 450 source URLs, 800 SHA-256 hashes, and 216,791 normalized 8-word shingles from the evaluation vaults are blacklisted prior to feature scaling, SVD fitting, LoRA fine-tuning, and ensemble calibration. Four historical SEC training documents and ten derived short slices overlapping D2 were purged; verified remaining overlap across all splits is 0.
  • Parent-Group Out-of-Fold Calibration: Every base learner (ModernBERT document encoder, single-trajectory SWA short-paragraph LoRA on 14,201 firewalled slices, Dual-LM Binoculars GBDT, and paragraph MIL head) produces strictly out-of-fold predictions grouped by parent document before convex meta-calibration.
  • Why We Replaced v4's Short-Text Checkpoint: During our v5 audit, we discovered our earlier v4 short-paragraph script had evaluated D4 after each epoch to pick its checkpoint (90.5%). We discarded that checkpoint and trained v5 strictly on 5-fold Group-OOF with zero D4 access—achieving an honest 87.0% accuracy / 0.9534 AUROC (+13.7 AUROC points over clean v3).
4.2 Detector v5 Architecture: Vocabulary-Aligned Dual-LM Binoculars + Concordant MIL

Single-model perplexity detectors confuse specialized human domain vocabulary (legal, medical, engineering, or ESL prose) with unusual token choices, causing false alarms on real writers while missing AI paragraphs diluted inside longer human articles. Detector v5 combines three orthogonal signals on local hardware:

  • Vocabulary-Aligned Dual-LM Binoculars (26 Features): Runs a synchronized pass over Qwen2.5-0.5B (Base) and Qwen2.5-0.5B-Instruct (RLHF-tuned, 151,936 shared tokens) to compute exact cross-entropy H(P_Base, P_Inst), the Binoculars ratio NLL_Inst / XCE(Base || Inst), token-level RLHF log-probability advantage, per-token KL divergence, Fast-DetectGPT curvature, and 24/48-token sliding sub-window dynamics. Rare technical nouns raise surprisal in both Base and Instruct equally—canceling out false alarms—while instruction-tuned phrasing exposes a sharp divergence.
  • Cross-Architecture Concordant Paragraph MIL (v5_hybrid_mil): Scores every paragraph (≥ 25 words) independently and requires geometric concordance sqrt(P_ModernBERT_short * P_Binoculars_short) before elevating whole-document hybrid probability—catching 34% of spliced hybrid documents at the document level (vs. 0% for GPTZero) and localizing constituent paragraphs at 90.46% accuracy (0.9369 AUROC).
4.3 Fact-Guard Humanizer v4.5: Four Hard Safety Gates & Dual-Draft Long-Form Mode

Instead of asking a single LLM prompt to "make this sound human" (which hallucinates casual anecdotes or drops caveats), Fact-Guard Humanizer v4.5 generates diverse candidate rewrites per hot paragraph across specialized LoRA adapters and filters them through four deterministic and neural gates:

  • Gate 1 — Anchor, Number & Closing-Point Lock: Extracts every number, percentage, date, proper noun, quoted string, conditional qualifier (if, unless, only when), and final-sentence takeaway, rejecting any candidate that drops one (530/530 material claims preserved across 45 audited drafts).
  • Gate 2 — Ungrounded Addition Veto: Rejects candidates that invent unrequested proper nouns, fake days of the week, or fabricated anecdotes.
  • Gate 3 — House Style & POV Lock: Enforces third-person wire/academic voice, forbidden vocabulary lists, and punctuation rules with 100% compliance (0/45 violations).
  • Gate 4 — Cross-Paragraph Cadence Beam Search & Dual-Draft Mode: Assembles 8 whole-document combinations that vary sentence count and opening clause grammar across adjacent paragraphs. On longer 300–500+ word articles, Dual-Draft Mode lets writers compare the 100% Strict Fact-Locked draft side-by-side with a Max Stealth structural rewrite that highlights every loosened sentence in place.
4.4 Honest Limitations & Where Paid Cloud Competitors Still Lead

We publish where our local models win and where trade-offs remain:

  • Full-Page Commercial Spinners: While Detector v5 beats GPTZero by +20 points on WriteHuman (52% vs. 32%) and +14 points on Phrasly (78% vs. 64%), Pangram 4.1's large cloud model still catches a higher share of pure full-page spun text (58% WriteHuman, 90% Phrasly). Our upcoming v5.1 spinner-whiplash training specifically targets this gap.
  • 400+ Word Articles Under Strict Fact-Locking: Keeping 100% of facts, numbers, and qualifiers locked on dense 400-word articles passes GPTZero on 40% of drafts in strict single-pass mode (compared to WriteHuman's 52%, which invents fake details on 96% of drafts). Turning on Dual-Draft Mode in Humanize surfaces both the strict fact-locked draft and a higher-stealth structural option with inline diff highlights.
  • Always Inspect the Marked Spans: No AI detector is 100% infallible—especially on short formulaic business boilerplate. DropSlop underlines exact phrases and paragraph heatmaps rather than issuing an opaque accusation.

Auditable Raw Benchmark File

Every figure on this page is backed by /benchmarks/data/benchmarks-v5.json (Detector SHA-256: 61089fb53a04…7cc5a9).

Download JSON