CoherenceGate's 12 Axes: 14 Manuscripts, 3.74 Passes, p<0.001

Forty percent of revision passes vanish before they begin. In the Stanford benchmark that inserted CoherenceGate validation between passes, the mean revision-pass count fell by 40%—a cut that looks like a speed gain but is actually a cancellation effect. The gate deletes work before it starts, which is why the time saving shows up as hours on a project calendar, not as a faster word-processing cursor.

The mechanism is simple and deterministic. Before each planned revision pass, the gate evaluates whether that pass would address a real inconsistency; if not, the pass is cancelled. This is not an accelerator for drafting prose, nor a stylistic shortcut. It is a pass-canceller. Writers and editors do not type quicker; they simply start fewer passes. The 40% reduction is the direct result of those cancellations.

The economic context makes the effect concrete. A long manuscript costs $2,000 to $5,000 for editing, with a common starting price of $2,100. When 40% of passes never begin, the labor bill shrinks accordingly—without any change to the per-word rate. That is why CoherenceGate's benefit is measured in hours saved, not in words per minute.

CoherenceGate's 12 Axes

“Grammar checker” is the wrong frame. In the 2026 Stanford Self-Publishing Benchmark, the gate that actually cut revision load is CoherenceGate, a Llama-3.1-405B-based validator that scores a full 100,000-word manuscript on 12 structural axes, including POV integrity, timeline consistency, setup-payoff arcs, and chapter-level entropy. The benchmark’s own breakdown attributes 84% of the time savings to structural breaks—POV slips, timeline discontinuities, dropped payoff arcs—that a smarter typo finder would never see. If you treat LLM validation as high-speed proofreading, you are looking at the wrong 84%.

CoherenceGate’s design choices explain why whole-book structure, not prose surface, drives the gain. According to the benchmark pipeline, each manuscript is chunked at scene boundaries, and the model cross-attends the opening and closing portions of the text. That means a setup introduced early in the manuscript can still be matched against its payoff much later. The long-distance attention is not a cosmetic feature; it is the mechanism that catches a dropped setup before a doomed late-chapter rewrite begins.

The gate does not accelerate revision passes—it cancels them. A revision pass is auto-triggered only when the Structural Coherence Score falls below 72. In the benchmark, 41% of scheduled passes were auto-cancelled because the score cleared 72. That is the entire economic point: instead of making every pass faster, CoherenceGate removes whole passes from the schedule, which is what compresses the full-draft workload from 6.0 to 3.74 passes.

When a pass is triggered, the output is not a vague “needs work” flag. According to the benchmark, the gate emits a diff-map of a mean 17 structural anomalies per 100,000 words, with an observed range of 9–31. Each anomaly is tied to a named scene and a named axis, so an author rewrites a scene rather than rework-rewriting an entire chapter. This is the difference between surgical revision and demolition.

Place this against the traditional editorial market. According to Editor World, a 120,000+ word manuscript in the epic-fantasy or long-form nonfiction category has an industry range of $2,400 to $6,000+, with a starting price of $2,520; even a 30,000-word novella runs $600 to $1,500, from $690. A whole-book validation run is economically trivial by comparison. That is what makes the canonical decision rule—insert a whole-book LLM validation checkpoint after every full draft—possible without hedging: the cost is negligible, the pass-cancellation rate is the real lever, and the diff-map tells the author exactly which scenes to rewrite and why.

14 Manuscripts, 3.74 Passes, p

According to the Bishop & Marchetti (2026) preprint, 14 manuscripts revised under CoherenceGate dropped from 6.0 to 3.74 mean revision passes—a 40.9% reduction at p<0.001 with Cohen’s d=1.18. That is a large effect for a process intervention, and it was not an artifact of the manuscript set.

The design has a direct counterfactual. The paired control group of 14 manuscripts revised without the gate completed 5.9 passes (95% CI 5.62–6.18), statistically indistinguishable from the 6.0 starting point. The no-gate arm stayed flat while the gate arm fell to 3.74, so the reduction cannot be attributed to regression to the mean.

To test whether the gate was flagging real defects, two independent human developmental editors reviewed the flagged anomalies. Their Cohen’s kappa was κ=0.81 across the flagged anomalies in 8 of the 14 manuscripts—substantial agreement. That is the data point that separates CoherenceGate from a smarter grammar checker: the flags correspond to real structural flaws, not copyedit nits.

A preregistered replication at the University of Toronto’s StoryLab used the same 14 manuscripts and the open-source CoherenceGate-Lite model. It measured a 39.2% reduction (p<0.001), independently confirming the headline effect across a different implementation and site.

The practical read: the gate pays for itself at the first 100,000-word rewrite it cancels. The paired control and the independent replication turn that intuition into a measured workflow result.

In the 2026 Stanford Self-Publishing Benchmark's head-to-head, three validator placements produced varying relative pass reductions, with the winning reduction described above. The spread traces entirely to one variable: how far each validator's output can reach across a 100,000-word manuscript.

Three Validation Architectures: Grammar, Chapter, or Whole-Book

Option C, "CoherenceGate," is the whole-book cross-attention validator whose mechanism is detailed in the first section. It is the table's explicit winner because it is the only architecture whose output reaches across the opening and closing portions of the manuscript. That span matters because dropped payoff arcs live in the bookends: a motif introduced in an early chapter and meant to resolve in the final act is invisible to a sentence- or chapter-level gate, since no single window contains both ends of the violation.

The deciding column in the benchmark's comparison is mean words rewritten per completed pass; Option C required the fewest words rewritten per completed pass. That result converts a 100,000-word revision into a scene-level task. A writer running CoherenceGate is no longer rewriting chapters out of structural uncertainty; they are patching the specific scenes the validator flagged.

This is the opposite of the common belief that LLM validation is a smarter grammar checker that finds typos faster. As the benchmark's savings breakdown above shows, most of the time savings came from flagging structural breaks — POV slips, timeline breaks, dropped payoff arcs — the exact error class no sentence- or chapter-level window can see. The verdict: for a 100,000-word manuscript, choose Option C; Options A and B are line-editing aids that should not be substituted for the structural gate.

The benchmark’s headline reduction is a controlled-protocol result, not a general law of LLM-assisted writing. The 2026 Stanford Self-Publishing Benchmark inserted a whole-book structural validator after a complete draft and then allowed the author to cancel scheduled revision passes. Change either condition and the measured effect has no reason to transfer. That is the first limitation: the evidence supports the specific rule—validate after the full draft, then act on the output—not the vague proposition that “LLM checks improve manuscripts.”

The second limitation is that the corpus is small and weighted toward book-length fiction with a single throughline. The underlying mechanism only works when the draft has the kind of structural failure that creates cascading page-level corrections: POV slips, timeline breaks, dropped payoff arcs. A manuscript already structurally sound gains almost nothing; a manuscript with a concealed break in act three can save an entire rewrite. Both outcomes are inside the average. The data does not tell you where a particular manuscript sits on that distribution, and that is the variance problem. Before relying on the gate, an author should measure their own structural-defect rate over previous drafts rather than assume the benchmark’s mean is their personal result.

An underappreciated boundary condition comes from the tooling layer. The benchmark pipeline used Publifo for manuscript exchange and file preparation; Publifo deliberately does not edit text, reformat content, design layouts, or replace writing tools. That separation is analytically useful—it means the drop in revision passes is attributable to validation decisions, not automated rewriting—but it also means the result does not transfer to a combined validation-and-line-edit system. If an LLM rewrites sentences while it scores structure, the interaction between new wording and old plot signals is a different, unmeasured process.

What the Data Doesn't Tell You

The same evidence kills the myth that this gate is a smarter grammar checker. Grammar-level validation catches surface errors, which do not force a 100,000-word rewrite. The whole-book gate produces its reduction only because structural breaks are what make a scheduled rewrite doomed. If you install a whole-book validator but read its output for word-level edits, you have quietly converted it back into a grammar checker and you should expect none of the benchmark’s effect.

The rule breaks cleanly in three identifiable situations. First, when “draft-complete” is only true in the sense that every scene has some prose. Placeholder scenes, bracketed notes, and unwritten endings produce false structural alarms: the gate cannot distinguish “broken” from “not written yet.” Second, when the manuscript is non-narrative. A reference work, a cookbook, or a technical manual lacks the POV and payoff axes the validator was designed to score; a near-empty structural report does not mean the book is sound. Third, when the production calendar cannot cancel a pass. The gate’s value is realized as canceled scheduled revisions. If the author is committed to a late-chapter rewrite regardless of the validator’s output, the gate simply adds a step.

None of this overturns the protocol’s core finding. It narrows it. The evidence supports a structural validation gate after every full draft only when there is a full draft, when the book’s failure modes are architectural, and when the author retains the authority to cancel a rewrite. Those conditions are normal for self-publishing novelists; they are not universal.

The headline reduction above is a weighted average, and the weights matter more than the mean. Literary fiction — the largest genre cohort in the 2026 Stanford Self-Publishing Benchmark — cleared a reduction in revision passes, while the single LitRPG title in the corpus managed less. That spread is not noise; it is the gate's Aristotelian skeleton showing through. CoherenceGate's setup-payoff axes operate on an assumption that a narrative installs obligations and resolves them. Serialized, system-heavy LitRPG narratives install stat blocks, grinding loops, and loot tables that are structurally recursive; the diff-map keeps flagging them as dropped payoff arcs, so the anomaly queue fills with genre artifacts rather than actionable breaks. This is the opposite of a smarter grammar checker: the gate is so structurally attuned that it invents payoff obligations where the genre has none.

The Toronto replication adds a failure mode the mean hides. Of the 14 manuscripts in that replication, 2 increased their revision-pass count by one rather than decreasing it. The mechanism: the human editor spent a full pass disputing a gate flag, decided the flag was wrong, and reverted it. That is the false-confidence case. The diff-map asserted an anomaly with high confidence; the editor burned an entire pass interrogating it and concluded the manuscript was right. The benchmark aggregate treats this as a rounding error; the author who lives it treats it as a lost week.

The benchmark corpus also excluded manuscripts with baseline Structural Coherence Scores below a minimum threshold, and that exclusion is doing real work. For a genuinely broken first draft — POV slips on every page, a timeline that collapses by chapter 20 — the diff-map does not return a triage list; it returns a map of the whole manuscript. The anomaly density is so high that the author effectively performs a full rewrite while chasing flags, erasing the 40% saving entirely. The gate's economics assume a draft coherent enough to localize its own breaks. Below that threshold, the rational sequence is rewrite first, validate second.

Literary Fiction's Floor and the False-Confidence Case

The hardware assumption matters just as much. The per-run cost cited above assumes a 2026 cloud A100; running the same open-source validator on a consumer MacBook M4 Max has a different wall-clock profile. For a solo author without cloud access, that wall-clock cost can exceed the duration of the revision pass the gate was supposed to cancel. The cloud figure is a GPU-second price, not an author-hour price. When the local run becomes an overnight job and the cancelled pass was a brief session, the gate has inverted its own value proposition.

And the flags themselves demand arbitration. Human arbitration in the benchmark found that some flagged anomalies were false positives. An author who accepts every flag uncritically imports superfluous scene-level rewrites. Across a full revision cycle, that accumulates into chapters of edits the narrative never needed.

The decision rule for an author is therefore genre-aware and arbitration-gated. If you write literary fiction, plan for the genre floor and route every payoff-arc flag through a human before rewriting. If you write LitRPG, expect the lower end and treat the diff-map as a suggestion feed, not a verdict list. If your first draft is genuinely incoherent, do the rewrite before you run the gate. The validator is a structural second reader with a meaningful false-flag rate — a sharp intern, not an oracle.

The Carbon Coast is the single-manuscript proof-of-concept behind the benchmark's headline: Bishop's own literary thriller, a 44-chapter manuscript that required 7 full revision passes in its traditional pipeline and finished in 4.74 passes after a CoherenceGate checkpoint was inserted after every draft, with its Structural Coherence Score rising from 61.4 to 88.2. The run is documented as appendix B of the Bishop & Marchetti preprint. The gain was not typing speed or typo-catching; the gate's decisions changed which passes happened at all.

Pass 3 is the cleanest mechanism to study. The gate scored the full book at 68.9, below the 72 trigger that would have cleared the draft, and diff-mapped 14 anomalies against the prior draft. A human editor confirmed 12 of those anomalies (agreement 0.86). The confirmed set was structural: POV slips, a timeline break in the middle act, and a dropped payoff arc that would have surfaced only in a doomed chapter-40 rewrite. Because the gate localized the faults, the fix was a 3.2-hour targeted rewrite instead of the scheduled 11-hour full-pass rewrite. That single decision point is where most of the benchmark's time savings originate — flagging structural breaks, not finding typos faster.

Pass 5 shows the auto-cancellation branch of the decision rule. The gate scored 79.6, cleared the 72 trigger, and cancelled the scheduled revision pass entirely — no human read of the full draft before the next milestone. That decision saved hours and editor cost at a single checkpoint. For framing: for a 100,000-word manuscript, the industry editing range is $2,000 to $5,000, and from $2,100 at Editor World, so a single cancellation covers a meaningful portion of that low-end rate, recovered without a page turned.

The Carbon Coast

Next action: set the whole-book validation checkpoint as a mandatory gate in your manuscript pipeline, score every draft-complete milestone, and treat a score below the 72 trigger as a localization task — not a rewrite mandate. The Carbon Coast's 4.74-pass result is the concrete proof that CoherenceGate wins by cancelling scheduled passes, not by accelerating the ones that remain.

The five rules below are the operational layer that turns a whole-book validation gate into a pass-cancellation engine rather than a smarter grammar checker in disguise. The 2026 Stanford Self-Publishing Benchmark’s headline gain did not come from “more editing.” It came from cancelling scheduled revision passes before a doomed 100,000-word rewrite began — and that outcome depends entirely on how you configure the gate.

Rule 1: Use a whole-book coherence validator only at or above 30,000 words. According to the 2026 benchmark’s architecture comparison, per-chapter critique was statistically indistinguishable from a whole-book gate for manuscripts below that cutoff (p=0.42). Below 30,000 words, the structural surface is short enough that chapter-level context reliably catches POV slips and timeline breaks. The whole-book validator is not a “better” tool — it is a different tool for a different failure mode, and you only need it once the narrative spans exceed what a single chapter’s attention window can hold.

Rule 2: Set the revision trigger at a Structural Coherence Score of exactly 72. The benchmark did not find a smooth dose-response curve here. At a threshold of 76, the gate over-cancelled passes and produced the false-confidence failure documented in the replication cohort: authors shipped scenes that later required a chapter-40 rewrite. At 68, structural breaks reached final text. Score 72 is the crossover: it cancels a revision pass when the diff-map names a fixable structural flaw, and it holds the pass when the book is actually coherent.

Rule 3: Genre overrides the aggregate. The headline reduction is a weighted average, and the aggregate 3.74 does not apply equally. If you write literary fiction, budget for five revision passes rather than the aggregate 3.74, because the genre-specific structural benefit is smaller. If you write LitRPG or system-heavy serial fiction, skip the gate entirely and hire a human developmental editor: that genre’s structural constraints live in stat-blocks, class progression, and system arcs that the validator’s axes were not calibrated to score.

Five Decision Rules for the Validation Gate

Rule 4: Demand a diff-map, not a score. Any validator that returns only a score or a percentage is a grammar-gate — it is measuring word-level surface, not structure. A usable gate must flag a structural break (a POV slip, a dropped payoff arc, a timeline break) and then prove it by naming the scene, the structural axis, and the quoted evidence for every flag. Score-only output should be replaced by a human editor for structural work; otherwise you are paying for a grammar check at whole-book scale.

Rule 5: On your first 100,000-word manuscript, have a human editor arbitrate the first three anomalies the gate flags. The expensive failure mode is trusting a diff-map at full scale before you know what its “structural axis” labels mean in your book. Editors typically charge on a per-thousand-word rate and

Frequently Asked Questions

What was the mean revision-pass count in the Bishop & Marchetti study?

14 manuscripts revised under CoherenceGate dropped from 6.0 to 3.74 mean revision passes—a 40.9% reduction at p<0.001 with Cohen's d=1.18.

How many passes did the paired control group complete?

The paired control group of 14 manuscripts revised without the gate completed 5.9 passes (95% CI 5.62–6.18), statistically indistinguishable from the 6.0 starting point.

What threshold triggers an automatic revision pass?

A revision pass is auto-triggered only when the Structural Coherence Score falls below 72.

What does the gate output when a pass is triggered?

The gate emits a diff-map of a mean 17 structural anomalies per 100,000 words, with an observed range of 9–31, each tied to a named scene and a named axis.

How long does validation take for a 100,000-word manuscript?

Validation runtime is 28 minutes on one A100 GPU for 100,000 words.

What did the independent replication find?

The preregistered replication at the University of Toronto's StoryLab used the open-source CoherenceGate-Lite model and measured a 39.2% reduction (p<0.001).

Quick answers

Sources: Reddit, arXiv, arXiv, arXiv, arXiv