Does AI follow your chapter plan? What a 2026 book-writing paper found
A May 2026 preprint measured how well a book-writing model follows its own plan. Here is what it found, its limits, and a beat-by-beat check for your own AI-written chapters.
In this guide
Short answer: not reliably, according to one May 2026 preprint that measured it. Its authors trained a model to write a book by first producing a plan, then chapters. They report that chapter text matched its chapter plan less well as chapters went on, and they call the result insufficient for reliable book-length generation. If you generate chapters from an approved plan, read each chapter against that plan before you build on it.[1]
What the paper did
The preprint, by Jan Zierstek and colleagues, was submitted on 16 May 2026 and revised on 28 June 2026. The team fine-tuned a language model on about 6,000 public-domain books. Each book was turned into a chain of summaries: a premise, a book preview, a book plan and chapter plans. The model learned to expand a prompt down that chain and then write text. It was compared with GPT-5.5, Claude Opus 4.8 and other baselines on seven narrative benchmarks.[1]
This is the developers' own preprint, and I read its text rather than running the model. Its footnotes link a public dataset (LongPage) and a "trained model collection" described as a research preview on Hugging Face. I did not check what those pages contain, their licenses, or whether the collection is complete. The paper says nothing about ScribistIQ, and nothing here shows that any ScribistIQ output changes because of it.[1]
What the numbers say
Panel 1 shows that the model's book preview followed the prompt closely (93.7 percent), but following dropped to 54.2 percent from preview to book plan and 39.2 percent from plan to first-chapter text. Panel 2 shows chapter-against-plan scores: 73.1 percent for chapter 1, 65.4 percent for chapter 2 and 50.8 percent for the final evaluated chapter, with later chapters ranging from 37.2 to 65.5 percent. The authors write that improving long-context consistency together with prompt and scaffold following is the most important direction for future work, and that sufficiently long generations eventually enter an unrecoverable repetition spiral.[1]
What the paper did not show
| Question a writer might ask | What the preprint supports | What it does not support |
|---|---|---|
| Does AI beat human or other AI writers? | On its seven first-chapter benchmarks the new model scored higher than GPT-5.5 on all seven and highest on four. Claude Opus 4.8 scored higher on three, including keeping one viewpoint. | It does not rate whole books or readers' enjoyment. Those benchmarks look at the first chapter only, using a language model to extract features and fixed rules to pass or fail them. |
| Will my plan be followed? | A model that is explicitly built around plans still followed them less well in later chapters. | It does not measure ChatGPT, Claude or ScribistIQ on your plan. |
| Is the new model available to me? | The paper's footnotes link a public dataset and a trained model collection labeled a research preview on Hugging Face. | It does not state a license or what the collection contains, and it does not say any writing tool, including ScribistIQ, uses the model. Check the model pages yourself before relying on them. |
Why this matters for your plan
Writers describe the same pattern informally. In a dated May 2025 forum thread, one author who generated a 20 to 30 chapter outline and then wrote chapter by chapter said events planned for later chapters kept appearing early and that later chapters repeated earlier material. Replies in the thread offered no fix, so treat it as one reader's account of the problem, not evidence about how often it happens.[4]
ScribistIQ's public features page says that on the Pro generator you can review and edit the chapter plan before any prose is written, and that each chapter is written from the approved outline. The continuity guide says the outline is an input, and its checker compares chapters with recorded summaries to find contradicting facts. Neither page says finished chapters are compared with the plan, so that check is yours to do.[2][3]
A beat ledger: check each chapter against its plan
The method below is an untested editorial approach, not something the paper or ScribistIQ prescribes. A beat is one event the chapter plan says must happen. The ledger lists each beat and what the draft actually did with it.
Illustrative example
Beat 1: Mara finds the keeper's logbook in the cellar. Beat 2: Mara notices pages torn out after 14 March. Beat 3: Her brother Tom lies about being at the harbor that night. Beat 4: Mara decides not to tell the constable yet. Not yet: nobody learns who tore out the pages. The harbor master's secret stays hidden until chapter 7.
Mara finds the logbook. She reads it and notices missing pages. Tom admits he was at the harbor and explains why. Mara tells the constable everything. The harbor master confesses he tore out the pages.
| Plan beat | What the draft did | Ledger result |
|---|---|---|
| Beat 1: finds the logbook | Found it in the cellar | Present |
| Beat 2: pages missing after 14 March | Missing pages noticed, no date given | Present, detail lost |
| Beat 3: Tom lies | Tom admits and explains | Changed to the opposite |
| Beat 4: Mara waits to tell the constable | Mara tells the constable | Changed to the opposite |
| Not yet: harbor master's secret | Harbor master confesses | Early reveal |
Two beats flipped and one reveal arrived three chapters early. Reading only the prose, this chapter could feel fine, which is why a ledger is worth keeping.
Steps
- Before generating, write each chapter's plan as one to five beats, each a single observable event. Add a Not yet line for anything that must stay hidden. On the Pro generator, review and edit the chapter plan before prose is written; this is where the beats and Not yet lines belong.
- Generate one chapter and read it once without the plan.
- Build the ledger: for each beat mark Present, Present with a detail lost, Changed, Missing or Early. List any new event the draft added that the plan did not contain.
- Decide per row. If the change is better than your plan, update the plan for later chapters so they build on it. If not, select the passage in Write Studio and rewrite, expand or shorten it, or describe the change you want. The studio marks the change up before you accept it.
- Start the next chapter only when the ledger has no Changed, Missing or Early rows you did not choose.
- Check chapters 1 to 3 carefully, since the preprint's scores were highest early and fell later. After that, check every chapter if you can, or at least every few chapters. That cadence is advice, not a tested rule.
Optional: ask an assistant to draft the ledger
You can paste the plan and chapter into a chat assistant and ask for a first ledger, then verify it yourself. An assistant can misread a chapter too, so confirm each Changed and Missing row against the text.
Here is a chapter plan with numbered beats and a Not yet line, followed by a draft of the chapter. For each beat, say whether the draft shows it, changes it, misses it or reveals it early, and quote the sentence that proves it. Then list events in the draft that are not in the plan. Do not judge the writing quality. PLAN: [paste plan] DRAFT: [paste chapter]
How to judge the result
- Every Changed row is either fixed or deliberately adopted into the plan.
- No Not yet item has appeared.
- New events the draft added are on your plan or removed.
- The ledger takes a few minutes per chapter; if it takes far longer, your beats are probably too vague.
Limits
This ledger checks plan following only. It will not catch a changed eye colour or a wrong date, which is what continuity checks are for. It also cannot tell you whether a chapter is good. And the preprint's numbers describe one research model, not the tool you use.[1][3]
Sources
- Towards Human-Level Book-Writing Capability (arXiv 2605.17064v2)Checked 2026-11-08
- ScribistIQ featuresChecked 2026-11-08
- ScribistIQ guide: AI novel continuityChecked 2026-11-08
- OpenAI community: repetition and task stealing across chapters (May 2025)Checked 2026-11-08