A proposal can preserve the dollar amount and still change 'approval pending' into 'approved.' If your first review misses that change, test a second opinion. Give a candidate editor examples with verified errors and examples that should pass, then check whether its findings would help you make a sound revision.
Becoming an AI Power User
Becoming an AI Power User
198 min leftPage 115 of 223 (est.)
3.3Decomposition and Parallel Work
They test a separate reviewer before trusting it on important work
Developing
Fresh context changes the review; shows whether it improves it
Ask a model to write something, then ask either that session or a fresh one to review it. Those are different review conditions because one has the drafting exchange and one does not. The separation itself says nothing about accuracy. Run both conditions against the same blinded case pack if you want to know which one makes fewer consequential errors on your task.
Build a set from the kind of draft you plan to review. Include errors you have verified and examples that meet the criteria. If you lack a suitable failure, create a test copy by deliberately altering a claim whose correct version is established from its source. Label it as test material in your records, and give the reviewer a neutral case ID. Keep expected findings in a separate answer key. For a close case, record what remains uncertain and what evidence would resolve it. Run candidates with the same criteria and source access, then count missed problems and false alarms separately. A separate preparation conversation can help assemble the examples and answer key. Check its proposed errors against the sources, then give only the cases, sources, and review criteria to the fresh editor conversation. Keep the answer key in the preparation conversation or a separate document.
Use checks that inspect different evidence. A source comparison asks whether claims match the record. A deterministic test evaluates an exact rule. A candidate editor evaluates the criteria in its prompt. Choose the evidence source for the failure, then measure whether the reviewer recognizes it on blinded cases.
Ask the editor to evaluate the draft, not to manufacture faults
Give the editor explicit criteria and ask for met, falls short, or unclear on each. Every proposed failure needs a quoted passage and a source comparison or explanation showing which criterion it violates. A quotation locates the passage; the comparison or reasoning supports the verdict. If the editor lacks the source needed to check a claim, it should report what remains unclear and request the missing evidence. Missing evidence alone does not establish that the claim is false. Allow a clean result and omit unsupported accusations.
Consider a client proposal with this source note: 'Proposed pilot budget: $3,000. Approval pending.' The draft says: 'The client approved a $3,000 pilot.' A useful editor identifies the exact difference: proposed has become approved. A suitable finding quotes both lines and asks for the commitment to be corrected. 'This sounds too confident' is less useful because it leaves the underlying factual change unnamed.
Include a clean counterpart in the set: "The proposed pilot budget is $3,000, pending client approval." The reviewer should leave that claim alone. Keep which case is flawed in your separate answer key, and give the reviewer each draft with its source note. A useful check catches the altered commitment without objecting to the accurate sentence.
- Draft with the first model. Give your preferred model full context, who it is writing as, who will read it, the goal, the constraints, and any source material, and let it produce the first version. Briefing and review are separate controls: the brief governs what the writer receives, while the editor's criteria and source access govern which claims it can evaluate.
- Review with the calibrated candidate. Start a fresh context, provide the draft, audience, sources, and criteria, and ask for met, falls short, or unclear on each criterion. Require a quotation or source comparison for every failure.
- Verify, revise, and save. Check each proposed problem against the draft and its sources. Revise only confirmed findings. Save the criteria, cases, and editor prompt together so a later model change can be retested against the same contract.
Request evidence-backed findings before asking for a rewrite
Ask for findings before a rewrite when you need to understand the proposed changes. A critique can identify the altered budget commitment while leaving sound paragraphs alone. Once the finding is verified, ask the writer or another assistant to repair it. Keeping diagnosis and revision distinguishable helps you inspect what changed; it does not require you to make every edit by hand.
Choose review effort from the consequences of an error
Match the review effort to the consequences of an error. A client proposal, research report, or published recommendation may justify testing a separate reviewer and verifying its findings. A routine internal note may need a lighter check. Compare the expected benefit with the extra work, without assuming either one model or two is accurate enough by default.
Ask what a missed error would change. An incorrect figure in a budget recommendation deserves a source check even when the prose looks fine. An awkward sentence in a disposable brainstorming note may not warrant another pass. Choose the check that addresses the consequence.
Calibrate the editor before handing it the important draft
Test and use an evidence-backed editor
Prepare draft excerpts and their sources under neutral case IDs. Keep your answer key and acceptance limits outside a fresh Claude conversation, and paste this prompt there to test the editor.
- Copy the promptUse the copy button above the box.
- Paste it in Claude or CodexMake sure your active project is enabled.
- Answer the questionsGive clear answers so it understands what you want made.
- Test it to see if it worksTry the output and refine until it does what you need.
Problem: A second AI reviewer may miss errors or flag sound writing, so its findings need to be tested against evidence.
Intent: I want to test you as an editor before using you on an important draft. Evaluate the material I provide without assuming it is flawed, and do not rewrite it.
Ask me for the deliverable type, intended audience, and calibration cases. Each case should contain a neutral case ID, a draft or excerpt, and the sources and constraints you would receive in ordinary use. Request missing inputs needed for the review. Do not ask for labels, expected findings, or my answer key.
If I have no cases ready, explain what to prepare outside this conversation: representative drafts with verified problems, clean examples, and a close case where the evidence may be insufficient. Any deliberately altered example must remain identifiable as test material in my own records. I will keep the expected findings and my acceptance limits in a separate document. Do not invent that material for me or infer my limits.
Use these criteria:
1. SUPPORTED CLAIMS: factual claims match the supplied sources, or are clearly qualified when the evidence is incomplete.
2. SPECIFICITY: the wording identifies the subject, audience, action, evidence, and consequence as needed for the passage.
3. NON-REPETITION: paragraphs contribute information, explanation, or useful consolidation.
4. COMPLETE REASONING: the reader has the reasoning needed to follow each inference or transition.
5. SUPPORTED CONCLUSIONS: conclusions follow from the evidence shown.
6. AUDIENCE FIT: tone, detail, and assumed knowledge fit the intended reader.
First, evaluate only the calibration cases. For every criterion on every case, return met, falls short, or unclear. A falls-short result must quote the passage and identify the supplied source comparison or reasoning that establishes the problem. When evidence is missing, state what cannot be determined and what information is needed. Omit unsupported accusations.
Return a case-by-case table of findings and identify any case that needs another source or qualified expertise. Stop after this run. Do not ask me to reveal the answer key or use guesses about hidden labels as evidence.
I will compare your findings with my answer key outside this conversation, counting known failures caught, known failures missed, clean cases correctly left alone, and false alarms. If I revise these instructions, I will test them on fresh blinded cases before treating the change as an improvement.
If I then choose to use you on an important draft, ask for the draft, its sources, and any additional constraints or settled decisions. Evaluate it with the same criteria and reporting format. Finish with evidence-backed shortfalls and unresolved checks in separate lists. I will decide which findings to verify and fix.
References
1 source- 1
Guneet Kohli · 2026 · Apple Machine Learning Research
Across nine frontier judges from seven model families, correlated mistakes reduced the panel to roughly two independent votes and left its accuracy 8 to 22 percentage points below the independent-voting ideal.
Using another model family creates a candidate reviewer, not independent evidence. Representative labeled cases must establish whether the reviewer adds useful coverage for the task.
The study tested natural-language-inference and preference evaluation datasets. A writing workflow should borrow the measurement principle, not assume the same numerical result.