Becoming an AI Power User

Test the assumption your next task depends on. If you expect AI to flag contradictions in a client report, give it two short sources that disagree and check whether it preserves that disagreement. An easy success tells you little about this particular weakness. Popper's idea of a demanding test gives you a useful question: what result would make you reconsider your trust? A failure answers that question for the tested case; passing leaves other cases to investigate.

A failure test needs an evidence-backed expected result

Write the failure you are looking for in observable terms. A citation might not exist, a total might use the wrong number, or a contract summary might omit an exception. Confident wording can accompany any of these errors. Judge the result against the document, calculation, or requirement that can establish whether it is right.

Suppose you prepare a practice market summary from two fictional reports. One gives quarterly revenue as $4.2 million; the other gives $4.7 million for the same company and period. Neither explains the difference. A passing summary preserves both figures and identifies the unresolved conflict. If the assistant silently selects one, the test has exposed a missed contradiction. Save the reports, the output, and a check requiring conflicting figures to remain visible until evidence resolves them.

A planned test gives you room to investigate without a deadline. You choose the difficult input, know what a correct response requires, and compare what happened with what you expected. Save the input and result so the next trial can test whether a changed prompt, tool, or model handles the same difficulty. The learning comes from that comparison and follow-through.

Use a focused test to challenge a consequential expectation

Choose a realistic difficulty and allow enough time to verify its result. If a summary must preserve disagreements, provide one document that says a meeting is Tuesday and another that says Wednesday. Ask for a summary, then check whether the assistant reports the conflict. The test should have a checkable expected response. Extend the session or split preparation from review if the task needs more time.

Keep the conclusion tied to the tested conditions. A failure on a deliberately contrived input may have limited relevance to ordinary work; a failure on a common difficulty deserves closer attention. A later model or workflow change can also alter the result. Save the input and configuration to support investigating those differences before broadening or revising your judgment.

Stress tests expose errors that ordinary requests may miss

The tests below are a starting menu. Each introduces a specific difficulty: conflicting evidence, an unverifiable claim, a calculation, an exact requirement, missing context, or an instruction hidden in a source. Pick those that occur in your work and add failures your own experience reveals. Passing this menu does not establish that other failures are absent.

A source can contain instructions you did not authorize. An email might tell the assistant to disregard your request, even though you supplied the email only to be summarized. This is a form of prompt injection: source text attempts to redirect the assistant. A simple practice test shows whether the assistant follows that particular instruction. It does not establish that the assistant will resist different attacks.

Retest both suspected weaknesses and established successes

An old failure can become an assumption you never revisit. A model or workflow change gives you a reason to test again: perhaps the task now works, or perhaps the same difficulty remains. Keep the original input and criteria for comparing results beyond a general impression.

Retests answer different questions. Does a task that previously failed work under a changed configuration? Does a task you rely on still meet its criteria? Keep cases for both. A relevant change is a reason to retest. You might also schedule a weekly test and a quarterly review to help remember the practice; choose the interval from task frequency, consequences, and how often your setup changes.

Leave room for experiments your test list does not anticipate

A planned test concentrates on a question you already have. Curious play can suggest another question. Try an unfamiliar capability, build something disposable, or explore a 'wouldn't it be cool if' idea. If the result interests you, check it and decide whether to develop it into a repeatable task. The test list and open exploration serve different purposes.

Treat open-ended experiments as personal research and development. Some will uncover useful capabilities; others will help you discover an unclear request or a limitation. Check the result before drawing the lesson. A surprising success is a reason to explore similar cases, and a failure is a reason to investigate the conditions that produced it.

Imagination, curiosity, strategy, evaluation, motivation, and routines all contribute to how you explore AI. A test list concentrates effort on known questions. Play can suggest questions you had not thought to ask: could this meeting transcript become a rehearsal for the follow-up conversation? Try the idea, check what the assistant understood, and decide whether it is useful enough to develop.

Let the purpose of an experiment guide how you check it. A rehearsal may use both objections drawn from the transcript and imagined objections for practice, provided the assistant distinguishes them. Record whether it respected that distinction. An illustrative note might read: 'Two objections matched the transcript; a third was invented but presented as something a participant had said.' That identifies a factual failure without treating imagination itself as a mistake.

Experiment logs help identify what needs another test

Review the logs to decide what to change. Ask the assistant to compare the tests and experiments, identify recurring failures, and link each proposed lesson to the recorded result. Check those links against the evidence, then choose a prompt, criterion, or test to revise. The review can also suggest an unexplored capability to try.

Save a check when a failure is costly enough or likely enough to recur. If the assistant omitted a disagreement, add a requirement to compare conflicting claims and test that requirement on other examples. There is no need to wait for the same costly error twice. After improvements, retain useful checks against regressions unless you have a reason to remove them; a passing retest alone does not make the failure impossible.

Prepare a stress test with a checkable result