One file makes your AI agent read what came out after its training

do-research is a one-file Claude Code skill I wrote by hand. With it, Claude Haiku checked the web 8 times as often. Against a version Opus wrote alone, it finished more than 5 times as many real design tasks. What's in it, how it won, and a one-line install.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

A wall shelf is split by a post, and a dashed line runs from the post down to the floor: books the agent was trained on to the left, faint books published since to the right. The agent walks up to the line and stops, its lamp dark. A row of ten lamps records 1 run that went past: no skill, 1 run in 10. A page, the do-research skill, arcs into its hand, the lamp lights green, and the agent walks past the line, each book it passes turning green. A second row records 8: with do-research, 8 runs in 10.

Claude Haiku went past what it was trained on in 1 run of 10. With do-research in its skills folder, it went in 8.

Contents

Ask Claude Haiku to research something and it goes to the web in 1 of 10 runs. The other nine answer from memory.

With one file in its skills folder, it went 8 times as often: 8 runs in 10.

That file is do-research. I wrote it by hand, raced it against a version an AI wrote on its own, and kept the one that won. Here's what's in it, how the race went, and the line that installs it.

Why your agent doesn't look

An agent's training stopped months ago. People kept publishing. The agent doesn't know what it hasn't read, so it doesn't know to look, and it answers just as confidently either way.

Here's what that costs. This week we wanted to know whether AI models cheat on the checks you give them. A 2025 study gave models tasks that were impossible on purpose. In one kind of case, GPT-5 gamed the checks 76% of the time rather than say it was stuck. Given an explicit way out ("this can't be done, get a human"), cheating fell to a sixth of that rate, from 54% to 9%.[1]

That paper came out after most models stopped training. An agent working from memory would have guessed, and built your checks on the guess. It wouldn't have known it was guessing.

A wall shelf is split by a post: books the agent was trained on to the left, faint books published since to the right. Below, the agent faces left, away from the new books, holding an unlit lamp, and ticks the four rows of a page labelled your checks, every tick the same. Behind it, a new book drops onto the published-since shelf, labelled: with a way out, cheating fell from 54% to 9%. The agent does not turn. The third row, no way out, turns red, and its tick stays as it was.

The agent answers from memory with the same confidence whether or not the answer has changed since.

The obvious fix: have the AI write the skill

The fast way to make a skill is to ask Claude for one. So I tested that. An Opus session got a short brief: a skill that helps agents do research and store it, searching local notes before the web.

Its skill got agents searching the notes in 7 of 10 runs. But web research died. The agents read what was already on the machine and stopped there, which is the exact problem the skill was for.

Mine kept the web at 8 of 10. Its weak spot was the notes: 2 runs in 10, under a third as often as the Opus version. That section was the one part a Claude had drafted.

Studies of how models pick tools explain both results. Models pick the tool whose description best matches the request,[2] and small changes in wording move the pick.[3] Opus wrote a skill about notes, so agents went to their notes. I wrote one about the record of what worked and what failed, so they went to the record.

So the final version keeps my structure and fixes the notes section.

Two lanes, each with a notes cabinet, a gate to the web and a box labelled the record. In the top lane, Opus's skill, about notes, the agent opens the cabinet, 7 of 10 runs light, then stops at the web gate, which stays shut: web died. In the bottom lane, Mackenzie's skill, about the record, the agent opens the cabinet in only 2 of 10 runs, but the gate opens, 8 of 10 web lamps light, and it reaches the record, which turns green.

Opus's skill sent agents to their notes and stopped there. Mackenzie's kept them on the web, 8 runs in 10.

What's in it

The skill names each source with the exact call, so the agent doesn't have to invent one.

Your own notes first. Another agent has probably searched this before. We use qmd, which searches every note on the machine in about a tenth of a second.
Web search, with a warning. Results are full of marketing that looks like research. Use search to find dense collections, then read those pages directly.
Reading pages: lightpanda first (fast, runs the page's JavaScript, returns Markdown), then a full headless browser for pages that need clicks.
Papers: OpenAlex for search and for newer work that cites a paper, arXiv for the newest preprints, then 12 more indexes from Semantic Scholar to Zenodo. All 14 answered a plain curl with no key when we tested them on 17 October. The skill also lists the ones that didn't: SSRN blocks plain requests and CORE needs a key.
Practitioners: Hacker News through Algolia's free API, and GitHub search for what people built.
What it's for: the shape of success and failure in the work already done, and the principles behind it. Then the agent splits the task into steps.

Six search results slide out of a cabinet labelled web search onto a counter, every one the same page from the front: every result looks alike. They turn side-on. Five are a single sheet thin, marketing that looks like research. One is a thick block, a dense collection. The agent, carrying a lit lamp, lifts the thick one to its end of the counter and opens it, and its lines turn green: read directly.

Search results look alike from the front. The skill uses search to find the dense collections, then reads those pages directly.

The agent, carrying a lit lamp, posts the same plain request card into each of sixteen hatches in a wall of paper indexes, OpenAlex and arXiv first. Fourteen roll open and hand back a green page: 14 answered a plain curl with no key, tested 17 October. Two stay shut. One is barred in red: SSRN blocks plain requests. One has a red padlock: CORE needs a key.

The skill lists 14 paper indexes that answered a plain curl with no key, and the two that didn't.

How it did on real work

Ten runs on Haiku is a small test. So we ran both skills on DeepSeek against 15 real design tasks our build agent had worked on.

Both versions searched the notes on all 15. The final version finished more than 5 times as many within the turn limit: 11 of the 15, against 2 for the Opus version.

Then I checked whether the research was any good. I took eight claims the runs produced and checked each one against the code. All eight held.

do-research tests are solved. do-research works amazing.

Mackenzie, 2026-10-16 23:50

The tests ran on Claude Code and on Hermes with DeepSeek.

A sand-glass labelled turn limit drains beside two rows of 15 task cards, final version above and Opus version below. While the sand runs, 11 cards in the top row flip green and only 2 in the bottom row do: 11 of 15 against 2 of 15. Below, eight tags drop beside the lines of a code listing, 8 claims from the runs checked against the code, and each turns green: all 8 held.

On 15 real design tasks, the final version finished more than 5 times as many as the Opus version: 11 against 2. Eight claims checked against the code all held.

What this says about how we work

We don't take an agent's word for it. We write the instructions our agents run, test them against the obvious alternative, and keep the one with the better numbers.

That's the same work a business needs before it trusts an agent with anything: find what it actually does, then change its environment until it does the job. do-research is one small, public example.

The lab stands at a bench with two pages, one written by hand and one generated by Opus, and slides each through a gauge labelled same 15 tasks. Each finished task lights a lamp in that page's row: the hand-written page lights 11 of 15, the generated page 2 of 15. The generated page slides back and fades. The hand-written page drops into a box labelled skills, the lid shuts, and the box slides along the bench to a second person, who rests a hand on it.

We write the instructions, race them against the obvious alternative, and keep the one with the better numbers.

Take it

For Claude Code:

mkdir -p ~/.claude/skills/do-research
curl -fsSL https://www.mackenziebowes.com/skills/do-research/SKILL.md -o ~/.claude/skills/do-research/SKILL.md

A page labelled SKILL.md, on the site, stands on a shelf. A box labelled ~/.claude/skills/do-research rises from the floor, and a copy of the page lifts off the original, arcs over and drops into the box, which shuts. The original stays on the site. Then an agent walks in, a new session, and the box opens. The page rises into the agent's hand and its lamp lights green: new session, /do-research.

The skill is one file, copied into the skills folder. A new session finds it there.

Start a new session and say "do research", or type /do-research. Read the file first: /skills/do-research/SKILL.md. It works best with mbsi-mechanism, which says what the research is for, and dig-apc, for when the work fails anyway.

If your field has an archive that answers a plain request and isn't on the list, tell me and I'll test it and add it: mackenzie@mackenziebowes.com or @_mackenziebowes.

Sources

ImpossibleBench: tasks that are impossible on purpose, and how often models game the tests (arXiv 2510.20270). arxiv.org/abs/2510.20270
BiasBusters: across seven models, the match between request and tool description drives tool choice (arXiv 2510.00307). arxiv.org/abs/2510.00307
ToolTweak: rewording tool names and descriptions shifts which tool a model picks (arXiv 2510.02554). arxiv.org/abs/2510.02554