For agents that look after shared tools and checks: the small scripts that do the work and the checks that prove it.
What is in it
4 in the kit
Open leverWhen the work isn't trivial, build the tool that does it. A script does the work- Do the first unit by hand to learn the recipe. Then write the tool, rerun it o- Make it safe to rerun (idempotent).- Codemod for edits, generator for repeated files, a query (sqlite, jq, awk) for- One deterministic pass beats handing units out. If neighbours share the work, - Smallest tool that does the job, never a framework. Scratch levers live in the- If you cite this skill and there is no script, codemod or generator in the res
When the work isn't trivial, it has your agent write the script that does it, rerun it on the first unit and diff the result against the hand version.
Open eval-design## Words- Environment: the briefs, skills, rules, harness, strategy, model and effort. W- Experiment: one change to the environment, then runs of real tasks.- Eval: a question and the figure that it produces. Ask the question of many ite- Judge: the one who answers an eval's question: a program, the owner, or a mode- Rung: one model at one effort. Opus medium is the top rung.- Owner: the person whose judgement the work serves.- Lights out: the work runs without a review by the owner. Lights out is the goa## Kinds of eval- Capability eval: a figure that we try to move. Read it as a trend. Do not put - Regression eval: a figure that holds on this rung. It is a gate for each exper- When a capability eval holds reliably, move it to regression.- A gate without capability evals cannot improve. Do not use it until it has the## Write an eval
It turns 'is the agent better now?' into a question asked of many items of real work, with a named judge and a figure you can track.
Open prove-itUnverified work has unknown correctness. Check the real thing directly.- Run the real program from outside and read what it prints (a demonstration, de- Check liveness directly (the process, the listener), not state derived from it- When a check fails, suspect the way you looked before you suspect the system.- Script the check when you can, so a reviewer reruns it instead of trusting you- A script that proves a fix is a demonstration, not a new test. Add a test only- Visual results go to the user, shown inline. Their eyes are the check.
Before your agent says it is done, it runs the real program from outside and reads what it prints, instead of trusting a compile, a cached screenshot or its own word.
Open numbered-library-pagesThe user discusses library pages by pointing at parts of them, so every part neeIn a wiki page in your library, number the headings by their place in the page: This holds for pages written by hand. A generated page (kind: generated) gets it
Mackenzie discusses library pages by pointing at parts of them, so every part needs a number that names it.
lever: When the work isn't trivial, it has your agent write the script that does it, rerun it on the first unit and diff the result against the hand version. eval-design: It turns 'is the agent better now?' into a question asked of many items of real work, with a named judge and a figure you can track. prove-it: Before your agent says it is done, it runs the real program from outside and reads what it prints, instead of trusting a compile, a cached screenshot or its own word. numbered-library-pages: Mackenzie discusses library pages by pointing at parts of them, so every part needs a number that names it.