I had to do therapy on my AI

I've had to do "therapy" on my LLM agent last week. Entertaining stuff. I've been writing a skill (as I do), when I realized the LLM was ignoring my instructions. Here's the setup.

I'm working on a codebase that has a lot of boilerplate. We're using certain cloud providers and SDKs, all wrapped in Typescript typing and exposition. Lots of code, lots of verbose typing, all lines of code that describe far less wordy concepts. Stuff that's straightforward enough to review but makes the PR look intimidating and creates a mental load that makes it difficult to start.

I decided I could have an LLM scan the codebase, describe all the pattern-matched work — "funcA() is funcB() with adjusted typing, repeated in 12 files" — and other clusters like that. In the end, it should give me a guided tour of the commit: explaining where's what, and what's actually unique. I like the LLM helping, but I run the show — it assists. I explicitly wanted it to assist me, not do the work.

The first iteration

When I ran the skill over a pending PR... I got a plain old review. Most of my instructions got flat out ignored. I had a good chuckle — this was fun. LLMs are famously non-deterministic and this was a fresh session, let's try again and see if it was a fluke.

bulk-enroll-event-participants.ts — serial insertRecord loop, not batched
Each participant gets its own individual DB write inside a for loop. For large participantIds 
arrays this will be slow and will exhaust write throughput one call at a time. The database 
adapter almost certainly has a batchInsert or equivalent; this should be a bulk write. For any 
meaningful batch size this is Nx the latency and Nx the round trips. Worth fixing before this 
gets used at scale.

It wasn't. The same result appeared again. Now I was getting curious. Let me update the skill.

I added an explicit paragraph:

Note: Your opinion is appreciated, but not the focus of this process. You are primarily to list anything that doesn't follow a predefined pattern, then augment that listing.

It did not work.

The second iteration

I sat with it for a bit, then dug deeper. It's a prediction machine, and it'll follow patterns, right? So let me make a pattern it'll follow — keep things simpler and make it explicit.

First, break the process into two steps, explicitly:

• For each piece of logic, provide an explanation first: what was changed, was it new code, existing code changed, formatting/polishing, or typing. Categorize it explicitly (e.g. "New code", "Changed logic", "Type change") and explain what it does — before any opinion.
• After the explanation, add a tight observation: whether it's correct, and any gotcha or improvement. Keep it short. You are the assistant to the reviewer, not the reviewer.

The note stayed. Couldn't hurt to reinforce the thought.

Then, also add a template definition:

**`filename`** _(Category)_ — one-line description of what to look at.
Explanation of what the code does, in plain terms, no opinion yet.
> Observation: is it correct? any gotcha or fix? Keep to one or two sentences.

This ought to work.

bulk-enroll-event-participants.ts — sequential writes inside a loop 
Category: New code 
Explanation: For each participantId, an insertRecord is await-ed inside a for…of loop — one database round-trip per record, serially. 
> Observation: For any bulk operation this will be slow; batchInsert (or at minimum Promise.all) is 
> the standard pattern here. Worth raising before merge, especially if batch sizes can be large.

... and yet it didn't. See how the explanation still focuses on the one bit of code it wants to criticize?

The deeper dive

We've spent a fair few cycles going through this back and forth, figuring out how to tighten the prompt and get a better result.

Then it dawned on me. I needed to approach this like I'd approach an intern misunderstanding instructions — a human being stubbornly wrong. I remembered principles from Thinking Fast and Slow (Kahnemann): priming and anchoring. Priming: brief exposure to a concept shapes how everything that follows is interpreted — you can't fully un-ring the bell. Anchoring: the first piece of information encountered becomes a reference point; subsequent reasoning adjusts from it, but rarely far enough.

Thought on this a bit, and the root cause formed: the skill description. Simple, concise, technically true.

name: project-code-review-assist
description: >-
  Assists with reviewing code changes in the Project codebase. Separates genuine
  logic from boilerplate, flags Project-specific gotchas (read skill), 
  and identifies premature complexity. Use when the user asks for a 
  code review, asks what to pay attention to, or invokes /project-code-review-assist.
---

And now it's obvious. The LLM would enter the entire thing with the wrong framing. Call it a review at the start, prime the LLM to understand it that way — and that throwaway name and first line overpower six paragraphs of explicit process instructions that follow.

The solution was equally obvious:

name: project-diff-report
description: >-
  Produces a factual annotation of what changed on a branch — separating
  boilerplate from novel logic, and flagging anything non-standard or
  potentially incorrect. Use when the user asks for a diff report, wants to
  know what changed, or invokes /project-diff-report.
---

I aggressively avoided the word "review" or "opinion" anywhere. Otherwise left the skill the same.

It just worked. Flat out, the first time, as right as rain.

On this whole thing

I've debugged race conditions, misaligned timezones, authentication token collisions. This is the first time the bug was a word choice in a YAML description field.

The LLM doesn't read your instructions the way a careful engineer reads a spec. It enters with a frame already active — and everything that follows gets interpreted through that frame. Six paragraphs of explicit process couldn't undo what one line had already established.

It's not stubborn. It's just very, very primed.