It Runs. That's Not the Same as It Works.
The code came back, and it looked right.
That is the part nobody warns you about. It was indented properly. The names made sense. There were comments. It had the confident shape of something written by a person who knew exactly what they were doing. I read it, I nodded, I kept it. And somewhere between three minutes and three weeks later I found out it had been wrong the whole time.
The suspicion usually shows up before the evidence does. You get a feeling that something is off, and no vocabulary for what.
Also, you may never look at the code. You just look at the results. The app launches, but something feels off. It seems slow in opening on your brand new iPhone.
The inspector isn't smarter than you
Buying a house is the closest thing I can compare it to. The house is finished. Somebody built it. The home has walls and a roof, and the doors that close. You walk through, and it looks like a house, because it is one.
Then the inspector walks through the same house and finds four things you didn't. He isn't smarter than you, and he isn't even a builder. He has a prepared list of things he looks for a list he learned and then over time refined for himself. He checks the same specific things in the same order every time, and the list is the entire job. He knows on two-story homes to check the top of the stairs. But on a single-story home, he checks the back door to see if the roofline lets water into the back door.
The AI knows code. You know the vision or the idea behind the code. I rarely look at code unless there is a reason. What I am looking at is whether it does what I want, and whether it does it reliably. I have a list of things I expect, and when I don't see them I ask the AI about them. Like the inspector, I developed a list over time that I know I may forget to ask or know that, as I talk to the AI, I may not fully explain that I want this.
Yes, I just said I may not. Everyone, and I mean humans, tries to explain things, and though we use the same words, the meaning is often not fully transferred. Sure, most of the time it doesn't matter, but in some cases it does. Talking ot AI is the same; The AI only knows what you tell it and how specific you are about it.
A note on the order
This is the second batch of words in the series. The first batch covered the things that bite before you type a prompt. These six are for the half hour after code comes back.
I've also put them in the order you'd actually use them, which is not the order I first wrote them down in. That's a habit, not a law. If you've been doing this a while, two or three of these are already automatic for you, and the word you actually need might be in a batch I haven't written yet. Take what's useful.
The six words
1. Smoke test
Classic meaning: the thirty-second "does it even start?" check. The name comes from hardware. Power it on and see whether smoke comes out. Does the logo or splash screen come up?
What changed: the position in the sequence. Smoke testing used to be a formality you did after review. Now it goes first, before you read a single line, because reading two hundred lines carefully and then discovering the thing doesn't compile is a waste of the one resource you have least of, which is patience.
Run it and see what happens. Is there an error message? Give that to the AI.
What to do: run it before you read it. If it doesn't start, the review is moot, and you've saved yourself twenty minutes of studying code that was never going to survive. You may also never read the code, because you don't know how to read it. Ask the AI to explain the code to you if you feel that will help.
2. Diff
Classic meaning: the exact lines that changed.
What changed: the diff became the main thing you read. When you write code yourself, you already know what you changed and the diff is a receipt. When the machine writes it, the diff is the only honest account of what just happened to your project. Reading diffs rather than files is the core reading skill of this whole way of working, and it's the reason the first guardrail I ever wrote down was about diffs.
Again, you may not ever read the code, so asking the AI what the Diff is may not help. What you could do is say that the program ran but did this, and you want it to do this.
The prompt:
Walk me through this diff one change at a time. For each one, tell me what it does and what would break if I removed it.
Again, this is helpful if you actually read the code.
3. Sanity check
Classic meaning: a quick, informal "does this even make sense?" glance. Cheap, fast, catches the absurd.
What changed: what you're glancing at. The sanity check is now about the shape of the change, not its content, and you can run it in four seconds without understanding any of the code.
You asked for a fix to a feature in your app. How many files did it touch? If the answer is forty, you have found the problem already, and you haven't read anything yet.
What to do: before you read, look at the size. Number of files, number of lines. If the number surprises you, that surprise is the finding. Ask why before you ask what.
4. Happy path vs. edge case
Classic meaning: the expected flow versus the weird inputs. The happy path is a user doing the thing you imagined. The edge case is everything else.
What changed: the machine writes a beautiful happy path and treats the rest as optional. It isn't being lazy. You described the thing you wanted, so it built the thing you described, and neither of you mentioned what happens when the field is empty.
The prompt:
What are the three most likely inputs that break this? Include empty, very long, and unexpected characters.
The ones that get me are boring. A blank field. An apostrophe in a name. No network. A date on the last day of a month.
5. Code smell
Classic meaning: Martin Fowler's phrase, and his definition is better than any paraphrase: a surface indication that usually corresponds to a deeper problem. He's careful to say a smell is quick to spot and doesn't always mean something is actually wrong. It means look closer.
What changed: the specific smells. The new ones are configuration options nobody asked for, an abstraction layer built for a future that isn't coming, and old code left sitting next to the code that replaced it. That last one is everywhere. The machine writes the new version and doesn't always delete the old one, and six weeks later you can't tell which one is live.
What to do: when something reads wrong, say so out loud instead of talking yourself out of it. "This function is doing four things" is a complete and useful review comment, and you don't need to prove it's a bug first.
6. Regression
Classic meaning: a change that breaks something which used to work. A regression test is the test that catches it.
What changed: this is the one I'd build the habit around first. The model has no memory of the bug it fixed for you last Tuesday. Start a new session and it will happily solve today's problem in a way that quietly undoes last week's, and it will be just as confident both times. You are the only continuity in the project. Nothing else remembers.
The prompt:
What existing behavior could this change break? List anything that reads or writes the same data, or lives in the same file.
None of it works without a goal
Six questions only mean something if you know what the code was supposed to do. "Is this any good?" is unanswerable until "good" points at something specific. Which brings me to the thing that actually changed my results, and it isn't on the list above.
For a long time I wrote code in order to learn code. Exercises, tutorials, follow along and see what happens. Almost none of it stuck.
Then I came up with an actual app. Not a lesson, a thing I wanted to exist. I laid out what it should do and started building, and I learned more in that one project than in everything that came before it, because every question I asked had a reason behind it. I wasn't asking to learn the answer. I was asking because I needed the answer to keep going.
That app is still not finished. It turned out to be considerably more complicated than I judged it to be, which is its own lesson, and one I'd rather have learned by walking into it than by reading a warning about it. I got far enough to hit real frustration, and then I made the decision I'd recommend to anyone in the same spot: I saved it and picked something smaller.
The smaller things were browser extensions. Three of them were accepted and published in about two weeks. Those were the first things I ever actually shipped. The ambitious idea that started all of it is still sitting there unfinished, and three simpler ideas that came after it are live and working.
So start. And start with something you want, or need, or would be glad existed, rather than a random exercise. The goal is what gives the work focus. It's also what makes every question in this post answerable, because you cannot inspect a house when nobody can tell you what it was supposed to be.
The half hour, in order
Six questions. None of them require you to be able to write the code yourself.
- Does it run?
- What exactly changed?
- Is this change the size I expected?
- What input breaks it?
- What in here would I not have written this way?
- What did this just break that used to work?
Comments
Post a Comment