There’s a familiar relationship we have with software updates. The application asks to install one, we’re in the middle of something, and we click “Remind me later.” Eventually, someone sends us a file we can’t open, and the update we’ve been avoiding becomes the thing we have to do before we can get on with our work.
“Later” turns out to be just before a meeting.
With AI, the update can happen without us doing anything. The service changes the model, we carry on using the same application, and the prompts we’ve saved stay exactly as they were.
Those prompts may still produce perfectly usable results. But some of the instructions could have been written to deal with difficulties an earlier model had. We keep explaining things in the same way without checking whether the newer model needs that explanation, or whether a different approach would work better.
Your business, your audience and the result you want may all be the same. As models change, it’s worth reviewing whether the way you communicate those things is still the most effective.
Keep the business brief intact
I’m not suggesting that a model update is a reason to rewrite the description of your business or yourself. The model still needs to know who the customer is, what you’re offering, what you’re trying to achieve and which requirements it has to follow.
What can change is how much explanation the model needs to understand what you mean.
Take a prompt for writing a campaign email. You might have built it around several examples of good copy, followed by examples of the things you don’t want. Perhaps you added those because an earlier model kept getting the tone wrong, and showing it acceptable and unacceptable versions was the way you got a useful result.
A newer model may understand the same requirement from a clear description, without needing all those examples. That’s something you can test while leaving the campaign brief and the requirements unchanged.
For example, the instruction might explain that the email is going to an existing customer who already knows the product. It should get straight to the new feature, explain what that feature lets the customer do, and avoid introducing the company again. An older version of the prompt might use several sample openings to get that point across. A revised version could state the requirement directly and see whether the model follows it.
The question is whether the examples still help with that task. If they communicate a distinctive voice that you haven’t captured in words, removing them could make the result worse.
Anthropic’s model-specific guidance describes cases where brief instructions can replace longer lists of behaviours. Its general prompting guidance still recommends examples for steering tone, format and structure. I’d use the guidance to decide what to try, then judge the results on our own work.
Compare the prompts on the model you’re using
When a company releases a new model, it will usually publish documentation explaining how to use it, including prompting guidance. Start there. Run the existing prompt, revise it using the guidance for that model, then try the new version and see what the difference is.
For the campaign email, give both versions the same audience, product information and campaign objective. Run each version in a separate fresh chat, using the same model and supplying the same brief and background information. If you compare the old prompt on an old model with the new prompt on a new model, you won’t know whether the prompt change helped.
I’d use a few briefs from the kind of work the team regularly does, and repeat the comparison rather than choosing whichever prompt happened to produce one good email. Check whether the copy follows the brief, gets the product details right and sounds appropriate for the customer. Look at how much editing you would need to do before using it.
Keep the standards the same for both versions. A shorter prompt that produces a more fluent email but drops an important qualification hasn’t solved the problem.
You also need to keep the original prompt. A dated copy in a shared folder is enough to start. Record which model you tested, what you changed and why you chose the version you’re keeping. That gives you a way to reverse the change and something useful to refer to at the next review.
If the existing version works just as well, keeping it is a reasonable result. There’s no requirement to find something to change.
Give the review a manageable cadence
I wouldn’t turn every model announcement into a project for the marketing team. Start with the models you use, and give one person responsibility for checking the relevant prompting guidance when there’s a major change. Commentary from people testing the model can suggest useful variations too, provided you try them on your own tasks before adopting them.
If following releases isn’t practical, a couple of reviews a year gives you an occasion to check what has changed. Begin with the prompts you reuse regularly, where a useful improvement will apply to more than one piece of work.
To get started:
- Pick one prompt your team uses regularly and save a dated copy.
- Read the current prompting guidance for the model you’re using, and make a revised version while keeping the business brief intact.
- Run each version in a separate fresh chat, using the same model, briefs and background information. Compare accuracy, tone, adherence to requirements and how much editing the results need.
- Record which version you’re keeping and why, then set the next review for a major model change or a few months from now.