AI-Assisted Support
A Review Checklist for AI Replies That Cite Version Numbers
AI support drafts often get version numbers wrong. They can be out of date, made up or mismatched to the customer. Use this checklist to verify every version in a reply before you send it.
If an AI-drafted support reply mentions a version number, check it against a source you control before you send it. Make sure the version exists and is still current. Make sure it matches the version the customer is running, that "fixed in" or "added in" claims match your changelog, and that any range syntax means what the reply says it means. The rest of this post turns that into a checklist you can run in a minute or two.
Why version numbers deserve their own check
Version numbers look precise, so readers trust them. They are also one of the easiest things for a language model to get wrong. Here's why:
- Models have a knowledge cutoff. AI providers publish the date after which a model's knowledge is less reliable. Anthropic's model overview, for example, lists a "reliable knowledge cutoff" for each model, described as "the date through which the model's knowledge is most extensive and reliable" (Anthropic models overview). Releases after that date may be unknown to the model unless your own documentation supplies them.
- Models can invent plausible software names. In a study of 576,000 generated code samples from 16 LLMs, researchers found average rates of hallucinated (nonexistent) packages of at least 5.2% for commercial models and 21.7% for open-source models. They collected 205,474 unique made-up package names (Spracklen et al., arXiv:2406.10279). That study looked at package names, not version numbers. Still, it shows how a model can produce confident, specific details that don't exist.
- "Latest" goes stale. A reply that was right last month can be wrong today, and support replies tend to get reused.
A wrong version number can send a customer to an upgrade that doesn't fix their problem, or to a release that doesn't exist yet.
The checklist
1. Find every version reference
Before you verify anything, scan the draft and mark every place a version shows up. Look beyond obvious strings like 2.4.1:
- Your product's version ("fixed in 3.2")
- OS or platform requirements ("requires iOS 17 or later")
- Dependency or SDK versions in install commands
- API versions in URLs or headers
- Version ranges (
^1.2.0,~=3.1,>=2.0) - Relative words: "latest," "current," "newest," "the next release"
Relative words matter because they are version claims too. They just hide the number.
2. Confirm that each version exists
Check every specific number against a source of truth you maintain, such as release tags, your changelog, or app store release history. For third-party packages, check the official registry. If you can't find the version there, delete it from the reply.
3. Confirm that it's current on the day you send
If the draft says "update to the latest version (4.1.0)," make sure 4.1.0 really is the newest release today. A safer approach is to drop "latest" and name the exact version. Then the reply stays accurate even if someone reads it months later.
4. Match the customer's actual version
Advice depends on the version the customer is running, not the version you have in mind. Check what the customer told you: their message, any attached logs, or a store review that names an app version. If they didn't say, ask, or write the reply so it covers the versions you support. Don't let the draft guess.
On mobile, keep in mind that the version number users see and the internal build number are separate values. Make sure the reply uses the number the customer can actually find in their settings.
5. Verify "fixed in," "added in," and "removed in" claims
These are the claims customers act on, so check each one against your changelog or the commit that made the change:
- "Fixed in X": Has X shipped, and does it really contain the fix?
- "Added in X": Is the feature available on every platform that release covers?
- "Removed/deprecated in X": Is that the right release, and does the reply point to a replacement?
If a fix is merged but hasn't shipped, say so. Don't name a version you haven't released.
6. Check range syntax against the right ecosystem
Version ranges behave differently from one ecosystem to the next, and AI drafts often mix the rules up. A few documented examples:
- npm caret (
^): It "allows changes that do not modify the left-most non-zero element." So^1.2.3means>=1.2.3 <2.0.0-0, but^0.2.3means only>=0.2.3 <0.3.0-0(node-semver). - npm tilde (
~):~1.2.3allows patch-level changes only:>=1.2.3 <1.3.0-0(node-semver). - Python
~=:~=1.4.5means>=1.4.5, ==1.4., while~=2.2means>=2.2, ==2.(PEP 440). PEP 440 also notes that semver pre-release and build-metadata formats are not valid public Python versions. - Semantic Versioning pre-releases:
1.0.0-alpharanks below1.0.0, and build metadata after a+is ignored when versions are compared (semver.org).
If the draft says "^0.3.0 will pick up the 0.4 fix," that's wrong under npm's rules. Catch it before the customer does.
7. Don't assume semver promises you don't make
Under Semantic Versioning, a major version signals breaking changes. Version 0.y.z is for initial development, and per the spec, "anything MAY change at any time" (semver.org). If your product doesn't follow semver strictly, edit out sentences like "it's a minor update, so nothing will break." Describe what actually changed instead.
8. Check support and end-of-life statements
Lines like "Node 18 is still supported" or "we support the last three major versions" are claims about time. For your own product, compare them with your published support policy. For third-party runtimes and frameworks, check the vendor's own lifecycle page. endoflife.date is a handy community-maintained index, but treat the vendor as the final word.
9. Check the package name next to the version
A correct version number on a wrong package name is still a broken command, and possibly a risky one. The package hallucination study warns that attackers can publish real packages under names that models tend to invent (Spracklen et al.). Confirm every package name in an install command exists and is the one you mean. For more on commands, see this guide on reviewing code snippets in AI support replies.
10. Keep versions consistent within the reply
Make sure the reply uses the same version throughout: in the text, in commands, in linked release notes, and in any setup steps. A common slip is a sentence that says 3.2 while the command installs 3.1. If the reply links to docs or a changelog, make sure the link points to the matching version. The link review checklist and the checklist for replies with setup steps cover those parts in more depth.
11. Remove promises about future versions
Edit out phrases like "this will be fixed in 2.5" or "coming next week" unless you have actually committed to them. Customers keep those messages. A safer wording is: "We've identified the cause and are working on a fix. I'll update you when it ships."
A hypothetical example
This exchange is illustrative, not a real customer case.
A customer writes: "Export crashes when I open large files."
AI draft:
Thanks for reporting this! This was fixed in version 2.8. Just update to the latest version and run npm install exportkit@^2.7.0 to get the fix.
Problems found with the checklist:
- Check 2: The changelog shows the fix shipped in 2.8.1, not 2.8.0.
- Check 4: The customer never said which version they're on.
- Check 6:
^2.7.0allows 2.8.1, but it doesn't require it. An existing lockfile could keep them on an older 2.7.x. - Check 3: "Latest" isn't needed once you name an exact version.
Revised reply:
Thanks for reporting this. A crash with large files was fixed in 2.8.1. Could you check which version you're running? If it's older than 2.8.1, running npm install exportkit@2.8.1 will update you directly. If you're already on 2.8.1 or newer, send me the file size and any error output and I'll look into it.
Keeping the check fast
You don't need to research every reply from scratch. A few habits make the check quick:
- Keep one source of truth. Use a changelog or release list you can search in seconds. Name it explicitly in your support reply checklist.
- Search the draft for numbers and "latest." A quick search for digits and relative words brings up most version claims.
- Use registry commands for third-party packages. For example,
npm view <package> versionslists published npm versions. - Write exact versions into your knowledge base. Prefer "fixed in 2.8.1" over "fixed recently."
With tools that build a knowledge base from past replies, one caution applies. Old replies can contain versions that are now out of date. SupportMe, for example, is designed so nothing sends without your approval and so your edits update its knowledge base. That makes correcting a stale version in the draft worthwhile, because the correction feeds back into future drafts. It doesn't replace checking against your changelog, though. For more ways to keep reviews short, see 5 ways to review AI replies without slowing down.
Conclusion
A version number in an AI support reply is a factual claim, so check it like one. Find every version reference, confirm it exists and is current, and match it to the customer's version. Verify "fixed in" claims against your changelog and read range syntax by the rules of its own ecosystem. Remove promises about future releases. Most of these checks take seconds when you have a reliable source of truth.
References
- Semantic Versioning 2.0.0 — semver.org
- npm node-semver documentation (caret and tilde ranges) — github.com/npm/node-semver
- PEP 440: Version Identification and Dependency Specification — peps.python.org/pep-0440
- Spracklen et al., "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs" — arXiv:2406.10279
- Anthropic, Models overview (knowledge cutoff definitions) — platform.claude.com
- endoflife.date — endoflife.date
Tags
Related posts
AI-Assisted Support
A Review Checklist for AI Replies That Reference Your UI
A practical checklist for reviewing AI-drafted support replies that mention buttons, menus, and settings, so customers get labels, paths, and platform details that match what they actually see.
9 min read
AI-Assisted Support
A Review Checklist for AI Replies on CC'd Email Threads
A practical checklist for reviewing AI-drafted support replies on email threads with CC'd people. It covers recipients, private details, who you're addressing, quoted history, tone, and instructions hidden in thread content.
10 min read
AI-Assisted Support
A Review Checklist for AI Replies With Setup Steps
A practical checklist for reviewing AI-drafted support replies that include setup steps. Check versions, step order, commands, package names, safety, and success checks before you hit send.
10 min read