Keeping a website AI assistant accurate as content changes

TL;DR

• A website AI assistant answers from a copy of your content. Every time a page changes and the copy doesn't, the assistant starts giving yesterday's answer with today's confidence.
• Fix it in three layers: a sync that notices changes quickly (including deletions), content that keeps each important fact in one place, and a short set of test questions you rerun after every change that matters.
• Give someone the job. Accuracy is a content operations problem more than an AI problem.

Why accurate answers go stale

Most website assistants use retrieval: they search an index of your pages for passages that match the question and write an answer from them. That index is your assistant's knowledge base, and it is a snapshot. When the website changes, the snapshot does not update itself.

The changes that cause trouble are ordinary ones:

• a price change, a new plan or a discontinued tier;
• a renamed product or feature;
• an updated refund, shipping, security or support policy;
• a removed page that is still in the index;
• a new page that hasn't been indexed yet;
• an old blog post or PDF that still states the previous version of a fact.

None of these is a model failure. The assistant can be perfectly grounded in its sources and still be wrong, because the sources are out of date or disagree with each other. OWASP lists misinformation among the main risks of LLM applications, describing it as false or misleading output "that appears credible," and lists retrieval from trusted, verified sources first among its mitigations (OWASP).

The business risk is real. In Moffatt v. Air Canada (2024), a tribunal held the airline responsible for a chatbot answer that contradicted its own policy page (decision). Visitors treat what your assistant says as what your company says.

How an assistant finds out that something changed

There are four ways a sync can learn about changes. Good setups combine at least two.

Scheduled re-crawls

The assistant re-reads your site on a schedule: daily, weekly or on demand. Simple and reliable, but every change waits for the next run. Efficient crawlers use HTTP conditional requests: they send back the page's ETag or Last-Modified value, and an unchanged page answers 304 Not Modified instead of the full content (RFC 9110). If your server returns correct validators, frequent re-crawls stay cheap for both sides.

Sitemaps with honest dates

A sitemap's lastmod field tells crawlers when each page last changed (sitemaps.org). Google says it uses lastmod only if the value is "consistently and verifiably" accurate, and that it should reflect "the last significant update to the page": a change to the main content, structured data or links counts, a new copyright year does not (Google Search Central). The same discipline helps any crawler, including your assistant's. A CMS that stamps every page with today's date on every deploy makes lastmod useless.

<url>
  <loc>https://example.com/pricing</loc>
  <lastmod>2026-10-14</lastmod>
</url>

Push notifications on publish

Instead of waiting to be crawled, your site announces changes. Many assistant platforms accept a webhook or API call that re-indexes a URL right away; a CMS publish hook can call it. The same idea exists for search engines: the IndexNow protocol lets a site notify participating search engines when a URL is "added, updated, or deleted," and recommends automating submission as soon as content changes (IndexNow).

Manual re-sync

A button someone presses after a big change. It is the least reliable method on its own and the most useful as a backup on launch days.

Deletions are the change everyone forgets

Adding and editing pages is visible. Removing them is not, and a removed page that is still in the index keeps answering questions.

• Return a real status code. A deleted page should answer 404 or 410, not a "page not found" message with a 200 status. Google treats both 404 and 410 as signals that the content doesn't exist and removes such URLs from its index; a moved page should 301 redirect to its replacement (Google). Well-behaved assistant crawlers rely on the same signals.
• Check that the index drops it. After the next sync, ask the assistant a question only the deleted page answered. It should no longer cite it.
• Watch files. PDFs, price sheets and slide decks are often linked from one page and forgotten. If the assistant indexes them, they need the same lifecycle as pages.

What goes stale first

Write content that survives change

Sync speed only helps if the content itself is consistent. A few habits make a large difference:

• One home for each important fact. State prices on the pricing page and link to it from everywhere else, instead of repeating "$49 a month" in ten blog posts.
• Visible dates. Show "Updated October 2026" on pages whose facts change, and keep the dateModified value in your structured data in step with it (Schema.org). Dates help both visitors and the assistant judge freshness.
• Mark superseded content. If an old post must stay, add a short note at the top: "Prices in this post are from 2025; see current pricing."
• Facts in text, not only in images. A price shown only inside a screenshot will not be updated by anyone and may not be read correctly.
• Exclude what you don't stand behind. Archive sections, old press releases and internal-looking pages can be left out of the assistant's sources entirely.

The guide to training a chatbot on your website covers removing contradictions before the first index; the habits above keep new ones from creeping in.

Test questions: your early-warning system

Keep a short, written set of questions with the answers you expect, and rerun it after every change that matters. Anthropic's guidance on evaluations recommends task-specific tests that mirror real use, automated grading where possible, and more questions with simple checks over a few hand-graded ones (Claude Docs).

Start with 20 to 40 questions taken from real conversations, weighted toward high-stakes facts:

- question: "How much is the Pro plan per month?"
  must_include: ["$99"]
  must_cite: ["/pricing"]
- question: "Do you still offer the Basic plan?"
  must_include: ["no longer"]
  must_not_include: ["$19"]
- question: "Can I get a refund after 30 days?"
  expected: fixed_answer:refund-policy
- question: "Do you integrate with Pipedrive?"
  must_cite: ["/integrations"]

Simple string checks catch most regressions: a missing new price, an old price that should never appear again, or a citation pointing to the wrong page. When the test fails, the fix is almost always in the content or the index, not in the model.

A change routine that fits in a normal week

Big changes need a release plan

A price change, a product rename or a new policy deserves the same care as a code release:

1. Before: update every page that mentions the old fact, add notes to posts that must keep it, and draft fixed answers if you use them.
2. At launch: publish, trigger a re-sync, and run the test questions against the live assistant.
3. After: read the first day's conversations on the topic. Visitors will ask about the change in ways your test set didn't predict, and those questions belong in the test set next time.

Don't let the assistant get ahead of you either. If an upcoming change is not public, keep drafts out of anything it indexes until launch.

Common mistakes

• Syncing only on a weekly schedule while prices change on a Tuesday.
• Ignoring deletions, so removed pages keep answering.
• Repeating key facts across many pages, guaranteeing that one of them will be missed.
• Testing only at launch and never after.
• No owner. Marketing assumes the vendor handles it; the vendor assumes marketing does.

Answers that cite their sources make all of this easier to check: when the assistant shows which page it used, a stale answer points straight at the page that needs fixing.

An assistant is only as current as the content behind it. Sync quickly, delete properly, keep each fact in one place and rerun a small set of questions whenever something important changes, and the answers will keep up with the business.