Browse the docs
Knowledge
Add a sitemap
Use your XML sitemap as a knowledge source, so ASDK reads exactly the pages it lists. Where to find your sitemap, how to add it and how to fix sitemap errors.
Last updated
On this page
A sitemap is a file on your website that lists its pages, usually at /sitemap.xml. When you add it as a source, ASDK reads exactly the pages it lists instead of discovering pages by following links. This gives you control over what goes into your knowledge and finds pages that no menu links to.
When to use a sitemap
Add a sitemap when:
- some pages aren't linked from anywhere, such as landing pages or help articles reachable only through search;
- your menus are built with JavaScript, so following links finds only a few pages;
- your site is large and you want to read only the sections your sitemap lists, such as products and documentation;
- you already maintain a sitemap for search engines, so it is the most accurate list of your pages.
You can use a sitemap together with the Website source that ASDK created for your site. A page that both sources find is kept once.
Find your sitemap
Most sites have one at a standard address. Try these in your browser:
https://yoursite.com/sitemap.xmlhttps://yoursite.com/sitemap_index.xml, common on WordPress with an SEO plugin
Your robots.txt file often names it too. Open https://yoursite.com/robots.txt and look for a line like this:
Sitemap: https://acme.com/sitemap_index.xmlA sitemap opens as a page of XML that lists addresses in loc tags. A sitemap index lists other sitemaps instead of pages; ASDK follows it to the sitemaps it lists. Compressed sitemaps (.xml.gz) work too.
Add the sitemap
Open Knowledge
In the dashboard, open Knowledge and pick the site if you have more than one.
Add a source
Choose Add source, then A sitemap. In Sitemap address, enter the full address, such as https://acme.com/sitemap.xml.
Leave out sections (optional)
Open Choose which pages (optional) and list paths under Skip pages under, one per line, such as /tag or /author. A sitemap reads the pages it lists, so it has no "only pages under" list: skip what you don't want instead.
Set a maximum (optional)
Most pages to read (optional) caps this source. Your plan's page limit applies either way.
Add it
Choose Add source. The source card shows progress, and the pages appear on its Pages list as they are read.
During setup you can also add a sitemap in the Reading your site step: enter the Sitemap address and choose Add sitemap.
What ASDK reads from it
- Only pages on your site. Addresses in the sitemap that point to another domain are ignored. The sitemap itself must be on your site too.
- Each listed page once. Duplicate entries are read once.
- Your robots.txt still applies. Pages your robots.txt disallows for
ASDK-Crawlerare not read, even if the sitemap lists them. - Nothing beyond the list. ASDK doesn't follow links from sitemap pages. To read linked pages too, keep a Website source.
The sitemap is read again on the same schedule as websites: every day, or every week on the Free plan. Pages you add to the sitemap are picked up then, and pages you remove from it are removed from your knowledge. To read it right away, choose Read again on the source.
Sitemap errors
The source card shows Needs attention with one of these messages:
| Message | What to do |
|---|---|
| This sitemap doesn't exist (it returned "not found"). Check the address. | Open the address in your browser. Fix typos, or try /sitemap_index.xml. |
| We couldn't read this sitemap. Check that the address opens an XML sitemap. | The address returns a web page or broken XML. Use the XML file, not an HTML sitemap page. |
| This sitemap lists no pages on your site. | The sitemap is empty, or lists addresses on another domain, such as a staging host. |
| We couldn't reach this sitemap. We'll try again. | A temporary network or server problem. ASDK retries on its own. |
| This sitemap isn't on the site anymore. Delete the source and add it again. | The site's address changed. Add the sitemap again with the new address. |
The sitemaps.org protocol page describes the format: one sitemap file holds up to 50,000 addresses, and larger sites split them across files listed in a sitemap index (checked October 2026).
Change or remove a sitemap source
- Edit opens Choose which pages, where you change skipped paths and the maximum. Saving reads the source again.
- Delete removes the source and its pages from your knowledge. Agents stop answering from them right away.
Related: Exclude pages, Page limits.