A knowledge base that reads your site for you
Add your address and ASDK reads your public pages, the way a search engine does. Sitemaps, files, text snippets and Q&A pairs fill in the rest. Every agent answers from this one knowledge base.
- 100 to 100,000 pages, by plan
- Read again daily, weekly on Free
- Q&A pairs live within seconds
Five ways to add what you know
Your website is the first source. The others fill in what isn’t on it, or isn’t easy for a crawler to read.
Your website
Starts at your home page and follows links. Reads public pages only, follows your robots.txt and keeps its request rate low.
A sitemap
Reads exactly the pages your sitemap lists, including pages no menu links to. Sitemap indexes and .xml.gz files work too.
Files
PDF, Word (.docx), text, Markdown, CSV and HTML, up to 20 MB each: price lists, spec sheets, policies.
Text snippets
Facts that aren’t on your site, written straight into ASDK. Up to 50,000 characters, edited in place.
Q&A pairs
The exact answer to one question. Agents prefer it over your pages, and it goes live within seconds.
Every passage remembers its page
ASDK keeps the text that carries meaning, splits long pages into short passages and indexes them by meaning. When a visitor asks, agents get only the few passages that match, each with a link back to its page.
Read
The crawler fetches each page and keeps its meaningful text. Duplicates and near-empty pages are skipped, with the reason shown.
Split
Long pages become short passages, one idea each. Every passage keeps a link to the page it came from.
Index
Passages are found by meaning, not only by shared words, so “Can you deliver to Toronto?” finds the page about shipping to Canada.
Stays current without you
Websites and sitemaps are read again every day, every week on the Free plan. ASDK compares each page with the version it has and processes only what changed.
Unchanged pages are skipped
Most syncs process only a few pages, so they finish quickly.
Changed and new pages are read
A changed page replaces its old version. New pages are added while your plan has room.
Removed pages leave
Pages that are gone from your site or your sitemap leave your knowledge too.
Read again, right now
Changed a price? Choose Read again on the source instead of waiting for the schedule.
You decide what goes in
Every source has a pages list: what was read, what was skipped and why. Keep sections out with a pattern, or one page with a click.
Skip sections with patterns
Add paths such as /blog or /**/print under Skip pages under, or read only the paths you list.
Exclude single pages
Keep an outdated page out for good, even while it stays online.
Cap a source
Set the most pages a source may read, so a large blog doesn’t crowd out your product pages.
Room for your whole site
Each plan holds a number of pages for the whole workspace. A file or a text snippet counts as one page, however long; Q&A pairs don’t count at all.
| Plan | Pages | Sites |
|---|---|---|
| Free | 100 | 1 |
| Starter | 1,000 | 1 |
| Pro | 5,000 | 3 |
| Business | 25,000 | 10 |
| Agency | 100,000 | 25 |
| 14-day trial | 1,000 | 1 |
At the limit nothing is deleted: pages already in your knowledge stay current, and new pages wait until there’s room.
Questions, answered
Something else? Write to hello@asdk.com
Does ASDK read pages behind a login?
No. The crawler reads public pages only: it doesn’t log in, fill in forms or click buttons. For content behind a login, upload it as a file or add it as a text snippet.
My site builds its pages with JavaScript. Will it work?
The crawler reads the page your server sends. If the text only appears after JavaScript runs, pages may be skipped as too short. Render pages on the server if you can, add a sitemap so every page is found, or upload the content as files.
How often is my site read again?
Every day, or every week on the Free plan. Files change when you upload a new version; text snippets and Q&A pairs update as soon as you save them. Read again picks up a change right away.
What happens when I reach my page limit?
Nothing is deleted. Pages already in your knowledge stay and keep being updated, and agents keep answering from them. New pages are skipped until there’s room: exclude pages you don’t need, or move to a bigger plan.
Does ASDK respect robots.txt?
Yes. ASDK-Crawler identifies itself, follows your robots.txt rules, honors Crawl-delay and keeps its request rate low. Pages your robots.txt disallows are not read, even when your sitemap lists them.
Is my content used to train AI models?
No. Each site’s knowledge is stored separately, agents on your site search only your site’s knowledge, and your content isn’t used to train foundation models.
See it answer from your own website
Enter your address to get started. Plans start free, and the 14-day trial needs no card.