Docs & code
Website
Crawl a documentation site, sitemap- and robots-aware.
GitHub
Repository markdown — READMEs, docs folders, ADRs, changelogs.
GitHub Issues
Issue threads, and their comments.
GitHub Pull Requests
PR descriptions and review threads.
GitHub Discussions
Community Q&A on a repository.
OpenAPI
API-reference answers from an OpenAPI 2.x/3.x spec.
Files & storage
File uploads
PDF, Word, Excel, Markdown, HTML and CSV.
Google Drive
The folders and files you pick, Docs and Sheets included.
OneDrive
One person’s drive, through Microsoft 365.
S3 bucket
Any S3-compatible bucket — AWS, MinIO, R2, Wasabi.
Wikis & trackers
Notion
Workspace pages and databases.
Confluence
Confluence Cloud spaces.
Jira
Project issues and their comment threads.
Linear
Completed issues and documents.
Slack
Channel message history.
Community
Discord
Channel history and forum posts, one document per post.
Discourse
Forum topics and replies.
Stack Overflow for Teams
Private team Q&A.
YouTube
Video transcripts.
Tickets & helpdesk
Zendesk
Help Center articles and resolved tickets.
Freshdesk
Resolved ticket conversations.
Front
Resolved shared-inbox conversations.
Intercom
Resolved Messenger conversations and tickets.
Jira Service Management
Resolved service-desk requests.
Salesforce Knowledge
Published Knowledge articles.
Salesforce Cases
Support cases, with comments and email threads.
Other
Curated Q&A
Answers you write yourself, when there’s no page to crawl.
Keeping content fresh
Every type re-syncs on its own fixed schedule — nothing to configure. Only changed content is re-fetched: a bucket object or a GitHub thread the upstream reports unchanged is never downloaded, and a web page your server reports unchanged is never read. Sync on a source’s page runs one on demand.
Git pushes trigger a reindex ahead of the hourly run. File uploads and curated Q&A have no Sync button at all: change them by uploading again, or by editing the entry.
Removing one document
Deleting a document from a source with an upstream doesn’t last — the next sync puts it straight back. Remove does both halves: the document is deleted now, and its URL is added to the source’s Excluded list, checked on every sync from then on. An entry ending in/ covers everything beneath it.
When a sync changes a lot
If a sync would remove more than 45% of a source’s documents, it doesn’t. The removals are held, the source moves to a review state, and someone approves or dismisses them. An expired token, a revoked permission and a paginated endpoint that stops early all look exactly like “this source has fewer documents now”. Documents the sync indexed go live either way; only deletions wait. A website crawl is judged on two more counts, because a site can break without losing a single URL. A crawl that rewrites more than 45% of its pages, or that suddenly finds far more pages than the source holds, goes to review as well — the first is usually a redesign or a changed content selector, the second an exclude pattern that stopped matching. Those pages are already indexed, so there is nothing to approve: the review is there so somebody looks before the next crawl builds on it, and automatic refreshes stay paused until it is resolved.Organizing sources
Group sources into named collections to model docs versions, product lines, or public-vs-internal content — and scope retrieval per surface. See Source Groups.Adding, editing and deleting sources needs the edit sources permission. See Roles & Permissions.