Point BeforeQuery at a bucket and it lists the objects, extracts the text of the ones it can read, and indexes them. Whatever produces your documents — a nightly export, a report generator, a system with no API worth connecting — can drop files in a bucket and they become answerable.

​
Prerequisites

  • The bucket name, and an access key with the two permissions below.
  • For a non-AWS store, the endpoint URL.
  • A BeforeQuery workspace owner role.

​
Data ingested

When you connect a bucket, BeforeQuery indexes every readable object under the prefix:
  • Object address — s3://bucket/key, or the public URL the bucket’s index.json maps that key to
  • Title, from the document’s own first heading and falling back to the object key
  • Content as markdown, extracted by file type — PDF, Word, Excel, CSV, HTML, markdown and text
  • The object’s ETag, so an object the bucket reports unchanged is never downloaded again
Objects of a type the pipeline cannot read are skipped, as is anything over the size limit; both are listed on the source with the reason.

​
Permissions required

Both are read-only — nothing is ever written to your bucket. We recommend creating dedicated credentials with only these two permissions, scoped to the bucket you’re connecting.

​
Set up the connector

1

Add the source

Open Knowledge Base → Sources → Add source and choose S3 Bucket. Enter the bucket, access key ID and secret access key, plus the endpoint if the store isn’t AWS.
2

Check the connection and add

Click Test connection — it lists the bucket and names what is wrong: a refused credential, a prefix matching nothing, or nothing in it we can read. Then click Add.

​
Connection settings

A prefix is matched against the start of the key, not walked as a directory. handbook/ indexes that folder; handbook also picks up handbook-archive/.

​
What gets indexed

Objects up to 32 MB whose type the extractor can read: .pdf, .docx, .xlsx, .xlsm, .xltx, .xltm, .md, .mdx, .txt, .html, .htm, .csv, .tsv. Images, archives and video are passed over — a bucket holding those alongside the documents is the ordinary case.

​
Making citations clickable

A document’s URL is s3://bucket/key, which identifies it but isn’t something a reader can click. If those documents are published somewhere, add an index.json at the root of the prefix and BeforeQuery cites the mapped address instead:
index.json
It’s read best-effort — a bucket without one is the normal case.

​
Keeping content in sync

The bucket is re-listed every 10 minutes — there is no schedule to configure, and Sync on the source’s page runs one now if you cannot wait. Only objects whose ETag changed are downloaded again, so a quiet bucket costs one listing. An object deleted from the bucket is removed from the index on the next sync, unless that would trip the 45% review threshold.

​
Troubleshooting

  • AccessDenied on sync? The key almost certainly has s3:GetObject but not s3:ListBucket — see Permissions required.
  • Signature rejected? The secret access key doesn’t match the access key ID. The region is not the cause — it’s detected from the bucket.
  • MinIO or Ceph returning NoSuchBucket? Turn on path-style addressing.
  • Nothing indexed? Usually the prefix. Run Test connection — it says whether the prefix matched anything.
  • A file missing? Objects over 32 MB and files no text could be read from are listed, with the reason, under the run in Sync history.
  • Citations reading s3://…? That’s the default. Add an index.json.