Knowledge Sources Explained: Crawls and Datasets

Welcome to your chatbot's brain - aka the Knowledge tab!

The Knowledge tab is the central hub for all the information your chatbot uses to respond to users. Whether you're importing content from your website or manually training your chatbot, this is where it all happens.

Two Key Sections: Crawls and Datasets

The Knowledge Tab is divided into two main sections: Crawls and Datasets.

Crawls

💡 Crawls allow the platform to automatically gather information from external sources—so you don't have to input everything manually.

Here’s what a crawl can do:

Website Crawling: We scan the text across your website to feed it into the chatbot's brain.
PDF Crawling: (Included in your web crawl) If you have PDFs hosted on your site, we extract text from inside those documents.
RSS Feed Crawling: Useful for events and updates—we can pull in event dates, times, and locations.
File Uploads: Upload your own PDFs or Excel sheets with URLs to guide our crawling process.

Datasets

💡 Think of datasets as your chatbot’s curated knowledge—your way of telling it exactly how to respond to certain queries.

Datasets house manual entries and training data—ideal for customizing or correcting how your chatbot responds.

Any time you train the bot through the Consider Training section of the Analytics tab, those entries can be sent directly into the dataset.
You can also click New Entry in the dataset to manually add custom Q&A pairs that you want the chatbot to use.