Your project’s knowledge base is the long term memory for your LLMs, you can populate it with any type of document, where it will be split into chunks and each chunk is vectorized and given an embedding.

Documents

‍

Adding Documents

Documents can be manually added by importing them from a website or uploading a document file from your knowledge base. They may also be added automatically using research agents as explained in the sections below.

‍

Upload File

Click the upload file button and type in a unique name for your document, then click to upload your .docx, .pdf, or .txt document.

‍

Import from Website

Click the import website and fill in the required fields. Use the filters field to exclude slugs that contain specific keywords. We recommend you find your website's sitemap and put the path in the dedicated field; otherwise Moonlit will try to find your sitemap using common paths like /sitemap.xml. This functionality can handle up to 1000 pages, if your use-case requires more, please reach out.

‍

Once a document is added it's chunked into smaller pieces. We use a smart mechanism for chunking, rather than being size based. The text is chunked based on semantic relevance. This avoid the problem of fetching a Chunk X that requires context from Chunk Y for example.

‍

Testing

Before using the knowledge base in your workflows, you can test it here directly. Moonlit uses an agentic approach to knowledge retrieval; meaning your prompt/query is handled by an AI agent that can perform semantic search on the knowledge base and return a well structured response.

‍

‍

‍

Research Agents

Go to your knowledge base, under the Documents tab, you will see a section titled Research Agents. This is where you can deploy and manage your agents.

Deploying Agents

1. Create a new agent by hitting the tab button, provide a topic keyword, specify the search country (you may leave that as worldwide) and the search language.
2. Upon creating the agent, it will start the initialization process, performing search on your keyword across multiple search types, and populating your 'Document' section.
3. Once the initial research is complete, it will be scheduled to run every 4 hours rechecking the search results to detect any new information, if detected, it will add the new information.

‍

You can monitor your agent's performance by clicking on it and viewing it's logs and activity chart.

‍

‍

‍

Variables

Variables allow you to define project wide values that can be reused throughout multiple apps. For example you can store your company overview as a variable and use that to support LLMs with context over your company.

Adding Variables

You can add variables from this page by clicking on the Add button and providing a name, value, and optionally, tags. The tags help organise your variables, so if your managing content for a software company for example you may tag multiple variables as 'Features'. This is useful for using variables in your dropdowns as discussed below.

‍

Using Variables in your Apps

You can reference variables using the double bracket syntax in any text field that accepts dynamic inputs. (ex. {{Variables.Company Overview}} ).

‍

Using variables as dropdown options

This allows you to use your variables as options for a dropdown select input. If you're for example building a content pipeline for different clients but want to easily switch the context for the AI you can use this feature to create a dropdown that accepts different client overviews.