.svg)
Your project’s knowledge base is the long term memory for your LLMs, you can populate it with any type of document, where it will be split into chunks and each chunk is vectorized and given an embedding.
Documents can be manually added by importing them from a website or uploading a document file from your knowledge base. They may also be added automatically using research agents as explained in the sections below.
Click the upload file button and type in a unique name for your document, then click to upload your .docx, .pdf, or .txt document.
Click the import website and fill in the required fields. Use the filters field to exclude slugs that contain specific keywords. We recommend you find your website's sitemap and put the path in the dedicated field; otherwise Moonlit will try to find your sitemap using common paths like /sitemap.xml. This functionality can handle up to 1000 pages, if your use-case requires more, please reach out.
Once a document is added it's chunked into smaller pieces. We use a smart mechanism for chunking, rather than being size based. The text is chunked based on semantic relevance. This avoid the problem of fetching a Chunk X that requires context from Chunk Y for example.
Before using the knowledge base in your workflows, you can test it here directly. Moonlit uses an agentic approach to knowledge retrieval; meaning your prompt/query is handled by an AI agent that can perform semantic search on the knowledge base and return a well structured response.
Go to your knowledge base, under the Documents tab, you will see a section titled Research Agents. This is where you can deploy and manage your agents.
You can monitor your agent's performance by clicking on it and viewing it's logs and activity chart.
Variables allow you to define project wide values that can be reused throughout multiple apps. For example you can store your company overview as a variable and use that to support LLMs with context over your company.
You can add variables from this page by clicking on the Add button and providing a name, value, and optionally, tags. The tags help organise your variables, so if your managing content for a software company for example you may tag multiple variables as 'Features'. This is useful for using variables in your dropdowns as discussed below.
You can reference variables using the double bracket syntax in any text field that accepts dynamic inputs. (ex. {{Variables.Company Overview}} ).
This allows you to use your variables as options for a dropdown select input. If you're for example building a content pipeline for different clients but want to easily switch the context for the AI you can use this feature to create a dropdown that accepts different client overviews.
Large language models are powerful content machines, but they lack when it comes to generating fully fledged articles with visual aid as a human writer would. Moonlit's "Image Knowledge Base" feature empowered these AI models to bypass this limitation.
This is done by using state-of-the-art models for accurately attaching a description to each image, the description is then vectorized so that we can later fetch the top X most relevant descriptions to a specified query along with the associated image url. We can then pass these relevant image with their descriptions to a language model and instruct it to use the images in the content it generates.
Navigate to your project's Image Knowledge Base section, and click the Add button. This will open up a modal with a form where you can upload your images (you can add up to 10 at once).
In addition to the upload box, you'll see a few extra options:
The first checkbox, when ticked will use the AI to generate a name for the image, otherwise the original upload file name will be used. We recommend this is always ticked unless the image is slightly abstract and the file name accurately describes within the proper context.
Context: As the field description suggests, this helps the AI understand what these images relate to. In the image example, we've set the context to "I run an outdoors equipement ecommerce store" so if I upload a picture of a forest, the AI model is more likely to associate it with hiking, outdoors, and camping. The more vague the images are, the more we'd recommend you be elaborate with the context. You can also tick the 'Save Context' checkbox if you want to add more images later that will use the same context.
When an Image is uploaded it will go through 3 stage: processing, vectorizing, and completed. When processing the AI will analyze the image and write the detailed description. Then in the vectorizing step, the generated description has embeddings created (Numerical representation of text) so that we can then semantically compare images and fetch only the ones we need in our workflows.
When an image status is set to "Completed" that means we can start using it in our apps. We created a simple flow for illustration purposes, in which we take an article title as input, pass it through the "Retreive Images" function which will return a list of image urls with descriptions that can relevant to the given query which in this case is our article title. We then passed the results of this function to the Large Language Model (LLM) to write the article with images using the following prompt:
Your task is to write an article with the title "{{Post Title}}".
I want you to also use images throughout your article so I will now supply you with a list of relevant images with their description and urls.
-----
# Images
{{Images}}
-----
The article must be in markdown format.
Please proceed with generating the post.
The following is the result of running the app described above. Given the title A Guide to Hiking with your Dog, it successfully generated an article with images retrieved from the knowledge base and naturally injected throughout the article.
Please reach out to us through the live chat widget on the bottom right corner or feel free to book a call with us. We're more than excited to explore and help you with your use case!
