How to Increase Context Size in Open WebUI: A Step-by-Step Guide

Step-by-Step Guide on Increasing Context Size in Open WebUI

Step-by-Step Guide on Increasing Context Size in Open WebUI

Open WebUI defaults to a 2048-character context size for Ollama models. This limitation can lead to the model forgetting ongoing conversations quickly. If you need an even larger context window, you can increase this to 131072, which is the 128k context limit that Llama 3.1 has.

To adjust the context size, you have two options: globally or for individual chats.

• For Individual Chats: Click on the settings icon next to your account circle in the top right corner. Then, adjust the context size or temperature values as needed. Remember that larger contexts require more data processing, which may slow down response times.
• For Global Parameters: Click on your username in the bottom left corner, choose Settings — General, and then expand the Advanced Parameters submenu by clicking Show. Here, you can modify the desired settings such as the default context length (num_ctx). If you would like to set it from 2048 to 4096 tokens, you may simplify your colleagues’ experience.

For setting a specific model, meaning to set a L2 model like Mythomax for a base 4k context, you would set compress_pos_emb to 1. If you wish to stretch the context to 8k, then set compress_pos_emb to 2. Additionally, for your chat model, find one that supports a good context window size, perhaps between 32k to 128k.

Be mindful that higher context lengths generally lead to better performance in retaining conversations, but this comes with trade-offs like slower generation times and increased memory usage.

Lastly, if you’re modifying the num_ctx value, you may have attempted to do so in the following files:

• open-webui-0.3.32/src/lib/components/chat/Settings/General.svelte
• open-webui-0.3.32/src/lib/components/chat/Settings/Advanced/AdvancedParams.svelte

However, if these modifications did not successfully apply, ensure you rebuild the container afterwards for the changes to take effect.

Advanced Settings for Customizing Context Length in Open WebUI

Open WebUI defaults to a 2048-character context size for Ollama models, which can lead to the model forgetting ongoing conversations quickly. Similarly, parameters like ‘Temperature’ affect responsiveness. To increase context size or temperature only for the current chat, click on the settings icon next to your account circle in the top right corner and adjust the values. Remember that larger contexts require more data processing, potentially slowing down the response time.

For global parameter changes, click on your username in the bottom left corner, choose Settings — General, then expand the Advanced Parameters submenu by clicking Show and modify the desired settings.

To further customize the context length in Open WebUI, you can:

• Adjust the context size or temperature for the current chat in the settings icon.
• Summarize conversation history with checkpoint summarization filter to manage context use.
• Truncate chat context length with the ‘token limit’ and ‘max turns’ using the context length filter.
• Utilize the chat context clipper to retain the latest n-th user and assistant messages while always keeping the system prompt and the first message pair.

Since many of my colleagues aren’t experts in large language models (LLMs), I’d like to simplify their experience by increasing the default context length (num_ctx) from 2048 to a higher value. However, it seems that the setting does not take effect as intended.