<h1>
OpenWebUI notes ...
</h1>
<p>
Open WebUI installer: <a moz-do-not-send="true"
href="https://github.com/freeload101/SCRIPTS/blob/master/Bash/OpenWebUI_Fast.bash">https://github.com/freeload101/SCRIPTS/blob/master/Bash/OpenWebUI_Fast.bash</a></p>
<p>
Older Professor synapse prompt you can use: <a
moz-do-not-send="true"
href="https://raw.githubusercontent.com/freeload101/SCRIPTS/refs/heads/master/Prof%20Synapse%20Old.txt">https://raw.githubusercontent.com/freeload101/SCRIPTS/refs/heads/master/Prof%20Synapse%20Old.txt</a></p>
<p>
Fabric prompts you can import into openwebui !!! ( <a
moz-do-not-send="true"
href="https://github.com/danielmiessler/fabric/tree/main/patterns">https://github.com/danielmiessler/fabric/tree/main/patterns</a>
) <a moz-do-not-send="true"
href="https://github.com/freeload101/SCRIPTS/blob/master/MISC/Fabric_Prompts_Open_WebUI_OpenWebUI_20241112.json">https://github.com/freeload101/SCRIPTS/blob/master/MISC/Fabric_Prompts_Open_WebUI_OpenWebUI_20241112.json</a></p>
<p>
Example AT windows task startup script to make it start and not
die on boot <a moz-do-not-send="true"
href="https://github.com/freeload101/SCRIPTS/blob/master/MISC/StartKokoro.xml">https://github.com/freeload101/SCRIPTS/blob/master/MISC/StartKokoro.xml</a></p>
<p>
Open WebUI RAG fail sause ... <a moz-do-not-send="true"
href="https://youtu.be/CfnLrTcnPtY">https://youtu.be/CfnLrTcnPtY</a></p>
<h2>
Open registration&nbsp;
</h2>
<p>
<br>
</p>
<p>
<a href="hpr4337_image_ext_1.png">
<img src="hpr4337_image_ext_1_tn.png">
</a>
</p>
<p>
Model list / order
</p>
<pre>
NAME &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ID&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; SIZE&nbsp; &nbsp; &nbsp; MODIFIED
</pre>
<pre>hf.co/mradermacher/L3-8B-Stheno-v3.2-i1-GGUF:Q4_K_S&nbsp; &nbsp; 017d7a278e7e&nbsp; &nbsp; 4.7 GB&nbsp; &nbsp; 2 days ago
</pre>
<pre>qwen2.5:32b&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 9f13ba1299af&nbsp; &nbsp; 19 GB &nbsp; &nbsp; 3 days ago
</pre>
<pre>deepsex:latest &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; c83a52741a8a&nbsp; &nbsp; 20 GB &nbsp; &nbsp; 3 days ago
</pre>
<pre>HammerAI/openhermes-2.5-mistral:latest &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; d98003b83e17&nbsp; &nbsp; 4.4 GB&nbsp; &nbsp; 2 weeks ago
</pre>
<pre>Sweaterdog/Andy-3.5:latest &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; d3d9dc04b65a&nbsp; &nbsp; 4.7 GB&nbsp; &nbsp; 2 weeks ago
</pre>
<pre>nomic-embed-text:latest&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0a109f422b47&nbsp; &nbsp; 274 MB&nbsp; &nbsp; 2 weeks ago
</pre>
<pre>deepseek-r1:32b&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 38056bbcbb2d&nbsp; &nbsp; 19 GB &nbsp; &nbsp; 4 weeks ago
</pre>
<pre>psyfighter2:latest &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; c1b3d5e5be73&nbsp; &nbsp; 7.9 GB&nbsp; &nbsp; 2 months ago
</pre>
<pre>CognitiveComputations/dolphin-llama3.1:latest&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ed9503dedda9&nbsp; &nbsp; 4.7 GB&nbsp; &nbsp; 2 months ago
</pre>
<p>
<br>
</p>
<p>
<a href="hpr4337_image_ext_2.png">
<img src="hpr4337_image_ext_2_tn.png">
</a>
</p>
<p>
<br>
</p>
<h2>
Disable Arena models 
</h2>

<p>
<img src="hpr4337_image_ext_3.png">
</p>
<h2>
Documents WIP RAG is not good .
</h2>
<p>
<a href="hpr4337_image_ext_4.png">
<img src="hpr4337_image_ext_4_tn.png">
</a>
</p>
<p>
<br>
</p>
<h2>
Discord notes;
</h2>
<p>
<a moz-do-not-send="true"
href="https://discord.com/channels/1170866489302188073/1340112218808909875"><span
style="background-color: transparent; color: rgb(0, 0, 0);">
https://discord.com/channels/1170866489302188073/1340112218808909875</a>
</p>
<p>
<br>
</p>
<ul>
<li>
Abhi Chaturvedi:&nbsp; @(Operat0r) try this To reduce latency
and improve accuracy, modify the .env file: Enable RAG
ENABLE_RAG=true
</li>
<li>
Use Hybrid Mode (Retrieval + Reranking for better context)
</li>
<li>
RAG_MODE=hybrid
</li>
<li>
Reduce the number of retrieved documents (default: 5)
</li>
<li>
RETRIEVAL_TOP_K=3
</li>
<li>
Use a Fast Embedding Model (instead of OpenAI's Ada-002)
</li>
<li>
EMBEDDING_MODEL=all-MiniLM-L6-v2 # Faster and lightweight .
Optimize the Vector Database VECTOR_DB_TYPE=chroma
CHROMA_DB_IMPL=hnsw # Faster search
CHROMA_DB_PATH=/root/open-webui/backend/data/vector_db.
Optimize Backend Performance # Increase Uvicorn worker count
(improves concurrency) UVICORN_WORKERS=4
</li>
<li>
Increase FastAPI request timeout (prevents RAG failures)
</li>
<li>
FASTAPI_TIMEOUT=60
</li>
<li>
Optimize database connection pool (for better query
performance)
</li>
<li>
SQLALCHEMY_POOL_SIZE=10
</li>

<li>
So probably the first thing to do is increase the top K value
in admin -&gt; settings -&gt; documents, or you could try the
new "full context mode" for rag documents. You may also need
to increase the context size on the model, but it will make it
slower, so you probably don't want to do that unless you start
seeing the "truncating input" warnings.
</li>
<li>
@JamesK
</li>
<li>
So probably the first thing to do is increase the top K value
in admin -&gt; settings -&gt; documents, or you could try the
new "full context mode" for rag documents. You may also need
to increase the context size on the model, but it will make it
slower, so you probably don't want to do that unless you start
seeing the "truncating input" warnings.
</li>


<li>
M]
</li>
<li>
JamesK:&nbsp; Ah, I see. The rag didn't work great for you in
this prompt. There are three hits and the first two are
duplicates, so there isn't much data for the model to work
with
</li>
<li>
[9:12 PM] JamesK:&nbsp; context section
</li>
<li>
I see a message warning that you are using the default 2048
context length, but not the message saying you've hit that
limit (from my logs the warning looks like
</li>
<li>
level=WARN source=runner.go:126 msg="truncating input prompt"
limit=32768 prompt=33434 numKeep=5
</li>
<li>
[6:06 AM] JamesK:&nbsp; If you set the env var OLLAMA_DEBUG=1
before running ollama serve it will dump the full prompt being
sent to the model, that should let you confirm what the rag
has put in the prompt
</li>
<li>
JamesK: Watch the console output from ollama and check for
warnings about overflowing the context. If you have the
default 2k context you may need to increase it until the
warnings go away
</li>
<li>
[8:58 PM] JamesK:&nbsp; But also, if you're using the default
rag, it chunks the input into small fragments, then matches
the fragments against your prompt and only inserts a few
fragments into the context, not the entire document. So it's
easily possible for the information you want to not be
present.
</li>
</ul>
<p>
<br>
</p>
<h2>
Auto updates
</h2>
<pre>echo '0,12 */4 * * * docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui' &gt;&gt; /etc/crontab
</pre>
<h2>
Search
</h2>
<p>
red note for API keys&nbsp;
</p>
<p>
<a href="hpr4337_image_ext_5.png">
<img src="hpr4337_image_ext_5_tn.png">
</a>
</p>
<p>
<br>
</p>
<ol>
<li>
Go to Google Developers, use 
<a href="https://developers.google.com/custom-search"
rel="noopener noreferrer" target="_blank"
style="background-color: transparent; color: rgb(17, 85,
204);">
Programmable Search Engine</a>
, and log on or create account.
</li>
<li>
Go to 
<a
href="https://programmablesearchengine.google.com/controlpanel/all"
rel="noopener noreferrer" target="_blank"
style="background-color: transparent; color: rgb(17, 85,
204);">
control panel</a>
and click 
Add
button
</li>
<li>
Enter a search engine name, set the other properties to suit
your needs, verify you're not a robot and click 
Create
button.
</li>
<li>
Generate 
API key
and get the 
Search engine ID
. (Available after the engine is created)
</li>
<li>
With 
API key
and 
Search engine ID
, open 
Open WebUI Admin panel
and click 
Settings
tab, and then click 
Web Search
</li>
<li>
Enable 
Web search
and Set 
Web Search Engine
to 
google_pse
</li>
<li>
Fill 
Google PSE API Key
with the 
API key
and 
Google PSE Engine Id
(# 4)
</li>
<li>
Click 
Save
</li>
</ol>
<p>
<a href="hpr4337_image_ext_6.png">
<img src="hpr4337_image_ext_6_tn.png">
</a>
</p>
<h4>
Note
<a
href="https://docs.openwebui.com/tutorials/web-search/google-pse/#note"
rel="noopener noreferrer" target="_blank"
style="background-color: transparent; color: rgb(17, 85, 204);">
​</a>
</h4>
<p>
You have to enable 
Web search
in the prompt field, using plus (
+
) button. Search the web ;-)
</p>
<p>
<img src="hpr4337_image_ext_7.png">
</p>
<p>
<br>
</p>
<h2>
Kokoro / Open Webui&nbsp;
</h2>
<p>
<a moz-do-not-send="true"
href="https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html">https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html</a>
</p>
<p>
<a moz-do-not-send="true"
href="https://github.com/remsky/Kokoro-FastAPI?tab=readme-ov-file">https://github.com/remsky/Kokoro-FastAPI?tab=readme-ov-file</a>
</p>
<pre>apt update
</pre>
<pre>apt upgrade
</pre>
<pre>curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg &nbsp; &amp;&amp; curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | &nbsp; &nbsp; sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | &nbsp; &nbsp; sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
</pre>
<pre>sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list
</pre>
<pre>sudo apt-get update
</pre>
<pre>sudo apt-get install -y nvidia-container-toolkit
</pre>
<pre>apt install docker.io -y
</pre>
<pre>docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2
</pre>
<p>
<a href="hpr4337_image_ext_8.png">
<img src="hpr4337_image_ext_8_tn.png">
</a>
</p>
<p>
<br>
</p>
<p>
http://localhost:8880/v1
</p>
<p>
af_bella
</p>
<p>
Import fabric prompts
</p>
<p>
<a
href="https://raw.githubusercontent.com/freeload101/Python/46317dee34ebb83b01c800ce70b0506352ae2f3c/Fabric_Prompts_Open_WebUI_OpenWebUI.py"
rel="noopener noreferrer" target="_blank"
style="background-color: transparent; color: rgb(17, 85, 204);">
https://raw.githubusercontent.com/freeload101/Python/46317dee34ebb83b01c800ce70b0506352ae2f3c/Fabric_Prompts_Open_WebUI_OpenWebUI.py</a>
</p>
<p>
<br>
</p>
<p>
<br>
</p>