Merge pull request #4434 from open-webui/dev

0.3.13

.github/ISSUE_TEMPLATE/bug_report.md

@@ -8,36 +8,43 @@ assignees: ''
 
 # Bug Report
 
-## Description
+## Installation Method
 
-**Bug Summary:**
-[Provide a brief but clear summary of the bug]
-
-**Steps to Reproduce:**
-[Outline the steps to reproduce the bug. Be as detailed as possible.]
-
-**Expected Behavior:**
-[Describe what you expected to happen.]
-
-**Actual Behavior:**
-[Describe what actually happened.]
+[Describe the method you used to install the project, e.g., git clone, Docker, pip, etc.]
 
 ## Environment
 
-- **Open WebUI Version:** [e.g., 0.1.120]
-- **Ollama (if applicable):** [e.g., 0.1.30, 0.1.32-rc1]
+- **Open WebUI Version:** [e.g., v0.3.11]
+- **Ollama (if applicable):** [e.g., v0.2.0, v0.1.32-rc1]
 
 - **Operating System:** [e.g., Windows 10, macOS Big Sur, Ubuntu 20.04]
 - **Browser (if applicable):** [e.g., Chrome 100.0, Firefox 98.0]
 
-## Reproduction Details
-
 **Confirmation:**
 
 - [ ] I have read and followed all the instructions provided in the README.md.
 - [ ] I am on the latest version of both Open WebUI and Ollama.
 - [ ] I have included the browser console logs.
 - [ ] I have included the Docker container logs.
+- [ ] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
+
+## Expected Behavior:
+
+[Describe what you expected to happen.]
+
+## Actual Behavior:
+
+[Describe what actually happened.]
+
+## Description
+
+**Bug Summary:**
+[Provide a brief but clear summary of the bug]
+
+## Reproduction Details
+
+**Steps to Reproduce:**
+[Outline the steps to reproduce the bug. Be as detailed as possible.]
 
 ## Logs and Screenshots
 
@@ -47,13 +54,9 @@ assignees: ''
 **Docker Container Logs:**
 [Include relevant Docker container logs, if applicable]
 
-**Screenshots (if applicable):**
+**Screenshots/Screen Recordings (if applicable):**
 [Attach any relevant screenshots to help illustrate the issue]
 
-## Installation Method
-
-[Describe the method you used to install the project, e.g., manual installation, Docker, package manager, etc.]
-
 ## Additional Information
 
 [Include any additional details that may help in understanding and reproducing the issue. This could include specific configurations, error messages, or anything else relevant to the bug.]

.github/workflows/integration-test.yml

@@ -15,6 +15,13 @@ jobs:
     name: Run Cypress Integration Tests
     runs-on: ubuntu-latest
     steps:
+      - name: Maximize build space
+        uses: AdityaGarg8/remove-unwanted-software@v4.1
+        with:
+          remove-android: 'true'
+          remove-haskell: 'true'
+          remove-codeql: 'true'
+
       - name: Checkout Repository
         uses: actions/checkout@v4
 
@@ -26,6 +33,10 @@ jobs:
             --file docker-compose.a1111-test.yaml \
             up --detach --build
 
+      - name: Delete Docker build cache
+        run: |
+          docker builder prune --all --force
+
       - name: Wait for Ollama to be up
         timeout-minutes: 5
         run: |
@@ -35,10 +46,6 @@ jobs:
           done
           echo "Service is up!"
 
-      - name: Delete Docker build cache
-        run: |
-          docker builder prune --all --force
-
       - name: Preload Ollama model
         run: |
           docker exec ollama ollama pull qwen:0.5b-chat-v1.5-q2_K

CHANGELOG.md

@@ -5,6 +5,55 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.3.13] - 2024-08-14
+
+### Added
+
+- **🎨 Enhanced Markdown Rendering**: Significant improvements in rendering markdown, ensuring smooth and reliable display of LaTeX and Mermaid charts, enhancing user experience with more robust visual content.
+- **🔄 Auto-Install Tools & Functions Python Dependencies**: For 'Tools' and 'Functions', Open WebUI now automatically install extra python requirements specified in the frontmatter, streamlining setup processes and customization.
+- **🌀 OAuth Email Claim Customization**: Introduced an 'OAUTH_EMAIL_CLAIM' variable to allow customization of the default "email" claim within OAuth configurations, providing greater flexibility in authentication processes.
+- **📶 Websocket Reconnection**: Enhanced reliability with the capability to automatically reconnect when a websocket is closed, ensuring consistent and stable communication.
+- **🤳 Haptic Feedback on Support Devices**: Android devices now support haptic feedback for an immersive tactile experience during certain interactions.
+
+### Fixed
+
+- **🛠️ ComfyUI Performance Improvement**: Addressed an issue causing FastAPI to stall when ComfyUI image generation was active; now runs in a separate thread to prevent UI unresponsiveness.
+- **🔀 Session Handling**: Fixed an issue mandating session_id on client-side to ensure smoother session management and transitions.
+- **🖋️ Minor Bug Fixes and Format Corrections**: Various minor fixes including typo corrections, backend formatting improvements, and test amendments enhancing overall system stability and performance.
+
+### Changed
+
+- **🚀 Migration to SvelteKit 2**: Upgraded the underlying framework to SvelteKit version 2, offering enhanced speed, better code structure, and improved deployment capabilities.
+- **🧹 General Cleanup and Refactoring**: Performed broad cleanup and refactoring across the platform, improving code efficiency and maintaining high standards of code health.
+- **🚧 Integration Testing Improvements**: Modified how Cypress integration tests detect chat messages and updated sharing tests for better reliability and accuracy.
+- **📁 Standardized '.safetensors' File Extension**: Renamed the '.sft' file extension to '.safetensors' for ComfyUI workflows, standardizing file formats across the platform.
+
+### Removed
+
+- **🗑️ Deprecated Frontend Functions**: Removed frontend functions that were migrated to backend to declutter the codebase and reduce redundancy.
+
+## [0.3.12] - 2024-08-07
+
+### Added
+
+- **🔄 Sidebar Infinite Scroll**: Added an infinite scroll feature in the sidebar for more efficient chat navigation, reducing load times and enhancing user experience.
+- **🚀 Enhanced Markdown Rendering**: Support for rendering all code blocks and making images clickable for preview; codespan styling is also enhanced to improve readability and user interaction.
+- **🔒 Admin Shared Chat Visibility**: Admins no longer have default visibility over shared chats when ENABLE_ADMIN_CHAT_ACCESS is set to false, tightening security and privacy settings for users.
+- **🌍 Language Updates**: Added Malay (Bahasa Malaysia) translation and updated Catalan and Traditional Chinese translations to improve accessibility for more users.
+
+### Fixed
+
+- **📊 Markdown Rendering Issues**: Resolved issues with markdown rendering to ensure consistent and correct display across components.
+- **🛠️ Styling Issues**: Multiple fixes applied to styling throughout the application, improving the overall visual experience and interface consistency.
+- **🗃️ Modal Handling**: Fixed an issue where modals were not closing correctly in various model chat scenarios, enhancing usability and interface reliability.
+- **📄 Missing OpenAI Usage Information**: Resolved issues where usage statistics for OpenAI services were not being correctly displayed, ensuring users have access to crucial data for managing and monitoring their API consumption.
+- **🔧 Non-Streaming Support for Functions Plugin**: Fixed a functionality issue with the Functions plugin where non-streaming operations were not functioning as intended, restoring full capabilities for async and sync integration within the platform.
+- **🔄 Environment Variable Type Correction (COMFYUI_FLUX_FP8_CLIP)**: Corrected the data type of the 'COMFYUI_FLUX_FP8_CLIP' environment variable from string to boolean, ensuring environment settings apply correctly and enhance configuration management.
+
+### Changed
+
+- **🔧 Backend Dependency Updates**: Updated several backend dependencies such as boto3, pypdf, python-pptx, validators, and black, ensuring up-to-date security and performance optimizations.
+
 ## [0.3.11] - 2024-08-02
 
 ### Added

backend/apps/audio/main.py

@@ -15,7 +15,7 @@ from fastapi.responses import StreamingResponse, JSONResponse, FileResponse
 from fastapi.middleware.cors import CORSMiddleware
 from pydantic import BaseModel
 
-from typing import List
+
 import uuid
 import requests
 import hashlib
@@ -244,7 +244,7 @@ async def speech(request: Request, user=Depends(get_verified_user)):
                     res = r.json()
                     if "error" in res:
                         error_detail = f"External: {res['error']['message']}"
-                except:
+                except Exception:
                     error_detail = f"External: {e}"
 
             raise HTTPException(
@@ -299,7 +299,7 @@ async def speech(request: Request, user=Depends(get_verified_user)):
                     res = r.json()
                     if "error" in res:
                         error_detail = f"External: {res['error']['message']}"
-                except:
+                except Exception:
                     error_detail = f"External: {e}"
 
             raise HTTPException(
@@ -353,7 +353,7 @@ def transcribe(
 
             try:
                 model = WhisperModel(**whisper_kwargs)
-            except:
+            except Exception:
                 log.warning(
                     "WhisperModel initialization failed, attempting download with local_files_only=False"
                 )
@@ -421,7 +421,7 @@ def transcribe(
                         res = r.json()
                         if "error" in res:
                             error_detail = f"External: {res['error']['message']}"
-                    except:
+                    except Exception:
                         error_detail = f"External: {e}"
 
                 raise HTTPException(
@@ -438,7 +438,7 @@ def transcribe(
         )
 
 
-def get_available_models() -> List[dict]:
+def get_available_models() -> list[dict]:
     if app.state.config.TTS_ENGINE == "openai":
         return [{"id": "tts-1"}, {"id": "tts-1-hd"}]
     elif app.state.config.TTS_ENGINE == "elevenlabs":
@@ -466,7 +466,7 @@ async def get_models(user=Depends(get_verified_user)):
     return {"models": get_available_models()}
 
 
-def get_available_voices() -> List[dict]:
+def get_available_voices() -> list[dict]:
     if app.state.config.TTS_ENGINE == "openai":
         return [
             {"name": "alloy", "id": "alloy"},

backend/apps/images/main.py

@@ -94,7 +94,7 @@ app.state.config.COMFYUI_FLUX_FP8_CLIP = COMFYUI_FLUX_FP8_CLIP
 
 
 def get_automatic1111_api_auth():
-    if app.state.config.AUTOMATIC1111_API_AUTH == None:
+    if app.state.config.AUTOMATIC1111_API_AUTH is None:
         return ""
     else:
         auth1111_byte_string = app.state.config.AUTOMATIC1111_API_AUTH.encode("utf-8")
@@ -145,28 +145,30 @@ async def get_engine_url(user=Depends(get_admin_user)):
 async def update_engine_url(
     form_data: EngineUrlUpdateForm, user=Depends(get_admin_user)
 ):
-    if form_data.AUTOMATIC1111_BASE_URL == None:
+    if form_data.AUTOMATIC1111_BASE_URL is None:
         app.state.config.AUTOMATIC1111_BASE_URL = AUTOMATIC1111_BASE_URL
     else:
         url = form_data.AUTOMATIC1111_BASE_URL.strip("/")
         try:
             r = requests.head(url)
+            r.raise_for_status()
             app.state.config.AUTOMATIC1111_BASE_URL = url
         except Exception as e:
-            raise HTTPException(status_code=400, detail=ERROR_MESSAGES.DEFAULT(e))
+            raise HTTPException(status_code=400, detail=ERROR_MESSAGES.INVALID_URL)
 
-    if form_data.COMFYUI_BASE_URL == None:
+    if form_data.COMFYUI_BASE_URL is None:
         app.state.config.COMFYUI_BASE_URL = COMFYUI_BASE_URL
     else:
         url = form_data.COMFYUI_BASE_URL.strip("/")
 
         try:
             r = requests.head(url)
+            r.raise_for_status()
             app.state.config.COMFYUI_BASE_URL = url
         except Exception as e:
-            raise HTTPException(status_code=400, detail=ERROR_MESSAGES.DEFAULT(e))
+            raise HTTPException(status_code=400, detail=ERROR_MESSAGES.INVALID_URL)
 
-    if form_data.AUTOMATIC1111_API_AUTH == None:
+    if form_data.AUTOMATIC1111_API_AUTH is None:
         app.state.config.AUTOMATIC1111_API_AUTH = AUTOMATIC1111_API_AUTH
     else:
         app.state.config.AUTOMATIC1111_API_AUTH = form_data.AUTOMATIC1111_API_AUTH
@@ -514,7 +516,7 @@ async def image_generations(
 
             data = ImageGenerationPayload(**data)
 
-            res = comfyui_generate_image(
+            res = await comfyui_generate_image(
                 app.state.config.MODEL,
                 data,
                 user.id,

backend/apps/images/utils/comfyui.py

@@ -1,5 +1,5 @@
+import asyncio
 import websocket  # NOTE: websocket-client (https://github.com/websocket-client/websocket-client)
-import uuid
 import json
 import urllib.request
 import urllib.parse
@@ -170,7 +170,7 @@ FLUX_DEFAULT_PROMPT = """
     },
     "10": {
         "inputs": {
-            "vae_name": "ae.sft"
+            "vae_name": "ae.safetensors"
         },
         "class_type": "VAELoader"
     },
@@ -184,7 +184,7 @@ FLUX_DEFAULT_PROMPT = """
     },
     "12": {
         "inputs": {
-            "unet_name": "flux1-dev.sft",
+            "unet_name": "flux1-dev.safetensors",
             "weight_dtype": "default"
         },
         "class_type": "UNETLoader"
@@ -328,7 +328,7 @@ class ImageGenerationPayload(BaseModel):
     flux_fp8_clip: Optional[bool] = None
 
 
-def comfyui_generate_image(
+async def comfyui_generate_image(
     model: str, payload: ImageGenerationPayload, client_id, base_url
 ):
     ws_url = base_url.replace("http://", "ws://").replace("https://", "wss://")
@@ -397,7 +397,9 @@ def comfyui_generate_image(
         return None
 
     try:
-        images = get_images(ws, comfyui_prompt, client_id, base_url)
+        images = await asyncio.to_thread(
+            get_images, ws, comfyui_prompt, client_id, base_url
+        )
     except Exception as e:
         log.exception(f"Error while receiving images: {e}")
         images = None

backend/apps/ollama/main.py

@@ -1,47 +1,36 @@
 from fastapi import (
     FastAPI,
     Request,
-    Response,
     HTTPException,
     Depends,
-    status,
     UploadFile,
     File,
-    BackgroundTasks,
 )
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import StreamingResponse
-from fastapi.concurrency import run_in_threadpool
 
 from pydantic import BaseModel, ConfigDict
 
 import os
 import re
-import copy
 import random
 import requests
 import json
-import uuid
 import aiohttp
 import asyncio
 import logging
 import time
 from urllib.parse import urlparse
-from typing import Optional, List, Union
+from typing import Optional, Union
 
 from starlette.background import BackgroundTask
 
 from apps.webui.models.models import Models
-from apps.webui.models.users import Users
 from constants import ERROR_MESSAGES
 from utils.utils import (
-    decode_token,
-    get_current_user,
     get_verified_user,
     get_admin_user,
 )
-from utils.task import prompt_template
-
 
 from config import (
     SRC_LOG_LEVELS,
@@ -53,7 +42,12 @@ from config import (
     UPLOAD_DIR,
     AppConfig,
 )
-from utils.misc import calculate_sha256, add_or_update_system_message
+from utils.misc import (
+    calculate_sha256,
+    apply_model_params_to_body_ollama,
+    apply_model_params_to_body_openai,
+    apply_model_system_prompt_to_body,
+)
 
 log = logging.getLogger(__name__)
 log.setLevel(SRC_LOG_LEVELS["OLLAMA"])
@@ -120,7 +114,7 @@ async def get_ollama_api_urls(user=Depends(get_admin_user)):
 
 
 class UrlUpdateForm(BaseModel):
-    urls: List[str]
+    urls: list[str]
 
 
 @app.post("/urls/update")
@@ -183,7 +177,7 @@ async def post_streaming_url(url: str, payload: str, stream: bool = True):
                 res = await r.json()
                 if "error" in res:
                     error_detail = f"Ollama: {res['error']}"
-            except:
+            except Exception:
                 error_detail = f"Ollama: {e}"
 
         raise HTTPException(
@@ -238,7 +232,7 @@ async def get_all_models():
 async def get_ollama_tags(
     url_idx: Optional[int] = None, user=Depends(get_verified_user)
 ):
-    if url_idx == None:
+    if url_idx is None:
         models = await get_all_models()
 
         if app.state.config.ENABLE_MODEL_FILTER:
@@ -269,7 +263,7 @@ async def get_ollama_tags(
                     res = r.json()
                     if "error" in res:
                         error_detail = f"Ollama: {res['error']}"
-                except:
+                except Exception:
                     error_detail = f"Ollama: {e}"
 
             raise HTTPException(
@@ -282,8 +276,7 @@ async def get_ollama_tags(
 @app.get("/api/version/{url_idx}")
 async def get_ollama_versions(url_idx: Optional[int] = None):
     if app.state.config.ENABLE_OLLAMA_API:
-        if url_idx == None:
-
+        if url_idx is None:
             # returns lowest version
             tasks = [
                 fetch_url(f"{url}/api/version")
@@ -323,7 +316,7 @@ async def get_ollama_versions(url_idx: Optional[int] = None):
                         res = r.json()
                         if "error" in res:
                             error_detail = f"Ollama: {res['error']}"
-                    except:
+                    except Exception:
                         error_detail = f"Ollama: {e}"
 
                 raise HTTPException(
@@ -346,8 +339,6 @@ async def pull_model(
     url = app.state.config.OLLAMA_BASE_URLS[url_idx]
     log.info(f"url: {url}")
 
-    r = None
-
     # Admin should be able to pull models from any source
     payload = {**form_data.model_dump(exclude_none=True), "insecure": True}
 
@@ -367,7 +358,7 @@ async def push_model(
     url_idx: Optional[int] = None,
     user=Depends(get_admin_user),
 ):
-    if url_idx == None:
+    if url_idx is None:
         if form_data.name in app.state.MODELS:
             url_idx = app.state.MODELS[form_data.name]["urls"][0]
         else:
@@ -417,7 +408,7 @@ async def copy_model(
     url_idx: Optional[int] = None,
     user=Depends(get_admin_user),
 ):
-    if url_idx == None:
+    if url_idx is None:
         if form_data.source in app.state.MODELS:
             url_idx = app.state.MODELS[form_data.source]["urls"][0]
         else:
@@ -428,13 +419,13 @@ async def copy_model(
 
     url = app.state.config.OLLAMA_BASE_URLS[url_idx]
     log.info(f"url: {url}")
+    r = requests.request(
+        method="POST",
+        url=f"{url}/api/copy",
+        data=form_data.model_dump_json(exclude_none=True).encode(),
+    )
 
     try:
-        r = requests.request(
-            method="POST",
-            url=f"{url}/api/copy",
-            data=form_data.model_dump_json(exclude_none=True).encode(),
-        )
         r.raise_for_status()
 
         log.debug(f"r.text: {r.text}")
@@ -448,7 +439,7 @@ async def copy_model(
                 res = r.json()
                 if "error" in res:
                     error_detail = f"Ollama: {res['error']}"
-            except:
+            except Exception:
                 error_detail = f"Ollama: {e}"
 
         raise HTTPException(
@@ -464,7 +455,7 @@ async def delete_model(
     url_idx: Optional[int] = None,
     user=Depends(get_admin_user),
 ):
-    if url_idx == None:
+    if url_idx is None:
         if form_data.name in app.state.MODELS:
             url_idx = app.state.MODELS[form_data.name]["urls"][0]
         else:
@@ -476,12 +467,12 @@ async def delete_model(
     url = app.state.config.OLLAMA_BASE_URLS[url_idx]
     log.info(f"url: {url}")
 
+    r = requests.request(
+        method="DELETE",
+        url=f"{url}/api/delete",
+        data=form_data.model_dump_json(exclude_none=True).encode(),
+    )
     try:
-        r = requests.request(
-            method="DELETE",
-            url=f"{url}/api/delete",
-            data=form_data.model_dump_json(exclude_none=True).encode(),
-        )
         r.raise_for_status()
 
         log.debug(f"r.text: {r.text}")
@@ -495,7 +486,7 @@ async def delete_model(
                 res = r.json()
                 if "error" in res:
                     error_detail = f"Ollama: {res['error']}"
-            except:
+            except Exception:
                 error_detail = f"Ollama: {e}"
 
         raise HTTPException(
@@ -516,12 +507,12 @@ async def show_model_info(form_data: ModelNameForm, user=Depends(get_verified_us
     url = app.state.config.OLLAMA_BASE_URLS[url_idx]
     log.info(f"url: {url}")
 
+    r = requests.request(
+        method="POST",
+        url=f"{url}/api/show",
+        data=form_data.model_dump_json(exclude_none=True).encode(),
+    )
     try:
-        r = requests.request(
-            method="POST",
-            url=f"{url}/api/show",
-            data=form_data.model_dump_json(exclude_none=True).encode(),
-        )
         r.raise_for_status()
 
         return r.json()
@@ -533,7 +524,7 @@ async def show_model_info(form_data: ModelNameForm, user=Depends(get_verified_us
                 res = r.json()
                 if "error" in res:
                     error_detail = f"Ollama: {res['error']}"
-            except:
+            except Exception:
                 error_detail = f"Ollama: {e}"
 
         raise HTTPException(
@@ -556,7 +547,7 @@ async def generate_embeddings(
     url_idx: Optional[int] = None,
     user=Depends(get_verified_user),
 ):
-    if url_idx == None:
+    if url_idx is None:
         model = form_data.model
 
         if ":" not in model:
@@ -573,12 +564,12 @@ async def generate_embeddings(
     url = app.state.config.OLLAMA_BASE_URLS[url_idx]
     log.info(f"url: {url}")
 
+    r = requests.request(
+        method="POST",
+        url=f"{url}/api/embeddings",
+        data=form_data.model_dump_json(exclude_none=True).encode(),
+    )
     try:
-        r = requests.request(
-            method="POST",
-            url=f"{url}/api/embeddings",
-            data=form_data.model_dump_json(exclude_none=True).encode(),
-        )
         r.raise_for_status()
 
         return r.json()
@@ -590,7 +581,7 @@ async def generate_embeddings(
                 res = r.json()
                 if "error" in res:
                     error_detail = f"Ollama: {res['error']}"
-            except:
+            except Exception:
                 error_detail = f"Ollama: {e}"
 
         raise HTTPException(
@@ -603,10 +594,9 @@ def generate_ollama_embeddings(
     form_data: GenerateEmbeddingsForm,
     url_idx: Optional[int] = None,
 ):
-
     log.info(f"generate_ollama_embeddings {form_data}")
 
-    if url_idx == None:
+    if url_idx is None:
         model = form_data.model
 
         if ":" not in model:
@@ -623,12 +613,12 @@ def generate_ollama_embeddings(
     url = app.state.config.OLLAMA_BASE_URLS[url_idx]
     log.info(f"url: {url}")
 
+    r = requests.request(
+        method="POST",
+        url=f"{url}/api/embeddings",
+        data=form_data.model_dump_json(exclude_none=True).encode(),
+    )
     try:
-        r = requests.request(
-            method="POST",
-            url=f"{url}/api/embeddings",
-            data=form_data.model_dump_json(exclude_none=True).encode(),
-        )
         r.raise_for_status()
 
         data = r.json()
@@ -638,7 +628,7 @@ def generate_ollama_embeddings(
         if "embedding" in data:
             return data["embedding"]
         else:
-            raise "Something went wrong :/"
+            raise Exception("Something went wrong :/")
     except Exception as e:
         log.exception(e)
         error_detail = "Open WebUI: Server Connection Error"
@@ -647,16 +637,16 @@ def generate_ollama_embeddings(
                 res = r.json()
                 if "error" in res:
                     error_detail = f"Ollama: {res['error']}"
-            except:
+            except Exception:
                 error_detail = f"Ollama: {e}"
 
-        raise error_detail
+        raise Exception(error_detail)
 
 
 class GenerateCompletionForm(BaseModel):
     model: str
     prompt: str
-    images: Optional[List[str]] = None
+    images: Optional[list[str]] = None
     format: Optional[str] = None
     options: Optional[dict] = None
     system: Optional[str] = None
@@ -674,8 +664,7 @@ async def generate_completion(
     url_idx: Optional[int] = None,
     user=Depends(get_verified_user),
 ):
-
-    if url_idx == None:
+    if url_idx is None:
         model = form_data.model
 
         if ":" not in model:
@@ -700,12 +689,12 @@ async def generate_completion(
 class ChatMessage(BaseModel):
     role: str
     content: str
-    images: Optional[List[str]] = None
+    images: Optional[list[str]] = None
 
 
 class GenerateChatCompletionForm(BaseModel):
     model: str
-    messages: List[ChatMessage]
+    messages: list[ChatMessage]
     format: Optional[str] = None
     options: Optional[dict] = None
     template: Optional[str] = None
@@ -713,6 +702,18 @@ class GenerateChatCompletionForm(BaseModel):
     keep_alive: Optional[Union[int, str]] = None
 
 
+def get_ollama_url(url_idx: Optional[int], model: str):
+    if url_idx is None:
+        if model not in app.state.MODELS:
+            raise HTTPException(
+                status_code=400,
+                detail=ERROR_MESSAGES.MODEL_NOT_FOUND(model),
+            )
+        url_idx = random.choice(app.state.MODELS[model]["urls"])
+    url = app.state.config.OLLAMA_BASE_URLS[url_idx]
+    return url
+
+
 @app.post("/api/chat")
 @app.post("/api/chat/{url_idx}")
 async def generate_chat_completion(
@@ -720,12 +721,7 @@ async def generate_chat_completion(
     url_idx: Optional[int] = None,
     user=Depends(get_verified_user),
 ):
-
-    log.debug(
-        "form_data.model_dump_json(exclude_none=True).encode(): {0} ".format(
-            form_data.model_dump_json(exclude_none=True).encode()
-        )
-    )
+    log.debug(f"{form_data.model_dump_json(exclude_none=True).encode()}=")
 
     payload = {
         **form_data.model_dump(exclude_none=True, exclude=["metadata"]),
@@ -740,185 +736,21 @@ async def generate_chat_completion(
         if model_info.base_model_id:
             payload["model"] = model_info.base_model_id
 
-        model_info.params = model_info.params.model_dump()
+        params = model_info.params.model_dump()
 
-        if model_info.params:
+        if params:
             if payload.get("options") is None:
                 payload["options"] = {}
 
-            if (
-                model_info.params.get("mirostat", None)
-                and payload["options"].get("mirostat") is None
-            ):
-                payload["options"]["mirostat"] = model_info.params.get("mirostat", None)
-
-            if (
-                model_info.params.get("mirostat_eta", None)
-                and payload["options"].get("mirostat_eta") is None
-            ):
-                payload["options"]["mirostat_eta"] = model_info.params.get(
-                    "mirostat_eta", None
-                )
-
-            if (
-                model_info.params.get("mirostat_tau", None)
-                and payload["options"].get("mirostat_tau") is None
-            ):
-                payload["options"]["mirostat_tau"] = model_info.params.get(
-                    "mirostat_tau", None
-                )
-
-            if (
-                model_info.params.get("num_ctx", None)
-                and payload["options"].get("num_ctx") is None
-            ):
-                payload["options"]["num_ctx"] = model_info.params.get("num_ctx", None)
-
-            if (
-                model_info.params.get("num_batch", None)
-                and payload["options"].get("num_batch") is None
-            ):
-                payload["options"]["num_batch"] = model_info.params.get(
-                    "num_batch", None
-                )
-
-            if (
-                model_info.params.get("num_keep", None)
-                and payload["options"].get("num_keep") is None
-            ):
-                payload["options"]["num_keep"] = model_info.params.get("num_keep", None)
-
-            if (
-                model_info.params.get("repeat_last_n", None)
-                and payload["options"].get("repeat_last_n") is None
-            ):
-                payload["options"]["repeat_last_n"] = model_info.params.get(
-                    "repeat_last_n", None
-                )
-
-            if (
-                model_info.params.get("frequency_penalty", None)
-                and payload["options"].get("frequency_penalty") is None
-            ):
-                payload["options"]["repeat_penalty"] = model_info.params.get(
-                    "frequency_penalty", None
-                )
-
-            if (
-                model_info.params.get("temperature", None) is not None
-                and payload["options"].get("temperature") is None
-            ):
-                payload["options"]["temperature"] = model_info.params.get(
-                    "temperature", None
-                )
-
-            if (
-                model_info.params.get("seed", None) is not None
-                and payload["options"].get("seed") is None
-            ):
-                payload["options"]["seed"] = model_info.params.get("seed", None)
-
-            if (
-                model_info.params.get("stop", None)
-                and payload["options"].get("stop") is None
-            ):
-                payload["options"]["stop"] = (
-                    [
-                        bytes(stop, "utf-8").decode("unicode_escape")
-                        for stop in model_info.params["stop"]
-                    ]
-                    if model_info.params.get("stop", None)
-                    else None
-                )
-
-            if (
-                model_info.params.get("tfs_z", None)
-                and payload["options"].get("tfs_z") is None
-            ):
-                payload["options"]["tfs_z"] = model_info.params.get("tfs_z", None)
-
-            if (
-                model_info.params.get("max_tokens", None)
-                and payload["options"].get("max_tokens") is None
-            ):
-                payload["options"]["num_predict"] = model_info.params.get(
-                    "max_tokens", None
-                )
-
-            if (
-                model_info.params.get("top_k", None)
-                and payload["options"].get("top_k") is None
-            ):
-                payload["options"]["top_k"] = model_info.params.get("top_k", None)
-
-            if (
-                model_info.params.get("top_p", None)
-                and payload["options"].get("top_p") is None
-            ):
-                payload["options"]["top_p"] = model_info.params.get("top_p", None)
-
-            if (
-                model_info.params.get("min_p", None)
-                and payload["options"].get("min_p") is None
-            ):
-                payload["options"]["min_p"] = model_info.params.get("min_p", None)
-
-            if (
-                model_info.params.get("use_mmap", None)
-                and payload["options"].get("use_mmap") is None
-            ):
-                payload["options"]["use_mmap"] = model_info.params.get("use_mmap", None)
-
-            if (
-                model_info.params.get("use_mlock", None)
-                and payload["options"].get("use_mlock") is None
-            ):
-                payload["options"]["use_mlock"] = model_info.params.get(
-                    "use_mlock", None
-                )
-
-            if (
-                model_info.params.get("num_thread", None)
-                and payload["options"].get("num_thread") is None
-            ):
-                payload["options"]["num_thread"] = model_info.params.get(
-                    "num_thread", None
-                )
-
-        system = model_info.params.get("system", None)
-        if system:
-            system = prompt_template(
-                system,
-                **(
-                    {
-                        "user_name": user.name,
-                        "user_location": (
-                            user.info.get("location") if user.info else None
-                        ),
-                    }
-                    if user
-                    else {}
-                ),
+            payload["options"] = apply_model_params_to_body_ollama(
+                params, payload["options"]
             )
+            payload = apply_model_system_prompt_to_body(params, payload, user)
 
-            if payload.get("messages"):
-                payload["messages"] = add_or_update_system_message(
-                    system, payload["messages"]
-                )
+    if ":" not in payload["model"]:
+        payload["model"] = f"{payload['model']}:latest"
 
-    if url_idx == None:
-        if ":" not in payload["model"]:
-            payload["model"] = f"{payload['model']}:latest"
-
-        if payload["model"] in app.state.MODELS:
-            url_idx = random.choice(app.state.MODELS[payload["model"]]["urls"])
-        else:
-            raise HTTPException(
-                status_code=400,
-                detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.model),
-            )
-
-    url = app.state.config.OLLAMA_BASE_URLS[url_idx]
+    url = get_ollama_url(url_idx, payload["model"])
     log.info(f"url: {url}")
     log.debug(payload)
 
@@ -940,7 +772,7 @@ class OpenAIChatMessage(BaseModel):
 
 class OpenAIChatCompletionForm(BaseModel):
     model: str
-    messages: List[OpenAIChatMessage]
+    messages: list[OpenAIChatMessage]
 
     model_config = ConfigDict(extra="allow")
 
@@ -952,83 +784,28 @@ async def generate_openai_chat_completion(
     url_idx: Optional[int] = None,
     user=Depends(get_verified_user),
 ):
-    form_data = OpenAIChatCompletionForm(**form_data)
-    payload = {**form_data.model_dump(exclude_none=True, exclude=["metadata"])}
-
+    completion_form = OpenAIChatCompletionForm(**form_data)
+    payload = {**completion_form.model_dump(exclude_none=True, exclude=["metadata"])}
     if "metadata" in payload:
         del payload["metadata"]
 
-    model_id = form_data.model
+    model_id = completion_form.model
     model_info = Models.get_model_by_id(model_id)
 
     if model_info:
         if model_info.base_model_id:
             payload["model"] = model_info.base_model_id
 
-        model_info.params = model_info.params.model_dump()
+        params = model_info.params.model_dump()
 
-        if model_info.params:
-            payload["temperature"] = model_info.params.get("temperature", None)
-            payload["top_p"] = model_info.params.get("top_p", None)
-            payload["max_tokens"] = model_info.params.get("max_tokens", None)
-            payload["frequency_penalty"] = model_info.params.get(
-                "frequency_penalty", None
-            )
-            payload["seed"] = model_info.params.get("seed", None)
-            payload["stop"] = (
-                [
-                    bytes(stop, "utf-8").decode("unicode_escape")
-                    for stop in model_info.params["stop"]
-                ]
-                if model_info.params.get("stop", None)
-                else None
-            )
+        if params:
+            payload = apply_model_params_to_body_openai(params, payload)
+            payload = apply_model_system_prompt_to_body(params, payload, user)
 
-        system = model_info.params.get("system", None)
+    if ":" not in payload["model"]:
+        payload["model"] = f"{payload['model']}:latest"
 
-        if system:
-            system = prompt_template(
-                system,
-                **(
-                    {
-                        "user_name": user.name,
-                        "user_location": (
-                            user.info.get("location") if user.info else None
-                        ),
-                    }
-                    if user
-                    else {}
-                ),
-            )
-            # Check if the payload already has a system message
-            # If not, add a system message to the payload
-            if payload.get("messages"):
-                for message in payload["messages"]:
-                    if message.get("role") == "system":
-                        message["content"] = system + message["content"]
-                        break
-                else:
-                    payload["messages"].insert(
-                        0,
-                        {
-                            "role": "system",
-                            "content": system,
-                        },
-                    )
-
-    if url_idx == None:
-        if ":" not in payload["model"]:
-            payload["model"] = f"{payload['model']}:latest"
-
-        if payload["model"] in app.state.MODELS:
-            url_idx = random.choice(app.state.MODELS[payload["model"]]["urls"])
-        else:
-            raise HTTPException(
-                status_code=400,
-                detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.model),
-            )
-
-    url = app.state.config.OLLAMA_BASE_URLS[url_idx]
+    url = get_ollama_url(url_idx, payload["model"])
     log.info(f"url: {url}")
 
     return await post_streaming_url(
@@ -1044,7 +821,7 @@ async def get_openai_models(
     url_idx: Optional[int] = None,
     user=Depends(get_verified_user),
 ):
-    if url_idx == None:
+    if url_idx is None:
         models = await get_all_models()
 
         if app.state.config.ENABLE_MODEL_FILTER:
@@ -1099,7 +876,7 @@ async def get_openai_models(
                     res = r.json()
                     if "error" in res:
                         error_detail = f"Ollama: {res['error']}"
-                except:
+                except Exception:
                     error_detail = f"Ollama: {e}"
 
             raise HTTPException(
@@ -1125,7 +902,6 @@ def parse_huggingface_url(hf_url):
         path_components = parsed_url.path.split("/")
 
         # Extract the desired output
-        user_repo = "/".join(path_components[1:3])
         model_file = path_components[-1]
 
         return model_file
@@ -1190,7 +966,6 @@ async def download_model(
     url_idx: Optional[int] = None,
     user=Depends(get_admin_user),
 ):
-
     allowed_hosts = ["https://huggingface.co/", "https://github.com/"]
 
     if not any(form_data.url.startswith(host) for host in allowed_hosts):
@@ -1199,7 +974,7 @@ async def download_model(
             detail="Invalid file_url. Only URLs from allowed hosts are permitted.",
         )
 
-    if url_idx == None:
+    if url_idx is None:
         url_idx = 0
     url = app.state.config.OLLAMA_BASE_URLS[url_idx]
 
@@ -1222,7 +997,7 @@ def upload_model(
     url_idx: Optional[int] = None,
     user=Depends(get_admin_user),
 ):
-    if url_idx == None:
+    if url_idx is None:
         url_idx = 0
     ollama_url = app.state.config.OLLAMA_BASE_URLS[url_idx]
 

backend/apps/openai/main.py

@@ -1,6 +1,6 @@
-from fastapi import FastAPI, Request, Response, HTTPException, Depends
+from fastapi import FastAPI, Request, HTTPException, Depends
 from fastapi.middleware.cors import CORSMiddleware
-from fastapi.responses import StreamingResponse, JSONResponse, FileResponse
+from fastapi.responses import StreamingResponse, FileResponse
 
 import requests
 import aiohttp
@@ -12,16 +12,15 @@ from pydantic import BaseModel
 from starlette.background import BackgroundTask
 
 from apps.webui.models.models import Models
-from apps.webui.models.users import Users
 from constants import ERROR_MESSAGES
 from utils.utils import (
-    decode_token,
-    get_verified_user,
     get_verified_user,
     get_admin_user,
 )
-from utils.task import prompt_template
-from utils.misc import add_or_update_system_message
+from utils.misc import (
+    apply_model_params_to_body_openai,
+    apply_model_system_prompt_to_body,
+)
 
 from config import (
     SRC_LOG_LEVELS,
@@ -34,7 +33,7 @@ from config import (
     MODEL_FILTER_LIST,
     AppConfig,
 )
-from typing import List, Optional
+from typing import Optional, Literal, overload
 
 
 import hashlib
@@ -69,8 +68,6 @@ app.state.MODELS = {}
 async def check_url(request: Request, call_next):
     if len(app.state.MODELS) == 0:
         await get_all_models()
-    else:
-        pass
 
     response = await call_next(request)
     return response
@@ -92,11 +89,11 @@ async def update_config(form_data: OpenAIConfigForm, user=Depends(get_admin_user
 
 
 class UrlsUpdateForm(BaseModel):
-    urls: List[str]
+    urls: list[str]
 
 
 class KeysUpdateForm(BaseModel):
-    keys: List[str]
+    keys: list[str]
 
 
 @app.get("/urls")
@@ -175,7 +172,7 @@ async def speech(request: Request, user=Depends(get_verified_user)):
                     res = r.json()
                     if "error" in res:
                         error_detail = f"External: {res['error']}"
-                except:
+                except Exception:
                     error_detail = f"External: {e}"
 
             raise HTTPException(
@@ -234,64 +231,68 @@ def merge_models_lists(model_lists):
     return merged_list
 
 
-async def get_all_models(raw: bool = False):
+def is_openai_api_disabled():
+    api_keys = app.state.config.OPENAI_API_KEYS
+    no_keys = len(api_keys) == 1 and api_keys[0] == ""
+    return no_keys or not app.state.config.ENABLE_OPENAI_API
+
+
+async def get_all_models_raw() -> list:
+    if is_openai_api_disabled():
+        return []
+
+    # Check if API KEYS length is same than API URLS length
+    num_urls = len(app.state.config.OPENAI_API_BASE_URLS)
+    num_keys = len(app.state.config.OPENAI_API_KEYS)
+
+    if num_keys != num_urls:
+        # if there are more keys than urls, remove the extra keys
+        if num_keys > num_urls:
+            new_keys = app.state.config.OPENAI_API_KEYS[:num_urls]
+            app.state.config.OPENAI_API_KEYS = new_keys
+        # if there are more urls than keys, add empty keys
+        else:
+            app.state.config.OPENAI_API_KEYS += [""] * (num_urls - num_keys)
+
+    tasks = [
+        fetch_url(f"{url}/models", app.state.config.OPENAI_API_KEYS[idx])
+        for idx, url in enumerate(app.state.config.OPENAI_API_BASE_URLS)
+    ]
+
+    responses = await asyncio.gather(*tasks)
+    log.debug(f"get_all_models:responses() {responses}")
+
+    return responses
+
+
+@overload
+async def get_all_models(raw: Literal[True]) -> list: ...
+
+
+@overload
+async def get_all_models(raw: Literal[False] = False) -> dict[str, list]: ...
+
+
+async def get_all_models(raw=False) -> dict[str, list] | list:
     log.info("get_all_models()")
+    if is_openai_api_disabled():
+        return [] if raw else {"data": []}
 
-    if (
-        len(app.state.config.OPENAI_API_KEYS) == 1
-        and app.state.config.OPENAI_API_KEYS[0] == ""
-    ) or not app.state.config.ENABLE_OPENAI_API:
-        models = {"data": []}
-    else:
-        # Check if API KEYS length is same than API URLS length
-        if len(app.state.config.OPENAI_API_KEYS) != len(
-            app.state.config.OPENAI_API_BASE_URLS
-        ):
-            # if there are more keys than urls, remove the extra keys
-            if len(app.state.config.OPENAI_API_KEYS) > len(
-                app.state.config.OPENAI_API_BASE_URLS
-            ):
-                app.state.config.OPENAI_API_KEYS = app.state.config.OPENAI_API_KEYS[
-                    : len(app.state.config.OPENAI_API_BASE_URLS)
-                ]
-            # if there are more urls than keys, add empty keys
-            else:
-                app.state.config.OPENAI_API_KEYS += [
-                    ""
-                    for _ in range(
-                        len(app.state.config.OPENAI_API_BASE_URLS)
-                        - len(app.state.config.OPENAI_API_KEYS)
-                    )
-                ]
+    responses = await get_all_models_raw()
+    if raw:
+        return responses
 
-        tasks = [
-            fetch_url(f"{url}/models", app.state.config.OPENAI_API_KEYS[idx])
-            for idx, url in enumerate(app.state.config.OPENAI_API_BASE_URLS)
-        ]
+    def extract_data(response):
+        if response and "data" in response:
+            return response["data"]
+        if isinstance(response, list):
+            return response
+        return None
 
-        responses = await asyncio.gather(*tasks)
-        log.debug(f"get_all_models:responses() {responses}")
+    models = {"data": merge_models_lists(map(extract_data, responses))}
 
-        if raw:
-            return responses
-
-        models = {
-            "data": merge_models_lists(
-                list(
-                    map(
-                        lambda response: (
-                            response["data"]
-                            if (response and "data" in response)
-                            else (response if isinstance(response, list) else None)
-                        ),
-                        responses,
-                    )
-                )
-            )
-        }
-
-        log.debug(f"models: {models}")
-        app.state.MODELS = {model["id"]: model for model in models["data"]}
+    log.debug(f"models: {models}")
+    app.state.MODELS = {model["id"]: model for model in models["data"]}
 
     return models
 
@@ -299,7 +300,7 @@ async def get_all_models(raw: bool = False):
 @app.get("/models")
 @app.get("/models/{url_idx}")
 async def get_models(url_idx: Optional[int] = None, user=Depends(get_verified_user)):
-    if url_idx == None:
+    if url_idx is None:
         models = await get_all_models()
         if app.state.config.ENABLE_MODEL_FILTER:
             if user.role == "user":
@@ -340,7 +341,7 @@ async def get_models(url_idx: Optional[int] = None, user=Depends(get_verified_us
                     res = r.json()
                     if "error" in res:
                         error_detail = f"External: {res['error']}"
-                except:
+                except Exception:
                     error_detail = f"External: {e}"
 
             raise HTTPException(
@@ -358,6 +359,7 @@ async def generate_chat_completion(
 ):
     idx = 0
     payload = {**form_data}
+
     if "metadata" in payload:
         del payload["metadata"]
 
@@ -368,70 +370,9 @@ async def generate_chat_completion(
         if model_info.base_model_id:
             payload["model"] = model_info.base_model_id
 
-        model_info.params = model_info.params.model_dump()
-
-        if model_info.params:
-            if (
-                model_info.params.get("temperature", None) is not None
-                and payload.get("temperature") is None
-            ):
-                payload["temperature"] = float(model_info.params.get("temperature"))
-
-            if model_info.params.get("top_p", None) and payload.get("top_p") is None:
-                payload["top_p"] = int(model_info.params.get("top_p", None))
-
-            if (
-                model_info.params.get("max_tokens", None)
-                and payload.get("max_tokens") is None
-            ):
-                payload["max_tokens"] = int(model_info.params.get("max_tokens", None))
-
-            if (
-                model_info.params.get("frequency_penalty", None)
-                and payload.get("frequency_penalty") is None
-            ):
-                payload["frequency_penalty"] = int(
-                    model_info.params.get("frequency_penalty", None)
-                )
-
-            if (
-                model_info.params.get("seed", None) is not None
-                and payload.get("seed") is None
-            ):
-                payload["seed"] = model_info.params.get("seed", None)
-
-            if model_info.params.get("stop", None) and payload.get("stop") is None:
-                payload["stop"] = (
-                    [
-                        bytes(stop, "utf-8").decode("unicode_escape")
-                        for stop in model_info.params["stop"]
-                    ]
-                    if model_info.params.get("stop", None)
-                    else None
-                )
-
-        system = model_info.params.get("system", None)
-        if system:
-            system = prompt_template(
-                system,
-                **(
-                    {
-                        "user_name": user.name,
-                        "user_location": (
-                            user.info.get("location") if user.info else None
-                        ),
-                    }
-                    if user
-                    else {}
-                ),
-            )
-            if payload.get("messages"):
-                payload["messages"] = add_or_update_system_message(
-                    system, payload["messages"]
-                )
-
-    else:
-        pass
+        params = model_info.params.model_dump()
+        payload = apply_model_params_to_body_openai(params, payload)
+        payload = apply_model_system_prompt_to_body(params, payload, user)
 
     model = app.state.MODELS[payload.get("model")]
     idx = model["urlIdx"]
@@ -444,13 +385,6 @@ async def generate_chat_completion(
             "role": user.role,
         }
 
-    # Check if the model is "gpt-4-vision-preview" and set "max_tokens" to 4000
-    # This is a workaround until OpenAI fixes the issue with this model
-    if payload.get("model") == "gpt-4-vision-preview":
-        if "max_tokens" not in payload:
-            payload["max_tokens"] = 4000
-        log.debug("Modified payload:", payload)
-
     # Convert the modified body back to JSON
     payload = json.dumps(payload)
 
@@ -506,7 +440,7 @@ async def generate_chat_completion(
                 print(res)
                 if "error" in res:
                     error_detail = f"External: {res['error']['message'] if 'message' in res['error'] else res['error']}"
-            except:
+            except Exception:
                 error_detail = f"External: {e}"
         raise HTTPException(status_code=r.status if r else 500, detail=error_detail)
     finally:
@@ -569,7 +503,7 @@ async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
                 print(res)
                 if "error" in res:
                     error_detail = f"External: {res['error']['message'] if 'message' in res['error'] else res['error']}"
-            except:
+            except Exception:
                 error_detail = f"External: {e}"
         raise HTTPException(status_code=r.status if r else 500, detail=error_detail)
     finally:

backend/apps/rag/main.py

@@ -13,7 +13,7 @@ import os, shutil, logging, re
 from datetime import datetime
 
 from pathlib import Path
-from typing import List, Union, Sequence, Iterator, Any
+from typing import Union, Sequence, Iterator, Any
 
 from chromadb.utils.batch_utils import create_batches
 from langchain_core.documents import Document
@@ -376,7 +376,7 @@ async def update_reranking_config(
     try:
         app.state.config.RAG_RERANKING_MODEL = form_data.reranking_model
 
-        update_reranking_model(app.state.config.RAG_RERANKING_MODEL), True
+        update_reranking_model(app.state.config.RAG_RERANKING_MODEL, True)
 
         return {
             "status": True,
@@ -439,7 +439,7 @@ class ChunkParamUpdateForm(BaseModel):
 
 
 class YoutubeLoaderConfig(BaseModel):
-    language: List[str]
+    language: list[str]
     translation: Optional[str] = None
 
 
@@ -642,7 +642,7 @@ def query_doc_handler(
 
 
 class QueryCollectionsForm(BaseModel):
-    collection_names: List[str]
+    collection_names: list[str]
     query: str
     k: Optional[int] = None
     r: Optional[float] = None
@@ -1021,7 +1021,7 @@ class TikaLoader:
         self.file_path = file_path
         self.mime_type = mime_type
 
-    def load(self) -> List[Document]:
+    def load(self) -> list[Document]:
         with open(self.file_path, "rb") as f:
             data = f.read()
 
@@ -1185,7 +1185,7 @@ def store_doc(
             f.close()
 
         f = open(file_path, "rb")
-        if collection_name == None:
+        if collection_name is None:
             collection_name = calculate_sha256(f)[:63]
         f.close()
 
@@ -1238,7 +1238,7 @@ def process_doc(
         f = open(file_path, "rb")
 
         collection_name = form_data.collection_name
-        if collection_name == None:
+        if collection_name is None:
             collection_name = calculate_sha256(f)[:63]
         f.close()
 
@@ -1296,7 +1296,7 @@ def store_text(
 ):
 
     collection_name = form_data.collection_name
-    if collection_name == None:
+    if collection_name is None:
         collection_name = calculate_sha256_string(form_data.content)
 
     result = store_text_in_vector_db(
@@ -1339,7 +1339,7 @@ def scan_docs_dir(user=Depends(get_admin_user)):
                         sanitized_filename = sanitize_filename(filename)
                         doc = Documents.get_doc_by_name(sanitized_filename)
 
-                        if doc == None:
+                        if doc is None:
                             doc = Documents.insert_new_doc(
                                 user.id,
                                 DocumentForm(

backend/apps/rag/search/brave.py

@@ -1,5 +1,5 @@
 import logging
-from typing import List, Optional
+from typing import Optional
 import requests
 
 from apps.rag.search.main import SearchResult, get_filtered_results
@@ -10,7 +10,7 @@ log.setLevel(SRC_LOG_LEVELS["RAG"])
 
 
 def search_brave(
-    api_key: str, query: str, count: int, filter_list: Optional[List[str]] = None
+    api_key: str, query: str, count: int, filter_list: Optional[list[str]] = None
 ) -> list[SearchResult]:
     """Search using Brave's Search API and return the results as a list of SearchResult objects.
 

backend/apps/rag/search/duckduckgo.py

@@ -1,5 +1,5 @@
 import logging
-from typing import List, Optional
+from typing import Optional
 from apps.rag.search.main import SearchResult, get_filtered_results
 from duckduckgo_search import DDGS
 from config import SRC_LOG_LEVELS
@@ -9,7 +9,7 @@ log.setLevel(SRC_LOG_LEVELS["RAG"])
 
 
 def search_duckduckgo(
-    query: str, count: int, filter_list: Optional[List[str]] = None
+    query: str, count: int, filter_list: Optional[list[str]] = None
 ) -> list[SearchResult]:
     """
     Search using DuckDuckGo's Search API and return the results as a list of SearchResult objects.
@@ -18,7 +18,7 @@ def search_duckduckgo(
         count (int): The number of results to return
 
     Returns:
-        List[SearchResult]: A list of search results
+        list[SearchResult]: A list of search results
     """
     # Use the DDGS context manager to create a DDGS object
     with DDGS() as ddgs:

backend/apps/rag/search/google_pse.py

@@ -1,6 +1,6 @@
 import json
 import logging
-from typing import List, Optional
+from typing import Optional
 import requests
 
 from apps.rag.search.main import SearchResult, get_filtered_results
@@ -15,7 +15,7 @@ def search_google_pse(
     search_engine_id: str,
     query: str,
     count: int,
-    filter_list: Optional[List[str]] = None,
+    filter_list: Optional[list[str]] = None,
 ) -> list[SearchResult]:
     """Search using Google's Programmable Search Engine API and return the results as a list of SearchResult objects.
 

backend/apps/rag/search/jina_search.py

@@ -17,7 +17,7 @@ def search_jina(query: str, count: int) -> list[SearchResult]:
         count (int): The number of results to return
 
     Returns:
-        List[SearchResult]: A list of search results
+        list[SearchResult]: A list of search results
     """
     jina_search_endpoint = "https://s.jina.ai/"
     headers = {

backend/apps/rag/search/searxng.py

@@ -1,7 +1,7 @@
 import logging
 import requests
 
-from typing import List, Optional
+from typing import Optional
 
 from apps.rag.search.main import SearchResult, get_filtered_results
 from config import SRC_LOG_LEVELS
@@ -14,9 +14,9 @@ def search_searxng(
     query_url: str,
     query: str,
     count: int,
-    filter_list: Optional[List[str]] = None,
+    filter_list: Optional[list[str]] = None,
     **kwargs,
-) -> List[SearchResult]:
+) -> list[SearchResult]:
     """
     Search a SearXNG instance for a given query and return the results as a list of SearchResult objects.
 
@@ -31,10 +31,10 @@ def search_searxng(
         language (str): Language filter for the search results; e.g., "en-US". Defaults to an empty string.
         safesearch (int): Safe search filter for safer web results; 0 = off, 1 = moderate, 2 = strict. Defaults to 1 (moderate).
         time_range (str): Time range for filtering results by date; e.g., "2023-04-05..today" or "all-time". Defaults to ''.
-        categories: (Optional[List[str]]): Specific categories within which the search should be performed, defaulting to an empty string if not provided.
+        categories: (Optional[list[str]]): Specific categories within which the search should be performed, defaulting to an empty string if not provided.
 
     Returns:
-        List[SearchResult]: A list of SearchResults sorted by relevance score in descending order.
+        list[SearchResult]: A list of SearchResults sorted by relevance score in descending order.
 
     Raise:
         requests.exceptions.RequestException: If a request error occurs during the search process.

backend/apps/rag/search/serper.py

@@ -1,6 +1,6 @@
 import json
 import logging
-from typing import List, Optional
+from typing import Optional
 import requests
 
 from apps.rag.search.main import SearchResult, get_filtered_results
@@ -11,7 +11,7 @@ log.setLevel(SRC_LOG_LEVELS["RAG"])
 
 
 def search_serper(
-    api_key: str, query: str, count: int, filter_list: Optional[List[str]] = None
+    api_key: str, query: str, count: int, filter_list: Optional[list[str]] = None
 ) -> list[SearchResult]:
     """Search using serper.dev's API and return the results as a list of SearchResult objects.
 

backend/apps/rag/search/serply.py

@@ -1,6 +1,6 @@
 import json
 import logging
-from typing import List, Optional
+from typing import Optional
 import requests
 from urllib.parse import urlencode
 
@@ -19,7 +19,7 @@ def search_serply(
     limit: int = 10,
     device_type: str = "desktop",
     proxy_location: str = "US",
-    filter_list: Optional[List[str]] = None,
+    filter_list: Optional[list[str]] = None,
 ) -> list[SearchResult]:
     """Search using serper.dev's API and return the results as a list of SearchResult objects.
 

backend/apps/rag/search/serpstack.py

@@ -1,6 +1,6 @@
 import json
 import logging
-from typing import List, Optional
+from typing import Optional
 import requests
 
 from apps.rag.search.main import SearchResult, get_filtered_results
@@ -14,7 +14,7 @@ def search_serpstack(
     api_key: str,
     query: str,
     count: int,
-    filter_list: Optional[List[str]] = None,
+    filter_list: Optional[list[str]] = None,
     https_enabled: bool = True,
 ) -> list[SearchResult]:
     """Search using serpstack.com's and return the results as a list of SearchResult objects.

backend/apps/rag/search/tavily.py

@@ -17,7 +17,7 @@ def search_tavily(api_key: str, query: str, count: int) -> list[SearchResult]:
         query (str): The query to search for
 
     Returns:
-        List[SearchResult]: A list of search results
+        list[SearchResult]: A list of search results
     """
     url = "https://api.tavily.com/search"
     data = {"query": query, "api_key": api_key}

backend/apps/rag/utils.py

@@ -2,7 +2,7 @@ import os
 import logging
 import requests
 
-from typing import List, Union
+from typing import Union
 
 from apps.ollama.main import (
     generate_ollama_embeddings,
@@ -142,7 +142,7 @@ def merge_and_sort_query_results(query_results, k, reverse=False):
 
 
 def query_collection(
-    collection_names: List[str],
+    collection_names: list[str],
     query: str,
     embedding_function,
     k: int,
@@ -157,13 +157,13 @@ def query_collection(
                 embedding_function=embedding_function,
             )
             results.append(result)
-        except:
+        except Exception:
             pass
     return merge_and_sort_query_results(results, k=k)
 
 
 def query_collection_with_hybrid_search(
-    collection_names: List[str],
+    collection_names: list[str],
     query: str,
     embedding_function,
     k: int,
@@ -182,7 +182,7 @@ def query_collection_with_hybrid_search(
                 r=r,
             )
             results.append(result)
-        except:
+        except Exception:
             pass
     return merge_and_sort_query_results(results, k=k, reverse=True)
 
@@ -411,7 +411,7 @@ class ChromaRetriever(BaseRetriever):
         query: str,
         *,
         run_manager: CallbackManagerForRetrieverRun,
-    ) -> List[Document]:
+    ) -> list[Document]:
         query_embeddings = self.embedding_function(query)
 
         results = self.collection.query(

backend/apps/socket/main.py

@@ -44,23 +44,26 @@ async def user_join(sid, data):
     print("user-join", sid, data)
 
     auth = data["auth"] if "auth" in data else None
+    if not auth or "token" not in auth:
+        return
 
-    if auth and "token" in auth:
-        data = decode_token(auth["token"])
+    data = decode_token(auth["token"])
+    if data is None or "id" not in data:
+        return
 
-        if data is not None and "id" in data:
-            user = Users.get_user_by_id(data["id"])
+    user = Users.get_user_by_id(data["id"])
+    if not user:
+        return
 
-        if user:
-            SESSION_POOL[sid] = user.id
-            if user.id in USER_POOL:
-                USER_POOL[user.id].append(sid)
-            else:
-                USER_POOL[user.id] = [sid]
+    SESSION_POOL[sid] = user.id
+    if user.id in USER_POOL:
+        USER_POOL[user.id].append(sid)
+    else:
+        USER_POOL[user.id] = [sid]
 
-            print(f"user {user.name}({user.id}) connected with session ID {sid}")
+    print(f"user {user.name}({user.id}) connected with session ID {sid}")
 
-            await sio.emit("user-count", {"count": len(set(USER_POOL))})
+    await sio.emit("user-count", {"count": len(set(USER_POOL))})
 
 
 @sio.on("user-count")

backend/apps/webui/main.py

@@ -22,9 +22,9 @@ from apps.webui.utils import load_function_module_by_id
 from utils.misc import (
     openai_chat_chunk_message_template,
     openai_chat_completion_message_template,
-    add_or_update_system_message,
+    apply_model_params_to_body_openai,
+    apply_model_system_prompt_to_body,
 )
-from utils.task import prompt_template
 
 
 from config import (
@@ -46,6 +46,7 @@ from config import (
     AppConfig,
     OAUTH_USERNAME_CLAIM,
     OAUTH_PICTURE_CLAIM,
+    OAUTH_EMAIL_CLAIM,
 )
 
 from apps.socket.main import get_event_call, get_event_emitter
@@ -84,6 +85,7 @@ app.state.config.ENABLE_COMMUNITY_SHARING = ENABLE_COMMUNITY_SHARING
 
 app.state.config.OAUTH_USERNAME_CLAIM = OAUTH_USERNAME_CLAIM
 app.state.config.OAUTH_PICTURE_CLAIM = OAUTH_PICTURE_CLAIM
+app.state.config.OAUTH_EMAIL_CLAIM = OAUTH_EMAIL_CLAIM
 
 app.state.MODELS = {}
 app.state.TOOLS = {}
@@ -269,47 +271,6 @@ def get_function_params(function_module, form_data, user, extra_params={}):
     return params
 
 
-# inplace function: form_data is modified
-def apply_model_params_to_body(params: dict, form_data: dict) -> dict:
-    if not params:
-        return form_data
-
-    mappings = {
-        "temperature": float,
-        "top_p": int,
-        "max_tokens": int,
-        "frequency_penalty": int,
-        "seed": lambda x: x,
-        "stop": lambda x: [bytes(s, "utf-8").decode("unicode_escape") for s in x],
-    }
-
-    for key, cast_func in mappings.items():
-        if (value := params.get(key)) is not None:
-            form_data[key] = cast_func(value)
-
-    return form_data
-
-
-# inplace function: form_data is modified
-def apply_model_system_prompt_to_body(params: dict, form_data: dict, user) -> dict:
-    system = params.get("system", None)
-    if not system:
-        return form_data
-
-    if user:
-        template_params = {
-            "user_name": user.name,
-            "user_location": user.info.get("location") if user.info else None,
-        }
-    else:
-        template_params = {}
-    system = prompt_template(system, **template_params)
-    form_data["messages"] = add_or_update_system_message(
-        system, form_data.get("messages", [])
-    )
-    return form_data
-
-
 async def generate_function_chat_completion(form_data, user):
     model_id = form_data.get("model")
     model_info = Models.get_model_by_id(model_id)
@@ -330,7 +291,7 @@ async def generate_function_chat_completion(form_data, user):
             form_data["model"] = model_info.base_model_id
 
         params = model_info.params.model_dump()
-        form_data = apply_model_params_to_body(params, form_data)
+        form_data = apply_model_params_to_body_openai(params, form_data)
         form_data = apply_model_system_prompt_to_body(params, form_data, user)
 
     pipe_id = get_pipe_id(form_data)

backend/apps/webui/models/auths.py

@@ -140,7 +140,7 @@ class AuthsTable:
                         return None
                 else:
                     return None
-        except:
+        except Exception:
             return None
 
     def authenticate_user_by_api_key(self, api_key: str) -> Optional[UserModel]:
@@ -152,7 +152,7 @@ class AuthsTable:
         try:
             user = Users.get_user_by_api_key(api_key)
             return user if user else None
-        except:
+        except Exception:
             return False
 
     def authenticate_user_by_trusted_header(self, email: str) -> Optional[UserModel]:
@@ -163,7 +163,7 @@ class AuthsTable:
                 if auth:
                     user = Users.get_user_by_id(auth.id)
                     return user
-        except:
+        except Exception:
             return None
 
     def update_user_password_by_id(self, id: str, new_password: str) -> bool:
@@ -174,7 +174,7 @@ class AuthsTable:
                 )
                 db.commit()
                 return True if result == 1 else False
-        except:
+        except Exception:
             return False
 
     def update_email_by_id(self, id: str, email: str) -> bool:
@@ -183,7 +183,7 @@ class AuthsTable:
                 result = db.query(Auth).filter_by(id=id).update({"email": email})
                 db.commit()
                 return True if result == 1 else False
-        except:
+        except Exception:
             return False
 
     def delete_auth_by_id(self, id: str) -> bool:
@@ -200,7 +200,7 @@ class AuthsTable:
                     return True
                 else:
                     return False
-        except:
+        except Exception:
             return False
 
 

backend/apps/webui/models/chats.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 import json
 import uuid
@@ -164,7 +164,7 @@ class ChatTable:
                 db.refresh(chat)
 
                 return self.get_chat_by_id(chat.share_id)
-        except:
+        except Exception:
             return None
 
     def delete_shared_chat_by_chat_id(self, chat_id: str) -> bool:
@@ -175,7 +175,7 @@ class ChatTable:
                 db.commit()
 
                 return True
-        except:
+        except Exception:
             return False
 
     def update_chat_share_id_by_id(
@@ -189,7 +189,7 @@ class ChatTable:
                 db.commit()
                 db.refresh(chat)
                 return ChatModel.model_validate(chat)
-        except:
+        except Exception:
             return None
 
     def toggle_chat_archive_by_id(self, id: str) -> Optional[ChatModel]:
@@ -201,7 +201,7 @@ class ChatTable:
                 db.commit()
                 db.refresh(chat)
                 return ChatModel.model_validate(chat)
-        except:
+        except Exception:
             return None
 
     def archive_all_chats_by_user_id(self, user_id: str) -> bool:
@@ -210,12 +210,12 @@ class ChatTable:
                 db.query(Chat).filter_by(user_id=user_id).update({"archived": True})
                 db.commit()
                 return True
-        except:
+        except Exception:
             return False
 
     def get_archived_chat_list_by_user_id(
         self, user_id: str, skip: int = 0, limit: int = 50
-    ) -> List[ChatModel]:
+    ) -> list[ChatModel]:
         with get_db() as db:
 
             all_chats = (
@@ -233,7 +233,7 @@ class ChatTable:
         include_archived: bool = False,
         skip: int = 0,
         limit: int = 50,
-    ) -> List[ChatModel]:
+    ) -> list[ChatModel]:
         with get_db() as db:
             query = db.query(Chat).filter_by(user_id=user_id)
             if not include_archived:
@@ -250,8 +250,8 @@ class ChatTable:
         user_id: str,
         include_archived: bool = False,
         skip: int = 0,
-        limit: int = 50,
-    ) -> List[ChatTitleIdResponse]:
+        limit: int = -1,
+    ) -> list[ChatTitleIdResponse]:
         with get_db() as db:
             query = db.query(Chat).filter_by(user_id=user_id)
             if not include_archived:
@@ -260,9 +260,10 @@ class ChatTable:
             all_chats = (
                 query.order_by(Chat.updated_at.desc())
                 # limit cols
-                .with_entities(
-                    Chat.id, Chat.title, Chat.updated_at, Chat.created_at
-                ).all()
+                .with_entities(Chat.id, Chat.title, Chat.updated_at, Chat.created_at)
+                .limit(limit)
+                .offset(skip)
+                .all()
             )
             # result has to be destrctured from sqlalchemy `row` and mapped to a dict since the `ChatModel`is not the returned dataclass.
             return [
@@ -278,8 +279,8 @@ class ChatTable:
             ]
 
     def get_chat_list_by_chat_ids(
-        self, chat_ids: List[str], skip: int = 0, limit: int = 50
-    ) -> List[ChatModel]:
+        self, chat_ids: list[str], skip: int = 0, limit: int = 50
+    ) -> list[ChatModel]:
         with get_db() as db:
             all_chats = (
                 db.query(Chat)
@@ -296,7 +297,7 @@ class ChatTable:
 
                 chat = db.get(Chat, id)
                 return ChatModel.model_validate(chat)
-        except:
+        except Exception:
             return None
 
     def get_chat_by_share_id(self, id: str) -> Optional[ChatModel]:
@@ -318,10 +319,10 @@ class ChatTable:
 
                 chat = db.query(Chat).filter_by(id=id, user_id=user_id).first()
                 return ChatModel.model_validate(chat)
-        except:
+        except Exception:
             return None
 
-    def get_chats(self, skip: int = 0, limit: int = 50) -> List[ChatModel]:
+    def get_chats(self, skip: int = 0, limit: int = 50) -> list[ChatModel]:
         with get_db() as db:
 
             all_chats = (
@@ -331,7 +332,7 @@ class ChatTable:
             )
             return [ChatModel.model_validate(chat) for chat in all_chats]
 
-    def get_chats_by_user_id(self, user_id: str) -> List[ChatModel]:
+    def get_chats_by_user_id(self, user_id: str) -> list[ChatModel]:
         with get_db() as db:
 
             all_chats = (
@@ -341,7 +342,7 @@ class ChatTable:
             )
             return [ChatModel.model_validate(chat) for chat in all_chats]
 
-    def get_archived_chats_by_user_id(self, user_id: str) -> List[ChatModel]:
+    def get_archived_chats_by_user_id(self, user_id: str) -> list[ChatModel]:
         with get_db() as db:
 
             all_chats = (
@@ -359,7 +360,7 @@ class ChatTable:
                 db.commit()
 
                 return True and self.delete_shared_chat_by_chat_id(id)
-        except:
+        except Exception:
             return False
 
     def delete_chat_by_id_and_user_id(self, id: str, user_id: str) -> bool:
@@ -370,7 +371,7 @@ class ChatTable:
                 db.commit()
 
                 return True and self.delete_shared_chat_by_chat_id(id)
-        except:
+        except Exception:
             return False
 
     def delete_chats_by_user_id(self, user_id: str) -> bool:
@@ -384,7 +385,7 @@ class ChatTable:
                 db.commit()
 
                 return True
-        except:
+        except Exception:
             return False
 
     def delete_shared_chats_by_user_id(self, user_id: str) -> bool:
@@ -399,7 +400,7 @@ class ChatTable:
                 db.commit()
 
                 return True
-        except:
+        except Exception:
             return False
 
 

backend/apps/webui/models/documents.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Optional
+from typing import Optional
 import time
 import logging
 
@@ -93,7 +93,7 @@ class DocumentsTable:
                     return DocumentModel.model_validate(result)
                 else:
                     return None
-            except:
+            except Exception:
                 return None
 
     def get_doc_by_name(self, name: str) -> Optional[DocumentModel]:
@@ -102,10 +102,10 @@ class DocumentsTable:
 
                 document = db.query(Document).filter_by(name=name).first()
                 return DocumentModel.model_validate(document) if document else None
-        except:
+        except Exception:
             return None
 
-    def get_docs(self) -> List[DocumentModel]:
+    def get_docs(self) -> list[DocumentModel]:
         with get_db() as db:
 
             return [
@@ -160,7 +160,7 @@ class DocumentsTable:
                 db.query(Document).filter_by(name=name).delete()
                 db.commit()
                 return True
-        except:
+        except Exception:
             return False
 
 

backend/apps/webui/models/files.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Union, Optional
+from typing import Union, Optional
 import time
 import logging
 
@@ -90,10 +90,10 @@ class FilesTable:
             try:
                 file = db.get(File, id)
                 return FileModel.model_validate(file)
-            except:
+            except Exception:
                 return None
 
-    def get_files(self) -> List[FileModel]:
+    def get_files(self) -> list[FileModel]:
         with get_db() as db:
 
             return [FileModel.model_validate(file) for file in db.query(File).all()]
@@ -107,7 +107,7 @@ class FilesTable:
                 db.commit()
 
                 return True
-            except:
+            except Exception:
                 return False
 
     def delete_all_files(self) -> bool:
@@ -119,7 +119,7 @@ class FilesTable:
                 db.commit()
 
                 return True
-            except:
+            except Exception:
                 return False
 
 

backend/apps/webui/models/functions.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Union, Optional
+from typing import Union, Optional
 import time
 import logging
 
@@ -122,10 +122,10 @@ class FunctionsTable:
 
                 function = db.get(Function, id)
                 return FunctionModel.model_validate(function)
-        except:
+        except Exception:
             return None
 
-    def get_functions(self, active_only=False) -> List[FunctionModel]:
+    def get_functions(self, active_only=False) -> list[FunctionModel]:
         with get_db() as db:
 
             if active_only:
@@ -141,7 +141,7 @@ class FunctionsTable:
 
     def get_functions_by_type(
         self, type: str, active_only=False
-    ) -> List[FunctionModel]:
+    ) -> list[FunctionModel]:
         with get_db() as db:
 
             if active_only:
@@ -157,7 +157,7 @@ class FunctionsTable:
                     for function in db.query(Function).filter_by(type=type).all()
                 ]
 
-    def get_global_filter_functions(self) -> List[FunctionModel]:
+    def get_global_filter_functions(self) -> list[FunctionModel]:
         with get_db() as db:
 
             return [
@@ -167,7 +167,7 @@ class FunctionsTable:
                 .all()
             ]
 
-    def get_global_action_functions(self) -> List[FunctionModel]:
+    def get_global_action_functions(self) -> list[FunctionModel]:
         with get_db() as db:
             return [
                 FunctionModel.model_validate(function)
@@ -198,7 +198,7 @@ class FunctionsTable:
                 db.commit()
                 db.refresh(function)
                 return self.get_function_by_id(id)
-            except:
+            except Exception:
                 return None
 
     def get_user_valves_by_id_and_user_id(
@@ -256,7 +256,7 @@ class FunctionsTable:
                 )
                 db.commit()
                 return self.get_function_by_id(id)
-            except:
+            except Exception:
                 return None
 
     def deactivate_all_functions(self) -> Optional[bool]:
@@ -271,7 +271,7 @@ class FunctionsTable:
                 )
                 db.commit()
                 return True
-            except:
+            except Exception:
                 return None
 
     def delete_function_by_id(self, id: str) -> bool:
@@ -281,7 +281,7 @@ class FunctionsTable:
                 db.commit()
 
                 return True
-            except:
+            except Exception:
                 return False
 
 

backend/apps/webui/models/memories.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 from sqlalchemy import Column, String, BigInteger, Text
 
@@ -80,25 +80,25 @@ class MemoriesTable:
                 )
                 db.commit()
                 return self.get_memory_by_id(id)
-            except:
+            except Exception:
                 return None
 
-    def get_memories(self) -> List[MemoryModel]:
+    def get_memories(self) -> list[MemoryModel]:
         with get_db() as db:
 
             try:
                 memories = db.query(Memory).all()
                 return [MemoryModel.model_validate(memory) for memory in memories]
-            except:
+            except Exception:
                 return None
 
-    def get_memories_by_user_id(self, user_id: str) -> List[MemoryModel]:
+    def get_memories_by_user_id(self, user_id: str) -> list[MemoryModel]:
         with get_db() as db:
 
             try:
                 memories = db.query(Memory).filter_by(user_id=user_id).all()
                 return [MemoryModel.model_validate(memory) for memory in memories]
-            except:
+            except Exception:
                 return None
 
     def get_memory_by_id(self, id: str) -> Optional[MemoryModel]:
@@ -107,7 +107,7 @@ class MemoriesTable:
             try:
                 memory = db.get(Memory, id)
                 return MemoryModel.model_validate(memory)
-            except:
+            except Exception:
                 return None
 
     def delete_memory_by_id(self, id: str) -> bool:
@@ -119,7 +119,7 @@ class MemoriesTable:
 
                 return True
 
-            except:
+            except Exception:
                 return False
 
     def delete_memories_by_user_id(self, user_id: str) -> bool:
@@ -130,7 +130,7 @@ class MemoriesTable:
                 db.commit()
 
                 return True
-            except:
+            except Exception:
                 return False
 
     def delete_memory_by_id_and_user_id(self, id: str, user_id: str) -> bool:
@@ -141,7 +141,7 @@ class MemoriesTable:
                 db.commit()
 
                 return True
-            except:
+            except Exception:
                 return False
 
 

backend/apps/webui/models/models.py

@@ -137,7 +137,7 @@ class ModelsTable:
             print(e)
             return None
 
-    def get_all_models(self) -> List[ModelModel]:
+    def get_all_models(self) -> list[ModelModel]:
         with get_db() as db:
             return [ModelModel.model_validate(model) for model in db.query(Model).all()]
 
@@ -146,7 +146,7 @@ class ModelsTable:
             with get_db() as db:
                 model = db.get(Model, id)
                 return ModelModel.model_validate(model)
-        except:
+        except Exception:
             return None
 
     def update_model_by_id(self, id: str, model: ModelForm) -> Optional[ModelModel]:
@@ -175,7 +175,7 @@ class ModelsTable:
                 db.commit()
 
                 return True
-        except:
+        except Exception:
             return False
 
 

backend/apps/webui/models/prompts.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Optional
+from typing import Optional
 import time
 
 from sqlalchemy import String, Column, BigInteger, Text
@@ -79,10 +79,10 @@ class PromptsTable:
 
                 prompt = db.query(Prompt).filter_by(command=command).first()
                 return PromptModel.model_validate(prompt)
-        except:
+        except Exception:
             return None
 
-    def get_prompts(self) -> List[PromptModel]:
+    def get_prompts(self) -> list[PromptModel]:
         with get_db() as db:
 
             return [
@@ -101,7 +101,7 @@ class PromptsTable:
                 prompt.timestamp = int(time.time())
                 db.commit()
                 return PromptModel.model_validate(prompt)
-        except:
+        except Exception:
             return None
 
     def delete_prompt_by_command(self, command: str) -> bool:
@@ -112,7 +112,7 @@ class PromptsTable:
                 db.commit()
 
                 return True
-        except:
+        except Exception:
             return False
 
 

backend/apps/webui/models/tags.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Optional
+from typing import Optional
 
 import json
 import uuid
@@ -69,11 +69,11 @@ class ChatIdTagForm(BaseModel):
 
 
 class TagChatIdsResponse(BaseModel):
-    chat_ids: List[str]
+    chat_ids: list[str]
 
 
 class ChatTagsResponse(BaseModel):
-    tags: List[str]
+    tags: list[str]
 
 
 class TagTable:
@@ -109,7 +109,7 @@ class TagTable:
         self, user_id: str, form_data: ChatIdTagForm
     ) -> Optional[ChatIdTagModel]:
         tag = self.get_tag_by_name_and_user_id(form_data.tag_name, user_id)
-        if tag == None:
+        if tag is None:
             tag = self.insert_new_tag(form_data.tag_name, user_id)
 
         id = str(uuid.uuid4())
@@ -132,10 +132,10 @@ class TagTable:
                     return ChatIdTagModel.model_validate(result)
                 else:
                     return None
-        except:
+        except Exception:
             return None
 
-    def get_tags_by_user_id(self, user_id: str) -> List[TagModel]:
+    def get_tags_by_user_id(self, user_id: str) -> list[TagModel]:
         with get_db() as db:
             tag_names = [
                 chat_id_tag.tag_name
@@ -159,7 +159,7 @@ class TagTable:
 
     def get_tags_by_chat_id_and_user_id(
         self, chat_id: str, user_id: str
-    ) -> List[TagModel]:
+    ) -> list[TagModel]:
         with get_db() as db:
 
             tag_names = [
@@ -184,7 +184,7 @@ class TagTable:
 
     def get_chat_ids_by_tag_name_and_user_id(
         self, tag_name: str, user_id: str
-    ) -> List[ChatIdTagModel]:
+    ) -> list[ChatIdTagModel]:
         with get_db() as db:
 
             return [

backend/apps/webui/models/tools.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict
-from typing import List, Optional
+from typing import Optional
 import time
 import logging
 from sqlalchemy import String, Column, BigInteger, Text
@@ -45,7 +45,7 @@ class ToolModel(BaseModel):
     user_id: str
     name: str
     content: str
-    specs: List[dict]
+    specs: list[dict]
     meta: ToolMeta
     updated_at: int  # timestamp in epoch
     created_at: int  # timestamp in epoch
@@ -81,7 +81,7 @@ class ToolValves(BaseModel):
 class ToolsTable:
 
     def insert_new_tool(
-        self, user_id: str, form_data: ToolForm, specs: List[dict]
+        self, user_id: str, form_data: ToolForm, specs: list[dict]
     ) -> Optional[ToolModel]:
 
         with get_db() as db:
@@ -115,10 +115,10 @@ class ToolsTable:
 
                 tool = db.get(Tool, id)
                 return ToolModel.model_validate(tool)
-        except:
+        except Exception:
             return None
 
-    def get_tools(self) -> List[ToolModel]:
+    def get_tools(self) -> list[ToolModel]:
         with get_db() as db:
             return [ToolModel.model_validate(tool) for tool in db.query(Tool).all()]
 
@@ -141,7 +141,7 @@ class ToolsTable:
                 )
                 db.commit()
                 return self.get_tool_by_id(id)
-        except:
+        except Exception:
             return None
 
     def get_user_valves_by_id_and_user_id(
@@ -196,7 +196,7 @@ class ToolsTable:
                 tool = db.query(Tool).get(id)
                 db.refresh(tool)
                 return ToolModel.model_validate(tool)
-        except:
+        except Exception:
             return None
 
     def delete_tool_by_id(self, id: str) -> bool:
@@ -206,7 +206,7 @@ class ToolsTable:
                 db.commit()
 
                 return True
-        except:
+        except Exception:
             return False
 
 

backend/apps/webui/models/users.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel, ConfigDict, parse_obj_as
-from typing import List, Union, Optional
+from typing import Union, Optional
 import time
 
 from sqlalchemy import String, Column, BigInteger, Text
@@ -125,7 +125,7 @@ class UsersTable:
 
                 user = db.query(User).filter_by(api_key=api_key).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
     def get_user_by_email(self, email: str) -> Optional[UserModel]:
@@ -134,7 +134,7 @@ class UsersTable:
 
                 user = db.query(User).filter_by(email=email).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
     def get_user_by_oauth_sub(self, sub: str) -> Optional[UserModel]:
@@ -143,10 +143,10 @@ class UsersTable:
 
                 user = db.query(User).filter_by(oauth_sub=sub).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
-    def get_users(self, skip: int = 0, limit: int = 50) -> List[UserModel]:
+    def get_users(self, skip: int = 0, limit: int = 50) -> list[UserModel]:
         with get_db() as db:
             users = (
                 db.query(User)
@@ -164,7 +164,7 @@ class UsersTable:
             with get_db() as db:
                 user = db.query(User).order_by(User.created_at).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
     def update_user_role_by_id(self, id: str, role: str) -> Optional[UserModel]:
@@ -174,7 +174,7 @@ class UsersTable:
                 db.commit()
                 user = db.query(User).filter_by(id=id).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
     def update_user_profile_image_url_by_id(
@@ -189,7 +189,7 @@ class UsersTable:
 
                 user = db.query(User).filter_by(id=id).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
     def update_user_last_active_by_id(self, id: str) -> Optional[UserModel]:
@@ -203,7 +203,7 @@ class UsersTable:
 
                 user = db.query(User).filter_by(id=id).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
     def update_user_oauth_sub_by_id(
@@ -216,7 +216,7 @@ class UsersTable:
 
                 user = db.query(User).filter_by(id=id).first()
                 return UserModel.model_validate(user)
-        except:
+        except Exception:
             return None
 
     def update_user_by_id(self, id: str, updated: dict) -> Optional[UserModel]:
@@ -245,7 +245,7 @@ class UsersTable:
                 return True
             else:
                 return False
-        except:
+        except Exception:
             return False
 
     def update_user_api_key_by_id(self, id: str, api_key: str) -> str:
@@ -254,7 +254,7 @@ class UsersTable:
                 result = db.query(User).filter_by(id=id).update({"api_key": api_key})
                 db.commit()
                 return True if result == 1 else False
-        except:
+        except Exception:
             return False
 
     def get_user_api_key_by_id(self, id: str) -> Optional[str]:

backend/apps/webui/routers/chats.py

@@ -1,6 +1,6 @@
 from fastapi import Depends, Request, HTTPException, status
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 from utils.utils import get_verified_user, get_admin_user
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -28,7 +28,7 @@ from apps.webui.models.tags import (
 
 from constants import ERROR_MESSAGES
 
-from config import SRC_LOG_LEVELS, ENABLE_ADMIN_EXPORT
+from config import SRC_LOG_LEVELS, ENABLE_ADMIN_EXPORT, ENABLE_ADMIN_CHAT_ACCESS
 
 log = logging.getLogger(__name__)
 log.setLevel(SRC_LOG_LEVELS["MODELS"])
@@ -40,12 +40,18 @@ router = APIRouter()
 ############################
 
 
-@router.get("/", response_model=List[ChatTitleIdResponse])
-@router.get("/list", response_model=List[ChatTitleIdResponse])
+@router.get("/", response_model=list[ChatTitleIdResponse])
+@router.get("/list", response_model=list[ChatTitleIdResponse])
 async def get_session_user_chat_list(
-    user=Depends(get_verified_user), skip: int = 0, limit: int = 50
+    user=Depends(get_verified_user), page: Optional[int] = None
 ):
-    return Chats.get_chat_title_id_list_by_user_id(user.id, skip=skip, limit=limit)
+    if page is not None:
+        limit = 60
+        skip = (page - 1) * limit
+
+        return Chats.get_chat_title_id_list_by_user_id(user.id, skip=skip, limit=limit)
+    else:
+        return Chats.get_chat_title_id_list_by_user_id(user.id)
 
 
 ############################
@@ -74,13 +80,18 @@ async def delete_all_user_chats(request: Request, user=Depends(get_verified_user
 ############################
 
 
-@router.get("/list/user/{user_id}", response_model=List[ChatTitleIdResponse])
+@router.get("/list/user/{user_id}", response_model=list[ChatTitleIdResponse])
 async def get_user_chat_list_by_user_id(
     user_id: str,
     user=Depends(get_admin_user),
     skip: int = 0,
     limit: int = 50,
 ):
+    if not ENABLE_ADMIN_CHAT_ACCESS:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail=ERROR_MESSAGES.ACCESS_PROHIBITED,
+        )
     return Chats.get_chat_list_by_user_id(
         user_id, include_archived=True, skip=skip, limit=limit
     )
@@ -108,7 +119,7 @@ async def create_new_chat(form_data: ChatForm, user=Depends(get_verified_user)):
 ############################
 
 
-@router.get("/all", response_model=List[ChatResponse])
+@router.get("/all", response_model=list[ChatResponse])
 async def get_user_chats(user=Depends(get_verified_user)):
     return [
         ChatResponse(**{**chat.model_dump(), "chat": json.loads(chat.chat)})
@@ -121,7 +132,7 @@ async def get_user_chats(user=Depends(get_verified_user)):
 ############################
 
 
-@router.get("/all/archived", response_model=List[ChatResponse])
+@router.get("/all/archived", response_model=list[ChatResponse])
 async def get_user_archived_chats(user=Depends(get_verified_user)):
     return [
         ChatResponse(**{**chat.model_dump(), "chat": json.loads(chat.chat)})
@@ -134,7 +145,7 @@ async def get_user_archived_chats(user=Depends(get_verified_user)):
 ############################
 
 
-@router.get("/all/db", response_model=List[ChatResponse])
+@router.get("/all/db", response_model=list[ChatResponse])
 async def get_all_user_chats_in_db(user=Depends(get_admin_user)):
     if not ENABLE_ADMIN_EXPORT:
         raise HTTPException(
@@ -152,7 +163,7 @@ async def get_all_user_chats_in_db(user=Depends(get_admin_user)):
 ############################
 
 
-@router.get("/archived", response_model=List[ChatTitleIdResponse])
+@router.get("/archived", response_model=list[ChatTitleIdResponse])
 async def get_archived_session_user_chat_list(
     user=Depends(get_verified_user), skip: int = 0, limit: int = 50
 ):
@@ -181,9 +192,9 @@ async def get_shared_chat_by_id(share_id: str, user=Depends(get_verified_user)):
             status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.NOT_FOUND
         )
 
-    if user.role == "user":
+    if user.role == "user" or (user.role == "admin" and not ENABLE_ADMIN_CHAT_ACCESS):
         chat = Chats.get_chat_by_share_id(share_id)
-    elif user.role == "admin":
+    elif user.role == "admin" and ENABLE_ADMIN_CHAT_ACCESS:
         chat = Chats.get_chat_by_id(share_id)
 
     if chat:
@@ -205,7 +216,7 @@ class TagNameForm(BaseModel):
     limit: Optional[int] = 50
 
 
-@router.post("/tags", response_model=List[ChatTitleIdResponse])
+@router.post("/tags", response_model=list[ChatTitleIdResponse])
 async def get_user_chat_list_by_tag_name(
     form_data: TagNameForm, user=Depends(get_verified_user)
 ):
@@ -230,7 +241,7 @@ async def get_user_chat_list_by_tag_name(
 ############################
 
 
-@router.get("/tags/all", response_model=List[TagModel])
+@router.get("/tags/all", response_model=list[TagModel])
 async def get_all_tags(user=Depends(get_verified_user)):
     try:
         tags = Tags.get_tags_by_user_id(user.id)
@@ -406,7 +417,7 @@ async def delete_shared_chat_by_id(id: str, user=Depends(get_verified_user)):
 ############################
 
 
-@router.get("/{id}/tags", response_model=List[TagModel])
+@router.get("/{id}/tags", response_model=list[TagModel])
 async def get_chat_tags_by_id(id: str, user=Depends(get_verified_user)):
     tags = Tags.get_tags_by_chat_id_and_user_id(id, user.id)
 

backend/apps/webui/routers/configs.py

@@ -1,7 +1,7 @@
 from fastapi import Response, Request
 from fastapi import Depends, FastAPI, HTTPException, status
 from datetime import datetime, timedelta
-from typing import List, Union
+from typing import Union
 
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -29,12 +29,12 @@ class SetDefaultModelsForm(BaseModel):
 
 
 class PromptSuggestion(BaseModel):
-    title: List[str]
+    title: list[str]
     content: str
 
 
 class SetDefaultSuggestionsForm(BaseModel):
-    suggestions: List[PromptSuggestion]
+    suggestions: list[PromptSuggestion]
 
 
 ############################
@@ -50,7 +50,7 @@ async def set_global_default_models(
     return request.app.state.config.DEFAULT_MODELS
 
 
-@router.post("/default/suggestions", response_model=List[PromptSuggestion])
+@router.post("/default/suggestions", response_model=list[PromptSuggestion])
 async def set_global_default_suggestions(
     request: Request,
     form_data: SetDefaultSuggestionsForm,
@@ -67,10 +67,10 @@ async def set_global_default_suggestions(
 
 
 class SetBannersForm(BaseModel):
-    banners: List[BannerModel]
+    banners: list[BannerModel]
 
 
-@router.post("/banners", response_model=List[BannerModel])
+@router.post("/banners", response_model=list[BannerModel])
 async def set_banners(
     request: Request,
     form_data: SetBannersForm,
@@ -81,7 +81,7 @@ async def set_banners(
     return request.app.state.config.BANNERS
 
 
-@router.get("/banners", response_model=List[BannerModel])
+@router.get("/banners", response_model=list[BannerModel])
 async def get_banners(
     request: Request,
     user=Depends(get_verified_user),

backend/apps/webui/routers/documents.py

@@ -1,6 +1,6 @@
 from fastapi import Depends, FastAPI, HTTPException, status
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -24,7 +24,7 @@ router = APIRouter()
 ############################
 
 
-@router.get("/", response_model=List[DocumentResponse])
+@router.get("/", response_model=list[DocumentResponse])
 async def get_documents(user=Depends(get_verified_user)):
     docs = [
         DocumentResponse(
@@ -46,7 +46,7 @@ async def get_documents(user=Depends(get_verified_user)):
 @router.post("/create", response_model=Optional[DocumentResponse])
 async def create_new_doc(form_data: DocumentForm, user=Depends(get_admin_user)):
     doc = Documents.get_doc_by_name(form_data.name)
-    if doc == None:
+    if doc is None:
         doc = Documents.insert_new_doc(user.id, form_data)
 
         if doc:
@@ -102,7 +102,7 @@ class TagItem(BaseModel):
 
 class TagDocumentForm(BaseModel):
     name: str
-    tags: List[dict]
+    tags: list[dict]
 
 
 @router.post("/doc/tags", response_model=Optional[DocumentResponse])

backend/apps/webui/routers/files.py

@@ -11,7 +11,7 @@ from fastapi import (
 
 
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 from pathlib import Path
 
 from fastapi import APIRouter
@@ -104,7 +104,7 @@ def upload_file(file: UploadFile = File(...), user=Depends(get_verified_user)):
 ############################
 
 
-@router.get("/", response_model=List[FileModel])
+@router.get("/", response_model=list[FileModel])
 async def list_files(user=Depends(get_verified_user)):
     files = Files.get_files()
     return files

backend/apps/webui/routers/functions.py

@@ -1,6 +1,6 @@
 from fastapi import Depends, FastAPI, HTTPException, status, Request
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -30,7 +30,7 @@ router = APIRouter()
 ############################
 
 
-@router.get("/", response_model=List[FunctionResponse])
+@router.get("/", response_model=list[FunctionResponse])
 async def get_functions(user=Depends(get_verified_user)):
     return Functions.get_functions()
 
@@ -40,7 +40,7 @@ async def get_functions(user=Depends(get_verified_user)):
 ############################
 
 
-@router.get("/export", response_model=List[FunctionModel])
+@router.get("/export", response_model=list[FunctionModel])
 async def get_functions(user=Depends(get_admin_user)):
     return Functions.get_functions()
 
@@ -63,7 +63,7 @@ async def create_new_function(
     form_data.id = form_data.id.lower()
 
     function = Functions.get_function_by_id(form_data.id)
-    if function == None:
+    if function is None:
         function_path = os.path.join(FUNCTIONS_DIR, f"{form_data.id}.py")
         try:
             with open(function_path, "w") as function_file:
@@ -235,7 +235,7 @@ async def delete_function_by_id(
         function_path = os.path.join(FUNCTIONS_DIR, f"{id}.py")
         try:
             os.remove(function_path)
-        except:
+        except Exception:
             pass
 
     return result

backend/apps/webui/routers/memories.py

@@ -1,7 +1,7 @@
 from fastapi import Response, Request
 from fastapi import Depends, FastAPI, HTTPException, status
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -30,7 +30,7 @@ async def get_embeddings(request: Request):
 ############################
 
 
-@router.get("/", response_model=List[MemoryModel])
+@router.get("/", response_model=list[MemoryModel])
 async def get_memories(user=Depends(get_verified_user)):
     return Memories.get_memories_by_user_id(user.id)
 

backend/apps/webui/routers/models.py

@@ -1,6 +1,6 @@
 from fastapi import Depends, FastAPI, HTTPException, status, Request
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -18,7 +18,7 @@ router = APIRouter()
 ###########################
 
 
-@router.get("/", response_model=List[ModelResponse])
+@router.get("/", response_model=list[ModelResponse])
 async def get_models(user=Depends(get_verified_user)):
     return Models.get_all_models()
 

backend/apps/webui/routers/prompts.py

@@ -1,6 +1,6 @@
 from fastapi import Depends, FastAPI, HTTPException, status
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -18,7 +18,7 @@ router = APIRouter()
 ############################
 
 
-@router.get("/", response_model=List[PromptModel])
+@router.get("/", response_model=list[PromptModel])
 async def get_prompts(user=Depends(get_verified_user)):
     return Prompts.get_prompts()
 
@@ -31,7 +31,7 @@ async def get_prompts(user=Depends(get_verified_user)):
 @router.post("/create", response_model=Optional[PromptModel])
 async def create_new_prompt(form_data: PromptForm, user=Depends(get_admin_user)):
     prompt = Prompts.get_prompt_by_command(form_data.command)
-    if prompt == None:
+    if prompt is None:
         prompt = Prompts.insert_new_prompt(user.id, form_data)
 
         if prompt:

backend/apps/webui/routers/tools.py

@@ -1,12 +1,8 @@
-from fastapi import Depends, FastAPI, HTTPException, status, Request
-from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from fastapi import Depends, HTTPException, status, Request
+from typing import Optional
 
 from fastapi import APIRouter
-from pydantic import BaseModel
-import json
 
-from apps.webui.models.users import Users
 from apps.webui.models.tools import Tools, ToolForm, ToolModel, ToolResponse
 from apps.webui.utils import load_toolkit_module_by_id
 
@@ -14,7 +10,6 @@ from utils.utils import get_admin_user, get_verified_user
 from utils.tools import get_tools_specs
 from constants import ERROR_MESSAGES
 
-from importlib import util
 import os
 from pathlib import Path
 
@@ -32,7 +27,7 @@ router = APIRouter()
 ############################
 
 
-@router.get("/", response_model=List[ToolResponse])
+@router.get("/", response_model=list[ToolResponse])
 async def get_toolkits(user=Depends(get_verified_user)):
     toolkits = [toolkit for toolkit in Tools.get_tools()]
     return toolkits
@@ -43,7 +38,7 @@ async def get_toolkits(user=Depends(get_verified_user)):
 ############################
 
 
-@router.get("/export", response_model=List[ToolModel])
+@router.get("/export", response_model=list[ToolModel])
 async def get_toolkits(user=Depends(get_admin_user)):
     toolkits = [toolkit for toolkit in Tools.get_tools()]
     return toolkits
@@ -69,7 +64,7 @@ async def create_new_toolkit(
     form_data.id = form_data.id.lower()
 
     toolkit = Tools.get_tool_by_id(form_data.id)
-    if toolkit == None:
+    if toolkit is None:
         toolkit_path = os.path.join(TOOLS_DIR, f"{form_data.id}.py")
         try:
             with open(toolkit_path, "w") as tool_file:
@@ -98,7 +93,7 @@ async def create_new_toolkit(
             print(e)
             raise HTTPException(
                 status_code=status.HTTP_400_BAD_REQUEST,
-                detail=ERROR_MESSAGES.DEFAULT(e),
+                detail=ERROR_MESSAGES.DEFAULT(str(e)),
             )
     else:
         raise HTTPException(
@@ -170,7 +165,7 @@ async def update_toolkit_by_id(
     except Exception as e:
         raise HTTPException(
             status_code=status.HTTP_400_BAD_REQUEST,
-            detail=ERROR_MESSAGES.DEFAULT(e),
+            detail=ERROR_MESSAGES.DEFAULT(str(e)),
         )
 
 
@@ -210,7 +205,7 @@ async def get_toolkit_valves_by_id(id: str, user=Depends(get_admin_user)):
         except Exception as e:
             raise HTTPException(
                 status_code=status.HTTP_400_BAD_REQUEST,
-                detail=ERROR_MESSAGES.DEFAULT(e),
+                detail=ERROR_MESSAGES.DEFAULT(str(e)),
             )
     else:
         raise HTTPException(
@@ -233,7 +228,7 @@ async def get_toolkit_valves_spec_by_id(
         if id in request.app.state.TOOLS:
             toolkit_module = request.app.state.TOOLS[id]
         else:
-            toolkit_module, frontmatter = load_toolkit_module_by_id(id)
+            toolkit_module, _ = load_toolkit_module_by_id(id)
             request.app.state.TOOLS[id] = toolkit_module
 
         if hasattr(toolkit_module, "Valves"):
@@ -261,7 +256,7 @@ async def update_toolkit_valves_by_id(
         if id in request.app.state.TOOLS:
             toolkit_module = request.app.state.TOOLS[id]
         else:
-            toolkit_module, frontmatter = load_toolkit_module_by_id(id)
+            toolkit_module, _ = load_toolkit_module_by_id(id)
             request.app.state.TOOLS[id] = toolkit_module
 
         if hasattr(toolkit_module, "Valves"):
@@ -276,7 +271,7 @@ async def update_toolkit_valves_by_id(
                 print(e)
                 raise HTTPException(
                     status_code=status.HTTP_400_BAD_REQUEST,
-                    detail=ERROR_MESSAGES.DEFAULT(e),
+                    detail=ERROR_MESSAGES.DEFAULT(str(e)),
                 )
         else:
             raise HTTPException(
@@ -306,7 +301,7 @@ async def get_toolkit_user_valves_by_id(id: str, user=Depends(get_verified_user)
         except Exception as e:
             raise HTTPException(
                 status_code=status.HTTP_400_BAD_REQUEST,
-                detail=ERROR_MESSAGES.DEFAULT(e),
+                detail=ERROR_MESSAGES.DEFAULT(str(e)),
             )
     else:
         raise HTTPException(
@@ -324,7 +319,7 @@ async def get_toolkit_user_valves_spec_by_id(
         if id in request.app.state.TOOLS:
             toolkit_module = request.app.state.TOOLS[id]
         else:
-            toolkit_module, frontmatter = load_toolkit_module_by_id(id)
+            toolkit_module, _ = load_toolkit_module_by_id(id)
             request.app.state.TOOLS[id] = toolkit_module
 
         if hasattr(toolkit_module, "UserValves"):
@@ -348,7 +343,7 @@ async def update_toolkit_user_valves_by_id(
         if id in request.app.state.TOOLS:
             toolkit_module = request.app.state.TOOLS[id]
         else:
-            toolkit_module, frontmatter = load_toolkit_module_by_id(id)
+            toolkit_module, _ = load_toolkit_module_by_id(id)
             request.app.state.TOOLS[id] = toolkit_module
 
         if hasattr(toolkit_module, "UserValves"):
@@ -365,7 +360,7 @@ async def update_toolkit_user_valves_by_id(
                 print(e)
                 raise HTTPException(
                     status_code=status.HTTP_400_BAD_REQUEST,
-                    detail=ERROR_MESSAGES.DEFAULT(e),
+                    detail=ERROR_MESSAGES.DEFAULT(str(e)),
                 )
         else:
             raise HTTPException(

backend/apps/webui/routers/users.py

@@ -1,7 +1,7 @@
 from fastapi import Response, Request
 from fastapi import Depends, FastAPI, HTTPException, status
 from datetime import datetime, timedelta
-from typing import List, Union, Optional
+from typing import Union, Optional
 
 from fastapi import APIRouter
 from pydantic import BaseModel
@@ -39,7 +39,7 @@ router = APIRouter()
 ############################
 
 
-@router.get("/", response_model=List[UserModel])
+@router.get("/", response_model=list[UserModel])
 async def get_users(skip: int = 0, limit: int = 50, user=Depends(get_admin_user)):
     return Users.get_users(skip, limit)
 

backend/apps/webui/routers/utils.py

@@ -17,7 +17,7 @@ from utils.misc import calculate_sha256, get_gravatar_url
 
 from config import OLLAMA_BASE_URLS, DATA_DIR, UPLOAD_DIR, ENABLE_ADMIN_EXPORT
 from constants import ERROR_MESSAGES
-from typing import List
+
 
 router = APIRouter()
 
@@ -57,7 +57,7 @@ async def get_html_from_markdown(
 
 class ChatForm(BaseModel):
     title: str
-    messages: List[dict]
+    messages: list[dict]
 
 
 @router.post("/pdf")

backend/apps/webui/utils.py

@@ -1,6 +1,8 @@
 from importlib import util
 import os
 import re
+import sys
+import subprocess
 
 from config import TOOLS_DIR, FUNCTIONS_DIR
 
@@ -52,6 +54,7 @@ def load_toolkit_module_by_id(toolkit_id):
     frontmatter = extract_frontmatter(toolkit_path)
 
     try:
+        install_frontmatter_requirements(frontmatter.get("requirements", ""))
         spec.loader.exec_module(module)
         print(f"Loaded module: {module.__name__}")
         if hasattr(module, "Tools"):
@@ -73,6 +76,7 @@ def load_function_module_by_id(function_id):
     frontmatter = extract_frontmatter(function_path)
 
     try:
+        install_frontmatter_requirements(frontmatter.get("requirements", ""))
         spec.loader.exec_module(module)
         print(f"Loaded module: {module.__name__}")
         if hasattr(module, "Pipe"):
@@ -88,3 +92,13 @@ def load_function_module_by_id(function_id):
         # Move the file to the error folder
         os.rename(function_path, f"{function_path}.error")
         raise e
+
+
+def install_frontmatter_requirements(requirements):
+    if requirements:
+        req_list = [req.strip() for req in requirements.split(",")]
+        for req in req_list:
+            print(f"Installing requirement: {req}")
+            subprocess.check_call([sys.executable, "-m", "pip", "install", req])
+    else:
+        print("No requirements found in frontmatter.")

backend/config.py

@@ -104,7 +104,7 @@ ENV = os.environ.get("ENV", "dev")
 
 try:
     PACKAGE_DATA = json.loads((BASE_DIR / "package.json").read_text())
-except:
+except Exception:
     try:
         PACKAGE_DATA = {"version": importlib.metadata.version("open-webui")}
     except importlib.metadata.PackageNotFoundError:
@@ -137,7 +137,7 @@ try:
     with open(str(changelog_path.absolute()), "r", encoding="utf8") as file:
         changelog_content = file.read()
 
-except:
+except Exception:
     changelog_content = (pkgutil.get_data("open_webui", "CHANGELOG.md") or b"").decode()
 
 
@@ -202,12 +202,12 @@ if RESET_CONFIG_ON_START:
         os.remove(f"{DATA_DIR}/config.json")
         with open(f"{DATA_DIR}/config.json", "w") as f:
             f.write("{}")
-    except:
+    except Exception:
         pass
 
 try:
     CONFIG_DATA = json.loads((DATA_DIR / "config.json").read_text())
-except:
+except Exception:
     CONFIG_DATA = {}
 
 
@@ -433,6 +433,12 @@ OAUTH_PICTURE_CLAIM = PersistentConfig(
     os.environ.get("OAUTH_PICTURE_CLAIM", "picture"),
 )
 
+OAUTH_EMAIL_CLAIM = PersistentConfig(
+    "OAUTH_EMAIL_CLAIM",
+    "oauth.oidc.email_claim",
+    os.environ.get("OAUTH_EMAIL_CLAIM", "email"),
+)
+
 
 def load_oauth_providers():
     OAUTH_PROVIDERS.clear()
@@ -641,7 +647,7 @@ if AIOHTTP_CLIENT_TIMEOUT == "":
 else:
     try:
         AIOHTTP_CLIENT_TIMEOUT = int(AIOHTTP_CLIENT_TIMEOUT)
-    except:
+    except Exception:
         AIOHTTP_CLIENT_TIMEOUT = 300
 
 
@@ -721,7 +727,7 @@ try:
     OPENAI_API_KEY = OPENAI_API_KEYS.value[
         OPENAI_API_BASE_URLS.value.index("https://api.openai.com/v1")
     ]
-except:
+except Exception:
     pass
 
 OPENAI_API_BASE_URL = "https://api.openai.com/v1"
@@ -824,6 +830,10 @@ WEBHOOK_URL = PersistentConfig(
 
 ENABLE_ADMIN_EXPORT = os.environ.get("ENABLE_ADMIN_EXPORT", "True").lower() == "true"
 
+ENABLE_ADMIN_CHAT_ACCESS = (
+    os.environ.get("ENABLE_ADMIN_CHAT_ACCESS", "True").lower() == "true"
+)
+
 ENABLE_COMMUNITY_SHARING = PersistentConfig(
     "ENABLE_COMMUNITY_SHARING",
     "ui.enable_community_sharing",
@@ -1033,7 +1043,7 @@ RAG_EMBEDDING_MODEL = PersistentConfig(
     "rag.embedding_model",
     os.environ.get("RAG_EMBEDDING_MODEL", "sentence-transformers/all-MiniLM-L6-v2"),
 )
-log.info(f"Embedding model set: {RAG_EMBEDDING_MODEL.value}"),
+log.info(f"Embedding model set: {RAG_EMBEDDING_MODEL.value}")
 
 RAG_EMBEDDING_MODEL_AUTO_UPDATE = (
     os.environ.get("RAG_EMBEDDING_MODEL_AUTO_UPDATE", "").lower() == "true"
@@ -1055,7 +1065,7 @@ RAG_RERANKING_MODEL = PersistentConfig(
     os.environ.get("RAG_RERANKING_MODEL", ""),
 )
 if RAG_RERANKING_MODEL.value != "":
-    log.info(f"Reranking model set: {RAG_RERANKING_MODEL.value}"),
+    log.info(f"Reranking model set: {RAG_RERANKING_MODEL.value}")
 
 RAG_RERANKING_MODEL_AUTO_UPDATE = (
     os.environ.get("RAG_RERANKING_MODEL_AUTO_UPDATE", "").lower() == "true"
@@ -1317,7 +1327,7 @@ COMFYUI_FLUX_WEIGHT_DTYPE = PersistentConfig(
 COMFYUI_FLUX_FP8_CLIP = PersistentConfig(
     "COMFYUI_FLUX_FP8_CLIP",
     "image_generation.comfyui.flux_fp8_clip",
-    os.getenv("COMFYUI_FLUX_FP8_CLIP", ""),
+    os.environ.get("COMFYUI_FLUX_FP8_CLIP", "").lower() == "true",
 )
 
 IMAGES_OPENAI_API_BASE_URL = PersistentConfig(

backend/main.py

@@ -51,7 +51,7 @@ from apps.webui.internal.db import Session
 
 
 from pydantic import BaseModel
-from typing import List, Optional
+from typing import Optional
 
 from apps.webui.models.auths import Auths
 from apps.webui.models.models import Models
@@ -116,6 +116,7 @@ from config import (
     WEBUI_SECRET_KEY,
     WEBUI_SESSION_COOKIE_SAME_SITE,
     WEBUI_SESSION_COOKIE_SECURE,
+    ENABLE_ADMIN_CHAT_ACCESS,
     AppConfig,
 )
 
@@ -957,7 +958,7 @@ async def get_all_models():
 
     custom_models = Models.get_all_models()
     for custom_model in custom_models:
-        if custom_model.base_model_id == None:
+        if custom_model.base_model_id is None:
             for model in models:
                 if (
                     custom_model.id == model["id"]
@@ -1662,7 +1663,7 @@ async def get_pipelines_list(user=Depends(get_admin_user)):
     urlIdxs = [
         idx
         for idx, response in enumerate(responses)
-        if response != None and "pipelines" in response
+        if response is not None and "pipelines" in response
     ]
 
     return {
@@ -1723,7 +1724,7 @@ async def upload_pipeline(
                 res = r.json()
                 if "detail" in res:
                     detail = res["detail"]
-            except:
+            except Exception:
                 pass
 
         raise HTTPException(
@@ -1769,7 +1770,7 @@ async def add_pipeline(form_data: AddPipelineForm, user=Depends(get_admin_user))
                 res = r.json()
                 if "detail" in res:
                     detail = res["detail"]
-            except:
+            except Exception:
                 pass
 
         raise HTTPException(
@@ -1811,7 +1812,7 @@ async def delete_pipeline(form_data: DeletePipelineForm, user=Depends(get_admin_
                 res = r.json()
                 if "detail" in res:
                     detail = res["detail"]
-            except:
+            except Exception:
                 pass
 
         raise HTTPException(
@@ -1844,7 +1845,7 @@ async def get_pipelines(urlIdx: Optional[int] = None, user=Depends(get_admin_use
                 res = r.json()
                 if "detail" in res:
                     detail = res["detail"]
-            except:
+            except Exception:
                 pass
 
         raise HTTPException(
@@ -1859,7 +1860,6 @@ async def get_pipeline_valves(
     pipeline_id: str,
     user=Depends(get_admin_user),
 ):
-    models = await get_all_models()
     r = None
     try:
         url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
@@ -1883,7 +1883,7 @@ async def get_pipeline_valves(
                 res = r.json()
                 if "detail" in res:
                     detail = res["detail"]
-            except:
+            except Exception:
                 pass
 
         raise HTTPException(
@@ -1898,8 +1898,6 @@ async def get_pipeline_valves_spec(
     pipeline_id: str,
     user=Depends(get_admin_user),
 ):
-    models = await get_all_models()
-
     r = None
     try:
         url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
@@ -1922,7 +1920,7 @@ async def get_pipeline_valves_spec(
                 res = r.json()
                 if "detail" in res:
                     detail = res["detail"]
-            except:
+            except Exception:
                 pass
 
         raise HTTPException(
@@ -1938,8 +1936,6 @@ async def update_pipeline_valves(
     form_data: dict,
     user=Depends(get_admin_user),
 ):
-    models = await get_all_models()
-
     r = None
     try:
         url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
@@ -1967,7 +1963,7 @@ async def update_pipeline_valves(
                 res = r.json()
                 if "detail" in res:
                     detail = res["detail"]
-            except:
+            except Exception:
                 pass
 
         raise HTTPException(
@@ -2001,6 +1997,7 @@ async def get_app_config():
             "enable_image_generation": images_app.state.config.ENABLED,
             "enable_community_sharing": webui_app.state.config.ENABLE_COMMUNITY_SHARING,
             "enable_admin_export": ENABLE_ADMIN_EXPORT,
+            "enable_admin_chat_access": ENABLE_ADMIN_CHAT_ACCESS,
         },
         "audio": {
             "tts": {
@@ -2030,7 +2027,7 @@ async def get_model_filter_config(user=Depends(get_admin_user)):
 
 class ModelFilterConfigForm(BaseModel):
     enabled: bool
-    models: List[str]
+    models: list[str]
 
 
 @app.post("/api/config/model/filter")
@@ -2068,7 +2065,7 @@ async def update_webhook_url(form_data: UrlForm, user=Depends(get_admin_user)):
 
 
 @app.get("/api/version")
-async def get_app_config():
+async def get_app_version():
     return {
         "version": VERSION,
     }
@@ -2091,7 +2088,7 @@ async def get_app_latest_release_version():
                 latest_version = data["tag_name"]
 
                 return {"current": VERSION, "latest": latest_version[1:]}
-    except aiohttp.ClientError as e:
+    except aiohttp.ClientError:
         raise HTTPException(
             status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
             detail=ERROR_MESSAGES.RATE_LIMIT_EXCEEDED,
@@ -2161,7 +2158,8 @@ async def oauth_callback(provider: str, request: Request, response: Response):
         log.warning(f"OAuth callback failed, sub is missing: {user_data}")
         raise HTTPException(400, detail=ERROR_MESSAGES.INVALID_CRED)
     provider_sub = f"{provider}@{sub}"
-    email = user_data.get("email", "").lower()
+    email_claim = webui_app.state.config.OAUTH_EMAIL_CLAIM
+    email = user_data.get(email_claim, "").lower()
     # We currently mandate that email addresses are provided
     if not email:
         log.warning(f"OAuth callback failed, email is missing: {user_data}")

backend/requirements.txt

@@ -11,7 +11,8 @@ python-jose==3.3.0
 passlib[bcrypt]==1.7.4
 
 requests==2.32.3
-aiohttp==3.9.5
+aiohttp==3.10.2
+
 sqlalchemy==2.0.31
 alembic==1.13.2
 peewee==3.17.6
@@ -19,10 +20,10 @@ peewee-migrate==1.12.2
 psycopg2-binary==2.9.9
 PyMySQL==1.1.1
 bcrypt==4.1.3
-SQLAlchemy
+
 pymongo
 redis
-boto3==1.34.110
+boto3==1.34.153
 
 argon2-cffi==23.1.0
 APScheduler==3.10.4
@@ -33,16 +34,16 @@ anthropic
 google-generativeai==0.7.2
 tiktoken
 
-langchain==0.2.11
+langchain==0.2.12
 langchain-community==0.2.10
 langchain-chroma==0.1.2
 
 fake-useragent==1.5.1
-chromadb==0.5.4
+chromadb==0.5.5
 sentence-transformers==3.0.1
-pypdf==4.2.0
+pypdf==4.3.1
 docx2txt==0.8
-python-pptx==0.6.23
+python-pptx==1.0.0
 unstructured==0.15.0
 Markdown==3.6
 pypandoc==1.13
@@ -50,7 +51,7 @@ pandas==2.2.2
 openpyxl==3.1.5
 pyxlsb==1.0.10
 xlrd==2.0.1
-validators==0.28.1
+validators==0.33.0
 psutil
 
 opencv-python-headless==4.10.0.84
@@ -61,11 +62,11 @@ rank-bm25==0.2.2
 
 faster-whisper==1.0.2
 
-PyJWT[crypto]==2.8.0
+PyJWT[crypto]==2.9.0
 authlib==1.3.1
 
-black==24.4.2
-langfuse==2.39.2
+black==24.8.0
+langfuse==2.43.3
 youtube-transcript-api==0.6.2
 pytube==15.0.0
 
@@ -75,5 +76,5 @@ duckduckgo-search~=6.2.1
 
 ## Tests
 docker~=7.1.0
-pytest~=8.2.2
+pytest~=8.3.2
 pytest-docker~=3.1.1

backend/start.sh

@@ -30,7 +30,6 @@ if [[ "${USE_CUDA_DOCKER,,}" == "true" ]]; then
   export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib/python3.11/site-packages/torch/lib:/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib"
 fi
 
-
 # Check if SPACE_ID is set, if so, configure for space
 if [ -n "$SPACE_ID" ]; then
   echo "Configuring for HuggingFace Space deployment"

backend/utils/misc.py

@@ -2,12 +2,14 @@ from pathlib import Path
 import hashlib
 import re
 from datetime import timedelta
-from typing import Optional, List, Tuple
+from typing import Optional, Callable
 import uuid
 import time
 
+from utils.task import prompt_template
 
-def get_last_user_message_item(messages: List[dict]) -> Optional[dict]:
+
+def get_last_user_message_item(messages: list[dict]) -> Optional[dict]:
     for message in reversed(messages):
         if message["role"] == "user":
             return message
@@ -24,7 +26,7 @@ def get_content_from_message(message: dict) -> Optional[str]:
     return None
 
 
-def get_last_user_message(messages: List[dict]) -> Optional[str]:
+def get_last_user_message(messages: list[dict]) -> Optional[str]:
     message = get_last_user_message_item(messages)
     if message is None:
         return None
@@ -32,31 +34,31 @@ def get_last_user_message(messages: List[dict]) -> Optional[str]:
     return get_content_from_message(message)
 
 
-def get_last_assistant_message(messages: List[dict]) -> Optional[str]:
+def get_last_assistant_message(messages: list[dict]) -> Optional[str]:
     for message in reversed(messages):
         if message["role"] == "assistant":
             return get_content_from_message(message)
     return None
 
 
-def get_system_message(messages: List[dict]) -> Optional[dict]:
+def get_system_message(messages: list[dict]) -> Optional[dict]:
     for message in messages:
         if message["role"] == "system":
             return message
     return None
 
 
-def remove_system_message(messages: List[dict]) -> List[dict]:
+def remove_system_message(messages: list[dict]) -> list[dict]:
     return [message for message in messages if message["role"] != "system"]
 
 
-def pop_system_message(messages: List[dict]) -> Tuple[Optional[dict], List[dict]]:
+def pop_system_message(messages: list[dict]) -> tuple[Optional[dict], list[dict]]:
     return get_system_message(messages), remove_system_message(messages)
 
 
 def prepend_to_first_user_message_content(
-    content: str, messages: List[dict]
-) -> List[dict]:
+    content: str, messages: list[dict]
+) -> list[dict]:
     for message in messages:
         if message["role"] == "user":
             if isinstance(message["content"], list):
@@ -69,7 +71,7 @@ def prepend_to_first_user_message_content(
     return messages
 
 
-def add_or_update_system_message(content: str, messages: List[dict]):
+def add_or_update_system_message(content: str, messages: list[dict]):
     """
     Adds a new system message at the beginning of the messages list
     or updates the existing system message at the beginning.
@@ -97,18 +99,101 @@ def openai_chat_message_template(model: str):
     }
 
 
-def openai_chat_chunk_message_template(model: str, message: str):
+def openai_chat_chunk_message_template(model: str, message: str) -> dict:
     template = openai_chat_message_template(model)
     template["object"] = "chat.completion.chunk"
     template["choices"][0]["delta"] = {"content": message}
     return template
 
 
-def openai_chat_completion_message_template(model: str, message: str):
+def openai_chat_completion_message_template(model: str, message: str) -> dict:
     template = openai_chat_message_template(model)
     template["object"] = "chat.completion"
     template["choices"][0]["message"] = {"content": message, "role": "assistant"}
     template["choices"][0]["finish_reason"] = "stop"
+    return template
+
+
+# inplace function: form_data is modified
+def apply_model_system_prompt_to_body(params: dict, form_data: dict, user) -> dict:
+    system = params.get("system", None)
+    if not system:
+        return form_data
+
+    if user:
+        template_params = {
+            "user_name": user.name,
+            "user_location": user.info.get("location") if user.info else None,
+        }
+    else:
+        template_params = {}
+    system = prompt_template(system, **template_params)
+    form_data["messages"] = add_or_update_system_message(
+        system, form_data.get("messages", [])
+    )
+    return form_data
+
+
+# inplace function: form_data is modified
+def apply_model_params_to_body(
+    params: dict, form_data: dict, mappings: dict[str, Callable]
+) -> dict:
+    if not params:
+        return form_data
+
+    for key, cast_func in mappings.items():
+        if (value := params.get(key)) is not None:
+            form_data[key] = cast_func(value)
+
+    return form_data
+
+
+# inplace function: form_data is modified
+def apply_model_params_to_body_openai(params: dict, form_data: dict) -> dict:
+    mappings = {
+        "temperature": float,
+        "top_p": int,
+        "max_tokens": int,
+        "frequency_penalty": int,
+        "seed": lambda x: x,
+        "stop": lambda x: [bytes(s, "utf-8").decode("unicode_escape") for s in x],
+    }
+    return apply_model_params_to_body(params, form_data, mappings)
+
+
+def apply_model_params_to_body_ollama(params: dict, form_data: dict) -> dict:
+    opts = [
+        "temperature",
+        "top_p",
+        "seed",
+        "mirostat",
+        "mirostat_eta",
+        "mirostat_tau",
+        "num_ctx",
+        "num_batch",
+        "num_keep",
+        "repeat_last_n",
+        "tfs_z",
+        "top_k",
+        "min_p",
+        "use_mmap",
+        "use_mlock",
+        "num_thread",
+        "num_gpu",
+    ]
+    mappings = {i: lambda x: x for i in opts}
+    form_data = apply_model_params_to_body(params, form_data, mappings)
+
+    name_differences = {
+        "max_tokens": "num_predict",
+        "frequency_penalty": "repeat_penalty",
+    }
+
+    for key, value in name_differences.items():
+        if (param := params.get(key, None)) is not None:
+            form_data[value] = param
+
+    return form_data
 
 
 def get_gravatar_url(email):

backend/utils/task.py

@@ -6,7 +6,7 @@ from typing import Optional
 
 
 def prompt_template(
-    template: str, user_name: str = None, user_location: str = None
+    template: str, user_name: Optional[str] = None, user_location: Optional[str] = None
 ) -> str:
     # Get the current date
     current_date = datetime.now()
@@ -83,7 +83,6 @@ def title_generation_template(
 def search_query_generation_template(
     template: str, prompt: str, user: Optional[dict] = None
 ) -> str:
-
     def replacement_function(match):
         full_match = match.group(0)
         start_length = match.group(1)

backend/utils/tools.py

@@ -1,5 +1,5 @@
 import inspect
-from typing import get_type_hints, List, Dict, Any
+from typing import get_type_hints
 
 
 def doc_to_dict(docstring):
@@ -16,7 +16,7 @@ def doc_to_dict(docstring):
     return ret_dict
 
 
-def get_tools_specs(tools) -> List[dict]:
+def get_tools_specs(tools) -> list[dict]:
     function_list = [
         {"name": func, "function": getattr(tools, func)}
         for func in dir(tools)

backend/utils/utils.py

@@ -1,15 +1,12 @@
 from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
 from fastapi import HTTPException, status, Depends, Request
-from sqlalchemy.orm import Session
 
 from apps.webui.models.users import Users
 
-from pydantic import BaseModel
 from typing import Union, Optional
 from constants import ERROR_MESSAGES
 from passlib.context import CryptContext
 from datetime import datetime, timedelta
-import requests
 import jwt
 import uuid
 import logging
@@ -54,7 +51,7 @@ def decode_token(token: str) -> Optional[dict]:
     try:
         decoded = jwt.decode(token, SESSION_SECRET, algorithms=[ALGORITHM])
         return decoded
-    except Exception as e:
+    except Exception:
         return None
 
 
@@ -71,7 +68,7 @@ def get_http_authorization_cred(auth_header: str):
     try:
         scheme, credentials = auth_header.split(" ")
         return HTTPAuthorizationCredentials(scheme=scheme, credentials=credentials)
-    except:
+    except Exception:
         raise ValueError(ERROR_MESSAGES.INVALID_TOKEN)
 
 
@@ -96,7 +93,7 @@ def get_current_user(
 
     # auth by jwt token
     data = decode_token(token)
-    if data != None and "id" in data:
+    if data is not None and "id" in data:
         user = Users.get_user_by_id(data["id"])
         if user is None:
             raise HTTPException(

cypress/e2e/chat.cy.ts

@@ -38,9 +38,10 @@ describe('Settings', () => {
 			// User's message should be visible
 			cy.get('.chat-user').should('exist');
 			// Wait for the response
-			cy.get('.chat-assistant', { timeout: 120_000 }) // .chat-assistant is created after the first token is received
-				.find('div[aria-label="Generation Info"]', { timeout: 120_000 }) // Generation Info is created after the stop token is received
-				.should('exist');
+			// .chat-assistant is created after the first token is received
+			cy.get('.chat-assistant', { timeout: 10_000 }).should('exist');
+			// Generation Info is created after the stop token is received
+			cy.get('div[aria-label="Generation Info"]', { timeout: 120_000 }).should('exist');
 		});
 
 		it('user can share chat', () => {
@@ -57,21 +58,24 @@ describe('Settings', () => {
 			// User's message should be visible
 			cy.get('.chat-user').should('exist');
 			// Wait for the response
-			cy.get('.chat-assistant', { timeout: 120_000 }) // .chat-assistant is created after the first token is received
-				.find('div[aria-label="Generation Info"]', { timeout: 120_000 }) // Generation Info is created after the stop token is received
-				.should('exist');
+			// .chat-assistant is created after the first token is received
+			cy.get('.chat-assistant', { timeout: 10_000 }).should('exist');
+			// Generation Info is created after the stop token is received
+			cy.get('div[aria-label="Generation Info"]', { timeout: 120_000 }).should('exist');
 			// spy on requests
 			const spy = cy.spy();
-			cy.intercept('GET', '/api/v1/chats/*', spy);
+			cy.intercept('POST', '/api/v1/chats/**/share', spy);
 			// Open context menu
 			cy.get('#chat-context-menu-button').click();
 			// Click share button
 			cy.get('#chat-share-button').click();
 			// Check if the share dialog is visible
 			cy.get('#copy-and-share-chat-button').should('exist');
-			cy.wrap({}, { timeout: 5000 }).should(() => {
-				// Check if the request was made twice (once for to replace chat object and once more due to change event)
-				expect(spy).to.be.callCount(2);
+			// Click the copy button
+			cy.get('#copy-and-share-chat-button').click();
+			cy.wrap({}, { timeout: 5_000 }).should(() => {
+				// Check if the share request was made
+				expect(spy).to.be.callCount(1);
 			});
 		});
 
@@ -89,9 +93,10 @@ describe('Settings', () => {
 			// User's message should be visible
 			cy.get('.chat-user').should('exist');
 			// Wait for the response
-			cy.get('.chat-assistant', { timeout: 120_000 }) // .chat-assistant is created after the first token is received
-				.find('div[aria-label="Generation Info"]', { timeout: 120_000 }) // Generation Info is created after the stop token is received
-				.should('exist');
+			// .chat-assistant is created after the first token is received
+			cy.get('.chat-assistant', { timeout: 10_000 }).should('exist');
+			// Generation Info is created after the stop token is received
+			cy.get('div[aria-label="Generation Info"]', { timeout: 120_000 }).should('exist');
 			// Click on the generate image button
 			cy.get('[aria-label="Generate Image"]').click();
 			// Wait for image to be visible

docs/CONTRIBUTING.md

@@ -22,7 +22,6 @@ Noticed something off? Have an idea? Check our [Issues tab](https://github.com/o
 > [!IMPORTANT]
 >
 > - **Template Compliance:** Please be aware that failure to follow the provided issue template, or not providing the requested information at all, will likely result in your issue being closed without further consideration. This approach is critical for maintaining the manageability and integrity of issue tracking.
->
 > - **Detail is Key:** To ensure your issue is understood and can be effectively addressed, it's imperative to include comprehensive details. Descriptions should be clear, including steps to reproduce, expected outcomes, and actual results. Lack of sufficient detail may hinder our ability to resolve your issue.
 
 ### 🧭 Scope of Support

docs/SECURITY.md

@@ -11,10 +11,26 @@ Our primary goal is to ensure the protection and confidentiality of sensitive da
 
 ## Reporting a Vulnerability
 
-If you discover a security issue within our system, please notify us immediately via a pull request or contact us on discord.
+We appreciate the community's interest in identifying potential vulnerabilities. However, effective immediately, we will **not** accept low-effort vulnerability reports. To ensure that submissions are constructive and actionable, please adhere to the following guidelines:
+
+1. **No Vague Reports**: Submissions such as "I found a vulnerability" without any details will be treated as spam and will not be accepted.
+
+2. **In-Depth Understanding Required**: Reports must reflect a clear understanding of the codebase and provide specific details about the vulnerability, including the affected components and potential impacts.
+
+3. **Proof of Concept (PoC) is Mandatory**: Each submission must include a well-documented proof of concept (PoC) that demonstrates the vulnerability. If confidentiality is a concern, reporters are encouraged to create a private fork of the repository and share access with the maintainers. Reports lacking valid evidence will be disregarded.
+
+4. **Required Patch Submission**: Along with the PoC, reporters must provide a patch or actionable steps to remediate the identified vulnerability. This helps us evaluate and implement fixes rapidly.
+
+5. **Streamlined Merging Process**: When vulnerability reports meet the above criteria, we can consider them for immediate merging, similar to regular pull requests. Well-structured and thorough submissions will expedite the process of enhancing our security.
+
+Submissions that do not meet these criteria will be closed, and repeat offenders may face a ban from future submissions. We aim to create a respectful and constructive reporting environment, where high-quality submissions foster better security for everyone.
 
 ## Product Security
 
-We regularly audit our internal processes and system's architecture for vulnerabilities using a combination of automated and manual testing techniques.
+We regularly audit our internal processes and system architecture for vulnerabilities using a combination of automated and manual testing techniques. We are also planning to implement SAST and SCA scans in our project soon.
 
-We are planning on implementing SAST and SCA scans in our project soon.
+For immediate concerns or detailed reports that meet our guidelines, please create an issue in our [issue tracker](/open-webui/open-webui/issues) or contact us on [Discord](https://discord.gg/5rJgQTnV4s).
+
+---
+
+_Last updated on **2024-08-06**._

package.json

@@ -1,6 +1,6 @@
 {
 	"name": "open-webui",
-	"version": "0.3.11",
+	"version": "0.3.13",
 	"private": true,
 	"scripts": {
 		"dev": "npm run pyodide:fetch && vite dev --host",
@@ -20,30 +20,31 @@
 		"pyodide:fetch": "node scripts/prepare-pyodide.js"
 	},
 	"devDependencies": {
-		"@sveltejs/adapter-auto": "^2.0.0",
-		"@sveltejs/adapter-static": "^2.0.3",
-		"@sveltejs/kit": "^1.30.0",
-		"@tailwindcss/typography": "^0.5.10",
+		"@sveltejs/adapter-auto": "3.2.2",
+		"@sveltejs/adapter-static": "^3.0.2",
+		"@sveltejs/kit": "^2.5.20",
+		"@sveltejs/vite-plugin-svelte": "^3.1.1",
+		"@tailwindcss/typography": "^0.5.13",
 		"@types/bun": "latest",
 		"@typescript-eslint/eslint-plugin": "^6.17.0",
 		"@typescript-eslint/parser": "^6.17.0",
 		"autoprefixer": "^10.4.16",
 		"cypress": "^13.8.1",
 		"eslint": "^8.56.0",
-		"eslint-config-prettier": "^8.5.0",
-		"eslint-plugin-cypress": "^3.0.2",
-		"eslint-plugin-svelte": "^2.30.0",
-		"i18next-parser": "^8.13.0",
+		"eslint-config-prettier": "^9.1.0",
+		"eslint-plugin-cypress": "^3.4.0",
+		"eslint-plugin-svelte": "^2.43.0",
+		"i18next-parser": "^9.0.1",
 		"postcss": "^8.4.31",
-		"prettier": "^2.8.0",
-		"prettier-plugin-svelte": "^2.10.1",
-		"svelte": "^4.0.5",
-		"svelte-check": "^3.4.3",
+		"prettier": "^3.3.3",
+		"prettier-plugin-svelte": "^3.2.6",
+		"svelte": "^4.2.18",
+		"svelte-check": "^3.8.5",
 		"svelte-confetti": "^1.3.2",
 		"tailwindcss": "^3.3.3",
 		"tslib": "^2.4.1",
-		"typescript": "^5.0.0",
-		"vite": "^4.4.2",
+		"typescript": "^5.5.4",
+		"vite": "^5.3.5",
 		"vitest": "^1.6.0"
 	},
 	"type": "module",
@@ -52,7 +53,7 @@
 		"@codemirror/lang-python": "^6.1.6",
 		"@codemirror/theme-one-dark": "^6.1.2",
 		"@pyscript/core": "^0.4.32",
-		"@sveltejs/adapter-node": "^1.3.1",
+		"@sveltejs/adapter-node": "^2.0.0",
 		"async": "^3.2.5",
 		"bits-ui": "^0.19.7",
 		"codemirror": "^6.0.1",
@@ -69,6 +70,7 @@
 		"js-sha256": "^0.10.1",
 		"katex": "^0.16.9",
 		"marked": "^9.1.0",
+		"marked-katex-extension": "^5.1.1",
 		"mermaid": "^10.9.1",
 		"pyodide": "^0.26.1",
 		"socket.io-client": "^4.2.0",
@@ -77,5 +79,9 @@
 		"tippy.js": "^6.3.7",
 		"turndown": "^7.2.0",
 		"uuid": "^9.0.1"
+	},
+	"engines": {
+		"node": ">=18.13.0 <=21.x.x",
+		"npm": ">=6.0.0"
 	}
 }

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "open-webui"
-description = "Open WebUI (Formerly Ollama WebUI)"
+description = "Open WebUI"
 authors = [
     { name = "Timothy Jaeryang Baek", email = "tim@openwebui.com" }
 ]
@@ -8,67 +8,82 @@ license = { file = "LICENSE" }
 dependencies = [
     "fastapi==0.111.0",
     "uvicorn[standard]==0.22.0",
-    "pydantic==2.7.1",
+    "pydantic==2.8.2",
     "python-multipart==0.0.9",
 
     "Flask==3.0.3",
     "Flask-Cors==4.0.1",
 
-    "python-socketio==5.11.2",
+    "python-socketio==5.11.3",
     "python-jose==3.3.0",
     "passlib[bcrypt]==1.7.4",
 
-    "requests==2.32.2",
-    "aiohttp==3.9.5",
-    "peewee==3.17.5",
+    "requests==2.32.3",
+    "aiohttp==3.10.2",
+
+    "sqlalchemy==2.0.31",
+    "alembic==1.13.2",
+    "peewee==3.17.6",
     "peewee-migrate==1.12.2",
     "psycopg2-binary==2.9.9",
     "PyMySQL==1.1.1",
     "bcrypt==4.1.3",
 
-    "boto3==1.34.110",
+    "pymongo",
+    "redis",
+    "boto3==1.34.153",
 
     "argon2-cffi==23.1.0",
     "APScheduler==3.10.4",
-    "google-generativeai==0.5.4",
 
-    "langchain==0.2.0",
-    "langchain-community==0.2.9",
-    "langchain-chroma==0.1.1",
+    "openai",
+    "anthropic",
+    "google-generativeai==0.7.2",
+    "tiktoken",
+
+    "langchain==0.2.12",
+    "langchain-community==0.2.10",
+    "langchain-chroma==0.1.2",
 
     "fake-useragent==1.5.1",
-    "chromadb==0.5.0",
-    "sentence-transformers==2.7.0",
-    "pypdf==4.2.0",
+    "chromadb==0.5.5",
+    "sentence-transformers==3.0.1",
+    "pypdf==4.3.1",
     "docx2txt==0.8",
-    "unstructured==0.14.0",
+    "python-pptx==1.0.0",
+    "unstructured==0.15.0",
     "Markdown==3.6",
     "pypandoc==1.13",
     "pandas==2.2.2",
-    "openpyxl==3.1.2",
+    "openpyxl==3.1.5",
     "pyxlsb==1.0.10",
     "xlrd==2.0.1",
-    "validators==0.28.1",
+    "validators==0.33.0",
+    "psutil",
 
-    "opencv-python-headless==4.9.0.80",
-    "rapidocr-onnxruntime==1.3.22",
+    "opencv-python-headless==4.10.0.84",
+    "rapidocr-onnxruntime==1.3.24",
 
     "fpdf2==2.7.9",
     "rank-bm25==0.2.2",
 
     "faster-whisper==1.0.2",
 
-    "PyJWT[crypto]==2.8.0",
+    "PyJWT[crypto]==2.9.0",
     "authlib==1.3.1",
 
-    "black==24.4.2",
-    "langfuse==2.33.0",
+    "black==24.8.0",
+    "langfuse==2.43.3",
     "youtube-transcript-api==0.6.2",
     "pytube==15.0.0",
+
     "extract_msg",
     "pydub",
-    "duckduckgo-search~=6.1.5"
+    "duckduckgo-search~=6.2.1",
 
+    "docker~=7.1.0",
+    "pytest~=8.2.2",
+    "pytest-docker~=3.1.1"
 ]
 readme = "README.md"
 requires-python = ">= 3.11, < 3.12.0a1"

requirements-dev.lock

@@ -10,16 +10,25 @@
 #   universal: false
 
 -e file:.
-aiohttp==3.9.5
+aiohappyeyeballs==2.3.5
+    # via aiohttp
+aiohttp==3.10.2
     # via langchain
     # via langchain-community
     # via open-webui
 aiosignal==1.3.1
     # via aiohttp
+alembic==1.13.2
+    # via open-webui
 annotated-types==0.6.0
     # via pydantic
-anyio==4.3.0
+anthropic==0.32.0
+    # via open-webui
+anyio==4.4.0
+    # via anthropic
     # via httpx
+    # via langfuse
+    # via openai
     # via starlette
     # via watchfiles
 apscheduler==3.10.4
@@ -32,6 +41,7 @@ asgiref==3.8.1
     # via opentelemetry-instrumentation-asgi
 attrs==23.2.0
     # via aiohttp
+    # via pytest-docker
 authlib==1.3.1
     # via open-webui
 av==11.0.0
@@ -49,13 +59,13 @@ beautifulsoup4==4.12.3
     # via unstructured
 bidict==0.23.1
     # via python-socketio
-black==24.4.2
+black==24.8.0
     # via open-webui
 blinker==1.8.2
     # via flask
-boto3==1.34.110
+boto3==1.34.153
     # via open-webui
-botocore==1.34.110
+botocore==1.34.155
     # via boto3
     # via s3transfer
 build==1.2.1
@@ -76,9 +86,9 @@ chardet==5.2.0
 charset-normalizer==3.3.2
     # via requests
     # via unstructured-client
-chroma-hnswlib==0.7.3
+chroma-hnswlib==0.7.6
     # via chromadb
-chromadb==0.5.0
+chromadb==0.5.5
     # via langchain-chroma
     # via open-webui
 click==8.1.7
@@ -102,7 +112,6 @@ cryptography==42.0.7
 ctranslate2==4.2.1
     # via faster-whisper
 dataclasses-json==0.6.6
-    # via langchain
     # via langchain-community
     # via unstructured
     # via unstructured-client
@@ -113,11 +122,17 @@ defusedxml==0.7.1
 deprecated==1.2.14
     # via opentelemetry-api
     # via opentelemetry-exporter-otlp-proto-grpc
+distro==1.9.0
+    # via anthropic
+    # via openai
 dnspython==2.6.1
     # via email-validator
+    # via pymongo
+docker==7.1.0
+    # via open-webui
 docx2txt==0.8
     # via open-webui
-duckduckgo-search==6.1.5
+duckduckgo-search==6.2.6
     # via open-webui
 easygui==0.98.3
     # via oletools
@@ -166,7 +181,7 @@ frozenlist==1.4.1
 fsspec==2024.3.1
     # via huggingface-hub
     # via torch
-google-ai-generativelanguage==0.6.4
+google-ai-generativelanguage==0.6.6
     # via google-generativeai
 google-api-core==2.19.0
     # via google-ai-generativelanguage
@@ -183,7 +198,7 @@ google-auth==2.29.0
     # via kubernetes
 google-auth-httplib2==0.2.0
     # via google-api-python-client
-google-generativeai==0.5.4
+google-generativeai==0.7.2
     # via open-webui
 googleapis-common-protos==1.63.0
     # via google-api-core
@@ -208,8 +223,11 @@ httplib2==0.22.0
 httptools==0.6.1
     # via uvicorn
 httpx==0.27.0
+    # via anthropic
+    # via chromadb
     # via fastapi
     # via langfuse
+    # via openai
 huggingface-hub==0.23.0
     # via faster-whisper
     # via sentence-transformers
@@ -229,12 +247,16 @@ importlib-metadata==7.0.0
     # via opentelemetry-api
 importlib-resources==6.4.0
     # via chromadb
+iniconfig==2.0.0
+    # via pytest
 itsdangerous==2.2.0
     # via flask
 jinja2==3.1.4
     # via fastapi
     # via flask
     # via torch
+jiter==0.5.0
+    # via anthropic
 jmespath==1.0.1
     # via boto3
     # via botocore
@@ -249,14 +271,14 @@ jsonpointer==2.4
     # via jsonpatch
 kubernetes==29.0.0
     # via chromadb
-langchain==0.2.0
+langchain==0.2.12
     # via langchain-community
     # via open-webui
-langchain-chroma==0.1.1
+langchain-chroma==0.1.2
     # via open-webui
-langchain-community==0.2.0
+langchain-community==0.2.10
     # via open-webui
-langchain-core==0.2.1
+langchain-core==0.2.28
     # via langchain
     # via langchain-chroma
     # via langchain-community
@@ -265,22 +287,26 @@ langchain-text-splitters==0.2.0
     # via langchain
 langdetect==1.0.9
     # via unstructured
-langfuse==2.33.0
+langfuse==2.43.3
     # via open-webui
-langsmith==0.1.57
+langsmith==0.1.96
     # via langchain
     # via langchain-community
     # via langchain-core
 lark==1.1.8
     # via rtfde
 lxml==5.2.2
+    # via python-pptx
     # via unstructured
+mako==1.3.5
+    # via alembic
 markdown==3.6
     # via open-webui
 markdown-it-py==3.0.0
     # via rich
 markupsafe==2.1.5
     # via jinja2
+    # via mako
     # via werkzeug
 marshmallow==3.21.2
     # via dataclasses-json
@@ -339,11 +365,13 @@ onnxruntime==1.17.3
     # via chromadb
     # via faster-whisper
     # via rapidocr-onnxruntime
+openai==1.38.0
+    # via open-webui
 opencv-python==4.9.0.80
     # via rapidocr-onnxruntime
-opencv-python-headless==4.9.0.80
+opencv-python-headless==4.10.0.84
     # via open-webui
-openpyxl==3.1.2
+openpyxl==3.1.5
     # via open-webui
 opentelemetry-api==1.24.0
     # via chromadb
@@ -380,7 +408,6 @@ ordered-set==4.1.0
     # via deepdiff
 orjson==3.10.3
     # via chromadb
-    # via duckduckgo-search
     # via fastapi
     # via langsmith
 overrides==7.7.0
@@ -393,6 +420,7 @@ packaging==23.2
     # via langfuse
     # via marshmallow
     # via onnxruntime
+    # via pytest
     # via transformers
     # via unstructured-client
 pandas==2.2.2
@@ -403,19 +431,24 @@ pathspec==0.12.1
     # via black
 pcodedmp==1.2.6
     # via oletools
-peewee==3.17.5
+peewee==3.17.6
     # via open-webui
     # via peewee-migrate
 peewee-migrate==1.12.2
     # via open-webui
 pillow==10.3.0
     # via fpdf2
+    # via python-pptx
     # via rapidocr-onnxruntime
     # via sentence-transformers
 platformdirs==4.2.1
     # via black
+pluggy==1.5.0
+    # via pytest
 posthog==3.5.0
     # via chromadb
+primp==0.5.5
+    # via duckduckgo-search
 proto-plus==1.23.0
     # via google-ai-generativelanguage
     # via google-api-core
@@ -428,6 +461,9 @@ protobuf==4.25.3
     # via onnxruntime
     # via opentelemetry-proto
     # via proto-plus
+psutil==6.0.0
+    # via open-webui
+    # via unstructured
 psycopg2-binary==2.9.9
     # via open-webui
 pyasn1==0.6.0
@@ -440,7 +476,8 @@ pyclipper==1.3.0.post5
     # via rapidocr-onnxruntime
 pycparser==2.22
     # via cffi
-pydantic==2.7.1
+pydantic==2.8.2
+    # via anthropic
     # via chromadb
     # via fastapi
     # via google-generativeai
@@ -449,30 +486,36 @@ pydantic==2.7.1
     # via langfuse
     # via langsmith
     # via open-webui
-pydantic-core==2.18.2
+    # via openai
+pydantic-core==2.20.1
     # via pydantic
 pydub==0.25.1
     # via open-webui
 pygments==2.18.0
     # via rich
-pyjwt==2.8.0
+pyjwt==2.9.0
     # via open-webui
-pymysql==1.1.0
+pymongo==4.8.0
+    # via open-webui
+pymysql==1.1.1
     # via open-webui
 pypandoc==1.13
     # via open-webui
 pyparsing==2.4.7
     # via httplib2
     # via oletools
-pypdf==4.2.0
+pypdf==4.3.1
     # via open-webui
     # via unstructured-client
 pypika==0.48.9
     # via chromadb
 pyproject-hooks==1.1.0
     # via build
-pyreqwest-impersonate==0.4.7
-    # via duckduckgo-search
+pytest==8.2.2
+    # via open-webui
+    # via pytest-docker
+pytest-docker==3.1.1
+    # via open-webui
 python-dateutil==2.9.0.post0
     # via botocore
     # via kubernetes
@@ -492,7 +535,9 @@ python-magic==0.4.27
 python-multipart==0.0.9
     # via fastapi
     # via open-webui
-python-socketio==5.11.2
+python-pptx==1.0.0
+    # via open-webui
+python-socketio==5.11.3
     # via open-webui
 pytube==15.0.0
     # via open-webui
@@ -516,15 +561,18 @@ rank-bm25==0.2.2
     # via open-webui
 rapidfuzz==3.9.0
     # via unstructured
-rapidocr-onnxruntime==1.3.22
+rapidocr-onnxruntime==1.3.24
     # via open-webui
 red-black-tree-mod==1.20
     # via extract-msg
+redis==5.0.8
+    # via open-webui
 regex==2024.5.10
     # via nltk
+    # via tiktoken
     # via transformers
-requests==2.32.2
-    # via chromadb
+requests==2.32.3
+    # via docker
     # via google-api-core
     # via huggingface-hub
     # via kubernetes
@@ -534,6 +582,7 @@ requests==2.32.2
     # via open-webui
     # via posthog
     # via requests-oauthlib
+    # via tiktoken
     # via transformers
     # via unstructured
     # via unstructured-client
@@ -556,12 +605,12 @@ scikit-learn==1.4.2
 scipy==1.13.0
     # via scikit-learn
     # via sentence-transformers
-sentence-transformers==2.7.0
+sentence-transformers==3.0.1
     # via open-webui
 setuptools==69.5.1
     # via ctranslate2
     # via opentelemetry-instrumentation
-shapely==2.0.4
+shapely==2.0.5
     # via rapidocr-onnxruntime
 shellingham==1.5.4
     # via typer
@@ -577,13 +626,17 @@ six==1.16.0
     # via rapidocr-onnxruntime
     # via unstructured-client
 sniffio==1.3.1
+    # via anthropic
     # via anyio
     # via httpx
+    # via openai
 soupsieve==2.5
     # via beautifulsoup4
-sqlalchemy==2.0.30
+sqlalchemy==2.0.31
+    # via alembic
     # via langchain
     # via langchain-community
+    # via open-webui
 starlette==0.37.2
     # via fastapi
 sympy==1.12
@@ -598,7 +651,10 @@ tenacity==8.3.0
     # via langchain-core
 threadpoolctl==3.5.0
     # via scikit-learn
+tiktoken==0.7.0
+    # via open-webui
 tokenizers==0.15.2
+    # via anthropic
     # via chromadb
     # via faster-whisper
     # via transformers
@@ -609,21 +665,28 @@ tqdm==4.66.4
     # via google-generativeai
     # via huggingface-hub
     # via nltk
+    # via openai
     # via sentence-transformers
     # via transformers
+    # via unstructured
 transformers==4.39.3
     # via sentence-transformers
 typer==0.12.3
     # via chromadb
     # via fastapi-cli
 typing-extensions==4.11.0
+    # via alembic
+    # via anthropic
     # via chromadb
     # via fastapi
     # via google-generativeai
     # via huggingface-hub
+    # via langchain-core
+    # via openai
     # via opentelemetry-sdk
     # via pydantic
     # via pydantic-core
+    # via python-pptx
     # via sqlalchemy
     # via torch
     # via typer
@@ -640,7 +703,7 @@ tzlocal==5.2
     # via extract-msg
 ujson==5.10.0
     # via fastapi
-unstructured==0.14.0
+unstructured==0.15.0
     # via open-webui
 unstructured-client==0.22.0
     # via unstructured
@@ -648,6 +711,7 @@ uritemplate==4.1.1
     # via google-api-python-client
 urllib3==2.2.1
     # via botocore
+    # via docker
     # via kubernetes
     # via requests
     # via unstructured-client
@@ -657,7 +721,7 @@ uvicorn==0.22.0
     # via open-webui
 uvloop==0.19.0
     # via uvicorn
-validators==0.28.1
+validators==0.33.0
     # via open-webui
 watchfiles==0.21.0
     # via uvicorn
@@ -676,6 +740,8 @@ wsproto==1.2.0
     # via simple-websocket
 xlrd==2.0.1
     # via open-webui
+xlsxwriter==3.2.0
+    # via python-pptx
 yarl==1.9.4
     # via aiohttp
 youtube-transcript-api==0.6.2

requirements.lock

@@ -10,16 +10,25 @@
 #   universal: false
 
 -e file:.
-aiohttp==3.9.5
+aiohappyeyeballs==2.3.5
+    # via aiohttp
+aiohttp==3.10.2
     # via langchain
     # via langchain-community
     # via open-webui
 aiosignal==1.3.1
     # via aiohttp
+alembic==1.13.2
+    # via open-webui
 annotated-types==0.6.0
     # via pydantic
-anyio==4.3.0
+anthropic==0.32.0
+    # via open-webui
+anyio==4.4.0
+    # via anthropic
     # via httpx
+    # via langfuse
+    # via openai
     # via starlette
     # via watchfiles
 apscheduler==3.10.4
@@ -32,6 +41,7 @@ asgiref==3.8.1
     # via opentelemetry-instrumentation-asgi
 attrs==23.2.0
     # via aiohttp
+    # via pytest-docker
 authlib==1.3.1
     # via open-webui
 av==11.0.0
@@ -49,13 +59,13 @@ beautifulsoup4==4.12.3
     # via unstructured
 bidict==0.23.1
     # via python-socketio
-black==24.4.2
+black==24.8.0
     # via open-webui
 blinker==1.8.2
     # via flask
-boto3==1.34.110
+boto3==1.34.153
     # via open-webui
-botocore==1.34.110
+botocore==1.34.155
     # via boto3
     # via s3transfer
 build==1.2.1
@@ -76,9 +86,9 @@ chardet==5.2.0
 charset-normalizer==3.3.2
     # via requests
     # via unstructured-client
-chroma-hnswlib==0.7.3
+chroma-hnswlib==0.7.6
     # via chromadb
-chromadb==0.5.0
+chromadb==0.5.5
     # via langchain-chroma
     # via open-webui
 click==8.1.7
@@ -102,7 +112,6 @@ cryptography==42.0.7
 ctranslate2==4.2.1
     # via faster-whisper
 dataclasses-json==0.6.6
-    # via langchain
     # via langchain-community
     # via unstructured
     # via unstructured-client
@@ -113,11 +122,17 @@ defusedxml==0.7.1
 deprecated==1.2.14
     # via opentelemetry-api
     # via opentelemetry-exporter-otlp-proto-grpc
+distro==1.9.0
+    # via anthropic
+    # via openai
 dnspython==2.6.1
     # via email-validator
+    # via pymongo
+docker==7.1.0
+    # via open-webui
 docx2txt==0.8
     # via open-webui
-duckduckgo-search==6.1.5
+duckduckgo-search==6.2.6
     # via open-webui
 easygui==0.98.3
     # via oletools
@@ -166,7 +181,7 @@ frozenlist==1.4.1
 fsspec==2024.3.1
     # via huggingface-hub
     # via torch
-google-ai-generativelanguage==0.6.4
+google-ai-generativelanguage==0.6.6
     # via google-generativeai
 google-api-core==2.19.0
     # via google-ai-generativelanguage
@@ -183,7 +198,7 @@ google-auth==2.29.0
     # via kubernetes
 google-auth-httplib2==0.2.0
     # via google-api-python-client
-google-generativeai==0.5.4
+google-generativeai==0.7.2
     # via open-webui
 googleapis-common-protos==1.63.0
     # via google-api-core
@@ -208,8 +223,11 @@ httplib2==0.22.0
 httptools==0.6.1
     # via uvicorn
 httpx==0.27.0
+    # via anthropic
+    # via chromadb
     # via fastapi
     # via langfuse
+    # via openai
 huggingface-hub==0.23.0
     # via faster-whisper
     # via sentence-transformers
@@ -229,12 +247,16 @@ importlib-metadata==7.0.0
     # via opentelemetry-api
 importlib-resources==6.4.0
     # via chromadb
+iniconfig==2.0.0
+    # via pytest
 itsdangerous==2.2.0
     # via flask
 jinja2==3.1.4
     # via fastapi
     # via flask
     # via torch
+jiter==0.5.0
+    # via anthropic
 jmespath==1.0.1
     # via boto3
     # via botocore
@@ -249,14 +271,14 @@ jsonpointer==2.4
     # via jsonpatch
 kubernetes==29.0.0
     # via chromadb
-langchain==0.2.0
+langchain==0.2.12
     # via langchain-community
     # via open-webui
-langchain-chroma==0.1.1
+langchain-chroma==0.1.2
     # via open-webui
-langchain-community==0.2.0
+langchain-community==0.2.10
     # via open-webui
-langchain-core==0.2.1
+langchain-core==0.2.28
     # via langchain
     # via langchain-chroma
     # via langchain-community
@@ -265,22 +287,26 @@ langchain-text-splitters==0.2.0
     # via langchain
 langdetect==1.0.9
     # via unstructured
-langfuse==2.33.0
+langfuse==2.43.3
     # via open-webui
-langsmith==0.1.57
+langsmith==0.1.96
     # via langchain
     # via langchain-community
     # via langchain-core
 lark==1.1.8
     # via rtfde
 lxml==5.2.2
+    # via python-pptx
     # via unstructured
+mako==1.3.5
+    # via alembic
 markdown==3.6
     # via open-webui
 markdown-it-py==3.0.0
     # via rich
 markupsafe==2.1.5
     # via jinja2
+    # via mako
     # via werkzeug
 marshmallow==3.21.2
     # via dataclasses-json
@@ -339,11 +365,13 @@ onnxruntime==1.17.3
     # via chromadb
     # via faster-whisper
     # via rapidocr-onnxruntime
+openai==1.38.0
+    # via open-webui
 opencv-python==4.9.0.80
     # via rapidocr-onnxruntime
-opencv-python-headless==4.9.0.80
+opencv-python-headless==4.10.0.84
     # via open-webui
-openpyxl==3.1.2
+openpyxl==3.1.5
     # via open-webui
 opentelemetry-api==1.24.0
     # via chromadb
@@ -380,7 +408,6 @@ ordered-set==4.1.0
     # via deepdiff
 orjson==3.10.3
     # via chromadb
-    # via duckduckgo-search
     # via fastapi
     # via langsmith
 overrides==7.7.0
@@ -393,6 +420,7 @@ packaging==23.2
     # via langfuse
     # via marshmallow
     # via onnxruntime
+    # via pytest
     # via transformers
     # via unstructured-client
 pandas==2.2.2
@@ -403,19 +431,24 @@ pathspec==0.12.1
     # via black
 pcodedmp==1.2.6
     # via oletools
-peewee==3.17.5
+peewee==3.17.6
     # via open-webui
     # via peewee-migrate
 peewee-migrate==1.12.2
     # via open-webui
 pillow==10.3.0
     # via fpdf2
+    # via python-pptx
     # via rapidocr-onnxruntime
     # via sentence-transformers
 platformdirs==4.2.1
     # via black
+pluggy==1.5.0
+    # via pytest
 posthog==3.5.0
     # via chromadb
+primp==0.5.5
+    # via duckduckgo-search
 proto-plus==1.23.0
     # via google-ai-generativelanguage
     # via google-api-core
@@ -428,6 +461,9 @@ protobuf==4.25.3
     # via onnxruntime
     # via opentelemetry-proto
     # via proto-plus
+psutil==6.0.0
+    # via open-webui
+    # via unstructured
 psycopg2-binary==2.9.9
     # via open-webui
 pyasn1==0.6.0
@@ -440,7 +476,8 @@ pyclipper==1.3.0.post5
     # via rapidocr-onnxruntime
 pycparser==2.22
     # via cffi
-pydantic==2.7.1
+pydantic==2.8.2
+    # via anthropic
     # via chromadb
     # via fastapi
     # via google-generativeai
@@ -449,30 +486,36 @@ pydantic==2.7.1
     # via langfuse
     # via langsmith
     # via open-webui
-pydantic-core==2.18.2
+    # via openai
+pydantic-core==2.20.1
     # via pydantic
 pydub==0.25.1
     # via open-webui
 pygments==2.18.0
     # via rich
-pyjwt==2.8.0
+pyjwt==2.9.0
     # via open-webui
-pymysql==1.1.0
+pymongo==4.8.0
+    # via open-webui
+pymysql==1.1.1
     # via open-webui
 pypandoc==1.13
     # via open-webui
 pyparsing==2.4.7
     # via httplib2
     # via oletools
-pypdf==4.2.0
+pypdf==4.3.1
     # via open-webui
     # via unstructured-client
 pypika==0.48.9
     # via chromadb
 pyproject-hooks==1.1.0
     # via build
-pyreqwest-impersonate==0.4.7
-    # via duckduckgo-search
+pytest==8.2.2
+    # via open-webui
+    # via pytest-docker
+pytest-docker==3.1.1
+    # via open-webui
 python-dateutil==2.9.0.post0
     # via botocore
     # via kubernetes
@@ -492,7 +535,9 @@ python-magic==0.4.27
 python-multipart==0.0.9
     # via fastapi
     # via open-webui
-python-socketio==5.11.2
+python-pptx==1.0.0
+    # via open-webui
+python-socketio==5.11.3
     # via open-webui
 pytube==15.0.0
     # via open-webui
@@ -516,15 +561,18 @@ rank-bm25==0.2.2
     # via open-webui
 rapidfuzz==3.9.0
     # via unstructured
-rapidocr-onnxruntime==1.3.22
+rapidocr-onnxruntime==1.3.24
     # via open-webui
 red-black-tree-mod==1.20
     # via extract-msg
+redis==5.0.8
+    # via open-webui
 regex==2024.5.10
     # via nltk
+    # via tiktoken
     # via transformers
-requests==2.32.2
-    # via chromadb
+requests==2.32.3
+    # via docker
     # via google-api-core
     # via huggingface-hub
     # via kubernetes
@@ -534,6 +582,7 @@ requests==2.32.2
     # via open-webui
     # via posthog
     # via requests-oauthlib
+    # via tiktoken
     # via transformers
     # via unstructured
     # via unstructured-client
@@ -556,12 +605,12 @@ scikit-learn==1.4.2
 scipy==1.13.0
     # via scikit-learn
     # via sentence-transformers
-sentence-transformers==2.7.0
+sentence-transformers==3.0.1
     # via open-webui
 setuptools==69.5.1
     # via ctranslate2
     # via opentelemetry-instrumentation
-shapely==2.0.4
+shapely==2.0.5
     # via rapidocr-onnxruntime
 shellingham==1.5.4
     # via typer
@@ -577,13 +626,17 @@ six==1.16.0
     # via rapidocr-onnxruntime
     # via unstructured-client
 sniffio==1.3.1
+    # via anthropic
     # via anyio
     # via httpx
+    # via openai
 soupsieve==2.5
     # via beautifulsoup4
-sqlalchemy==2.0.30
+sqlalchemy==2.0.31
+    # via alembic
     # via langchain
     # via langchain-community
+    # via open-webui
 starlette==0.37.2
     # via fastapi
 sympy==1.12
@@ -598,7 +651,10 @@ tenacity==8.3.0
     # via langchain-core
 threadpoolctl==3.5.0
     # via scikit-learn
+tiktoken==0.7.0
+    # via open-webui
 tokenizers==0.15.2
+    # via anthropic
     # via chromadb
     # via faster-whisper
     # via transformers
@@ -609,21 +665,28 @@ tqdm==4.66.4
     # via google-generativeai
     # via huggingface-hub
     # via nltk
+    # via openai
     # via sentence-transformers
     # via transformers
+    # via unstructured
 transformers==4.39.3
     # via sentence-transformers
 typer==0.12.3
     # via chromadb
     # via fastapi-cli
 typing-extensions==4.11.0
+    # via alembic
+    # via anthropic
     # via chromadb
     # via fastapi
     # via google-generativeai
     # via huggingface-hub
+    # via langchain-core
+    # via openai
     # via opentelemetry-sdk
     # via pydantic
     # via pydantic-core
+    # via python-pptx
     # via sqlalchemy
     # via torch
     # via typer
@@ -640,7 +703,7 @@ tzlocal==5.2
     # via extract-msg
 ujson==5.10.0
     # via fastapi
-unstructured==0.14.0
+unstructured==0.15.0
     # via open-webui
 unstructured-client==0.22.0
     # via unstructured
@@ -648,6 +711,7 @@ uritemplate==4.1.1
     # via google-api-python-client
 urllib3==2.2.1
     # via botocore
+    # via docker
     # via kubernetes
     # via requests
     # via unstructured-client
@@ -657,7 +721,7 @@ uvicorn==0.22.0
     # via open-webui
 uvloop==0.19.0
     # via uvicorn
-validators==0.28.1
+validators==0.33.0
     # via open-webui
 watchfiles==0.21.0
     # via uvicorn
@@ -676,6 +740,8 @@ wsproto==1.2.0
     # via simple-websocket
 xlrd==2.0.1
     # via open-webui
+xlsxwriter==3.2.0
+    # via python-pptx
 yarl==1.9.4
     # via aiohttp
 youtube-transcript-api==0.6.2

src/app.css

@@ -158,3 +158,12 @@ input[type='number'] {
 .password {
 	-webkit-text-security: disc;
 }
+
+.codespan {
+	color: #eb5757;
+	border-width: 0px;
+	padding: 3px 8px;
+	font-size: 0.8em;
+	font-weight: 600;
+	@apply rounded-md dark:bg-gray-800 bg-gray-100 mx-0.5;
+}

src/app.html

@@ -1,4 +1,4 @@
-<!DOCTYPE html>
+<!doctype html>
 <html lang="en">
 	<head>
 		<meta charset="utf-8" />

src/lib/apis/chats/index.ts

@@ -32,10 +32,15 @@ export const createNewChat = async (token: string, chat: object) => {
 	return res;
 };
 
-export const getChatList = async (token: string = '') => {
+export const getChatList = async (token: string = '', page: number | null = null) => {
 	let error = null;
+	const searchParams = new URLSearchParams();
 
-	const res = await fetch(`${WEBUI_API_BASE_URL}/chats/`, {
+	if (page !== null) {
+		searchParams.append('page', `${page}`);
+	}
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/chats/?${searchParams.toString()}`, {
 		method: 'GET',
 		headers: {
 			Accept: 'application/json',

src/lib/apis/index.ts

@@ -69,6 +69,7 @@ type ChatCompletedForm = {
 	model: string;
 	messages: string[];
 	chat_id: string;
+	session_id: string;
 };
 
 export const chatCompleted = async (token: string, body: ChatCompletedForm) => {

src/lib/apis/ollama/index.ts

@@ -1,5 +1,4 @@
 import { OLLAMA_API_BASE_URL } from '$lib/constants';
-import { titleGenerationTemplate } from '$lib/utils';
 
 export const getOllamaConfig = async (token: string = '') => {
 	let error = null;
@@ -203,55 +202,6 @@ export const getOllamaModels = async (token: string = '') => {
 		});
 };
 
-// TODO: migrate to backend
-export const generateTitle = async (
-	token: string = '',
-	template: string,
-	model: string,
-	prompt: string
-) => {
-	let error = null;
-
-	template = titleGenerationTemplate(template, prompt);
-
-	console.log(template);
-
-	const res = await fetch(`${OLLAMA_API_BASE_URL}/api/generate`, {
-		method: 'POST',
-		headers: {
-			Accept: 'application/json',
-			'Content-Type': 'application/json',
-			Authorization: `Bearer ${token}`
-		},
-		body: JSON.stringify({
-			model: model,
-			prompt: template,
-			stream: false,
-			options: {
-				// Restrict the number of tokens generated to 50
-				num_predict: 50
-			}
-		})
-	})
-		.then(async (res) => {
-			if (!res.ok) throw await res.json();
-			return res.json();
-		})
-		.catch((err) => {
-			console.log(err);
-			if ('detail' in err) {
-				error = err.detail;
-			}
-			return null;
-		});
-
-	if (error) {
-		throw error;
-	}
-
-	return res?.response.replace(/["']/g, '') ?? 'New Chat';
-};
-
 export const generatePrompt = async (token: string = '', model: string, conversation: string) => {
 	let error = null;
 

src/lib/apis/openai/index.ts

@@ -1,6 +1,4 @@
 import { OPENAI_API_BASE_URL } from '$lib/constants';
-import { titleGenerationTemplate } from '$lib/utils';
-import { type Model, models, settings } from '$lib/stores';
 
 export const getOpenAIConfig = async (token: string = '') => {
 	let error = null;
@@ -260,7 +258,7 @@ export const getOpenAIModelsDirect = async (
 		throw error;
 	}
 
-	const models = Array.isArray(res) ? res : res?.data ?? null;
+	const models = Array.isArray(res) ? res : (res?.data ?? null);
 
 	return models
 		.map((model) => ({ id: model.id, name: model.name ?? model.id, external: true }))
@@ -330,126 +328,3 @@ export const synthesizeOpenAISpeech = async (
 
 	return res;
 };
-
-export const generateTitle = async (
-	token: string = '',
-	template: string,
-	model: string,
-	prompt: string,
-	chat_id?: string,
-	url: string = OPENAI_API_BASE_URL
-) => {
-	let error = null;
-
-	template = titleGenerationTemplate(template, prompt);
-
-	console.log(template);
-
-	const res = await fetch(`${url}/chat/completions`, {
-		method: 'POST',
-		headers: {
-			Accept: 'application/json',
-			'Content-Type': 'application/json',
-			Authorization: `Bearer ${token}`
-		},
-		body: JSON.stringify({
-			model: model,
-			messages: [
-				{
-					role: 'user',
-					content: template
-				}
-			],
-			stream: false,
-			// Restricting the max tokens to 50 to avoid long titles
-			max_tokens: 50,
-			...(chat_id && { chat_id: chat_id }),
-			title: true
-		})
-	})
-		.then(async (res) => {
-			if (!res.ok) throw await res.json();
-			return res.json();
-		})
-		.catch((err) => {
-			console.log(err);
-			if ('detail' in err) {
-				error = err.detail;
-			}
-			return null;
-		});
-
-	if (error) {
-		throw error;
-	}
-
-	return res?.choices[0]?.message?.content.replace(/["']/g, '') ?? 'New Chat';
-};
-
-export const generateSearchQuery = async (
-	token: string = '',
-	model: string,
-	previousMessages: string[],
-	prompt: string,
-	url: string = OPENAI_API_BASE_URL
-): Promise<string | undefined> => {
-	let error = null;
-
-	// TODO: Allow users to specify the prompt
-	// Get the current date in the format "January 20, 2024"
-	const currentDate = new Intl.DateTimeFormat('en-US', {
-		year: 'numeric',
-		month: 'long',
-		day: '2-digit'
-	}).format(new Date());
-
-	const res = await fetch(`${url}/chat/completions`, {
-		method: 'POST',
-		headers: {
-			Accept: 'application/json',
-			'Content-Type': 'application/json',
-			Authorization: `Bearer ${token}`
-		},
-		body: JSON.stringify({
-			model: model,
-			// Few shot prompting
-			messages: [
-				{
-					role: 'assistant',
-					content: `You are tasked with generating web search queries. Give me an appropriate query to answer my question for google search. Answer with only the query. Today is ${currentDate}.`
-				},
-				{
-					role: 'user',
-					content: prompt
-				}
-				// {
-				// 	role: 'user',
-				// 	content:
-				// 		(previousMessages.length > 0
-				// 			? `Previous Questions:\n${previousMessages.join('\n')}\n\n`
-				// 			: '') + `Current Question: ${prompt}`
-				// }
-			],
-			stream: false,
-			// Restricting the max tokens to 30 to avoid long search queries
-			max_tokens: 30
-		})
-	})
-		.then(async (res) => {
-			if (!res.ok) throw await res.json();
-			return res.json();
-		})
-		.catch((err) => {
-			console.log(err);
-			if ('detail' in err) {
-				error = err.detail;
-			}
-			return undefined;
-		});
-
-	if (error) {
-		throw error;
-	}
-
-	return res?.choices[0]?.message?.content.replace(/["']/g, '') ?? undefined;
-};

src/lib/components/ChangelogModal.svelte

@@ -75,12 +75,12 @@
 										class="font-semibold uppercase text-xs {section === 'added'
 											? 'text-white bg-blue-600'
 											: section === 'fixed'
-											? 'text-white bg-green-600'
-											: section === 'changed'
-											? 'text-white bg-yellow-600'
-											: section === 'removed'
-											? 'text-white bg-red-600'
-											: ''}  w-fit px-3 rounded-full my-2.5"
+												? 'text-white bg-green-600'
+												: section === 'changed'
+													? 'text-white bg-yellow-600'
+													: section === 'removed'
+														? 'text-white bg-red-600'
+														: ''}  w-fit px-3 rounded-full my-2.5"
 									>
 										{section}
 									</div>

src/lib/components/admin/Settings/Documents.svelte

@@ -112,7 +112,7 @@
 							url: OpenAIUrl,
 							batch_size: OpenAIBatchSize
 						}
-				  }
+					}
 				: {})
 		}).catch(async (error) => {
 			toast.error(error);

src/lib/components/chat/Chat.svelte

@@ -25,7 +25,8 @@
 		user,
 		socket,
 		showCallOverlay,
-		tools
+		tools,
+		currentChatPage
 	} from '$lib/stores';
 	import {
 		convertMessagesToHistory,
@@ -421,7 +422,9 @@
 					params: params,
 					files: chatFiles
 				});
-				await chats.set(await getChatList(localStorage.token));
+
+				currentChatPage.set(1);
+				await chats.set(await getChatList(localStorage.token, $currentChatPage));
 			}
 		}
 	};
@@ -467,7 +470,9 @@
 					params: params,
 					files: chatFiles
 				});
-				await chats.set(await getChatList(localStorage.token));
+
+				currentChatPage.set(1);
+				await chats.set(await getChatList(localStorage.token, $currentChatPage));
 			}
 		}
 	};
@@ -574,8 +579,8 @@
 		let selectedModelIds = modelId
 			? [modelId]
 			: atSelectedModel !== undefined
-			? [atSelectedModel.id]
-			: selectedModels;
+				? [atSelectedModel.id]
+				: selectedModels;
 
 		// Create response messages for each selected model
 		const responseMessageIds = {};
@@ -627,7 +632,9 @@
 					tags: [],
 					timestamp: Date.now()
 				});
-				await chats.set(await getChatList(localStorage.token));
+
+				currentChatPage.set(1);
+				await chats.set(await getChatList(localStorage.token, $currentChatPage));
 				await chatId.set(chat.id);
 			} else {
 				await chatId.set('local');
@@ -703,7 +710,9 @@
 			})
 		);
 
-		await chats.set(await getChatList(localStorage.token));
+		currentChatPage.set(1);
+		await chats.set(await getChatList(localStorage.token, $currentChatPage));
+
 		return _responses;
 	};
 
@@ -730,11 +739,11 @@
 								? await getAndUpdateUserLocation(localStorage.token)
 								: undefined
 						)}${
-							responseMessage?.userContext ?? null
+							(responseMessage?.userContext ?? null)
 								? `\n\nUser Context:\n${responseMessage?.userContext ?? ''}`
 								: ''
 						}`
-				  }
+					}
 				: undefined,
 			...messages
 		]
@@ -802,10 +811,10 @@
 			options: {
 				...(params ?? $settings.params ?? {}),
 				stop:
-					params?.stop ?? $settings?.params?.stop ?? undefined
-						? (params?.stop ?? $settings.params.stop).map((str) =>
-								decodeURIComponent(JSON.parse('"' + str.replace(/\"/g, '\\"') + '"'))
-						  )
+					(params?.stop ?? $settings?.params?.stop ?? undefined)
+						? (params?.stop.split(',').map((token) => token.trim()) ?? $settings.params.stop).map(
+								(str) => decodeURIComponent(JSON.parse('"' + str.replace(/\"/g, '\\"') + '"'))
+							)
 						: undefined,
 				num_predict: params?.max_tokens ?? $settings?.params?.max_tokens ?? undefined,
 				repeat_penalty:
@@ -868,6 +877,10 @@
 								} else {
 									responseMessage.content += data.message.content;
 
+									if (navigator.vibrate && ($settings?.hapticFeedback ?? false)) {
+										navigator.vibrate(5);
+									}
+
 									const sentences = extractSentencesForAudio(responseMessage.content);
 									sentences.pop();
 
@@ -949,7 +962,9 @@
 						params: params,
 						files: chatFiles
 					});
-					await chats.set(await getChatList(localStorage.token));
+
+					currentChatPage.set(1);
+					await chats.set(await getChatList(localStorage.token, $currentChatPage));
 				}
 			}
 		} else {
@@ -1045,10 +1060,10 @@
 					stream: true,
 					model: model.id,
 					stream_options:
-						model.info?.meta?.capabilities?.usage ?? false
+						(model.info?.meta?.capabilities?.usage ?? false)
 							? {
 									include_usage: true
-							  }
+								}
 							: undefined,
 					messages: [
 						params?.system || $settings.system || (responseMessage?.userContext ?? null)
@@ -1061,11 +1076,11 @@
 											? await getAndUpdateUserLocation(localStorage.token)
 											: undefined
 									)}${
-										responseMessage?.userContext ?? null
+										(responseMessage?.userContext ?? null)
 											? `\n\nUser Context:\n${responseMessage?.userContext ?? ''}`
 											: ''
 									}`
-							  }
+								}
 							: undefined,
 						...messages
 					]
@@ -1081,7 +1096,7 @@
 												text:
 													arr.length - 1 !== idx
 														? message.content
-														: message?.raContent ?? message.content
+														: (message?.raContent ?? message.content)
 											},
 											...message.files
 												.filter((file) => file.type === 'image')
@@ -1092,20 +1107,20 @@
 													}
 												}))
 										]
-								  }
+									}
 								: {
 										content:
 											arr.length - 1 !== idx
 												? message.content
-												: message?.raContent ?? message.content
-								  })
+												: (message?.raContent ?? message.content)
+									})
 						})),
 					seed: params?.seed ?? $settings?.params?.seed ?? undefined,
 					stop:
-						params?.stop ?? $settings?.params?.stop ?? undefined
-							? (params?.stop ?? $settings.params.stop).map((str) =>
-									decodeURIComponent(JSON.parse('"' + str.replace(/\"/g, '\\"') + '"'))
-							  )
+						(params?.stop ?? $settings?.params?.stop ?? undefined)
+							? (params?.stop.split(',').map((token) => token.trim()) ?? $settings.params.stop).map(
+									(str) => decodeURIComponent(JSON.parse('"' + str.replace(/\"/g, '\\"') + '"'))
+								)
 							: undefined,
 					temperature: params?.temperature ?? $settings?.params?.temperature ?? undefined,
 					top_p: params?.top_p ?? $settings?.params?.top_p ?? undefined,
@@ -1128,7 +1143,6 @@
 
 			if (res && res.ok && res.body) {
 				const textStream = await createOpenAITextStream(res.body, $settings.splitLargeChunks);
-				let lastUsage = null;
 
 				for await (const update of textStream) {
 					const { value, done, citations, error, usage } = update;
@@ -1154,7 +1168,7 @@
 					}
 
 					if (usage) {
-						lastUsage = usage;
+						responseMessage.info = { ...usage, openai: true };
 					}
 
 					if (citations) {
@@ -1167,6 +1181,10 @@
 					} else {
 						responseMessage.content += value;
 
+						if (navigator.vibrate && ($settings?.hapticFeedback ?? false)) {
+							navigator.vibrate(5);
+						}
+
 						const sentences = extractSentencesForAudio(responseMessage.content);
 						sentences.pop();
 
@@ -1208,10 +1226,6 @@
 					document.getElementById(`speak-button-${responseMessage.id}`)?.click();
 				}
 
-				if (lastUsage) {
-					responseMessage.info = { ...lastUsage, openai: true };
-				}
-
 				if ($chatId == _chatId) {
 					if ($settings.saveChatHistory ?? true) {
 						chat = await updateChatById(localStorage.token, _chatId, {
@@ -1221,7 +1235,9 @@
 							params: params,
 							files: chatFiles
 						});
-						await chats.set(await getChatList(localStorage.token));
+
+						currentChatPage.set(1);
+						await chats.set(await getChatList(localStorage.token, $currentChatPage));
 					}
 				}
 			} else {
@@ -1386,7 +1402,9 @@
 
 		if ($settings.saveChatHistory ?? true) {
 			chat = await updateChatById(localStorage.token, _chatId, { title: _title });
-			await chats.set(await getChatList(localStorage.token));
+
+			currentChatPage.set(1);
+			await chats.set(await getChatList(localStorage.token, $currentChatPage));
 		}
 	};
 

src/lib/components/chat/Controls/Controls.svelte

@@ -9,6 +9,8 @@
 	import FileItem from '$lib/components/common/FileItem.svelte';
 	import Collapsible from '$lib/components/common/Collapsible.svelte';
 
+	import { user } from '$lib/stores';
+
 	export let models = [];
 
 	export let chatFiles = [];
@@ -78,7 +80,7 @@
 		<Collapsible title={$i18n.t('Advanced Params')} open={true}>
 			<div class="text-sm mt-1.5" slot="content">
 				<div>
-					<AdvancedParams bind:params />
+					<AdvancedParams admin={$user?.role === 'admin'} bind:params />
 				</div>
 			</div>
 		</Collapsible>

src/lib/components/chat/MessageInput.svelte

@@ -384,7 +384,7 @@
 
 				{#if atSelectedModel !== undefined}
 					<div
-						class="px-3 py-2.5 text-left w-full flex justify-between items-center absolute bottom-0 left-0 right-0 bg-gradient-to-t from-50% from-white dark:from-gray-900"
+						class="px-3 py-2.5 text-left w-full flex justify-between items-center absolute bottom-0 left-0 right-0 bg-gradient-to-t from-50% from-white dark:from-gray-900 z-50"
 					>
 						<div class="flex items-center gap-2 text-sm dark:text-gray-500">
 							<img

src/lib/components/chat/MessageInput/CallOverlay.svelte

@@ -609,10 +609,10 @@
 								style="font-size:{rmsLevel * 100 > 4
 									? '4.5'
 									: rmsLevel * 100 > 2
-									? '4.25'
-									: rmsLevel * 100 > 1
-									? '3.75'
-									: '3.5'}rem;width: 100%; text-align:center;"
+										? '4.25'
+										: rmsLevel * 100 > 1
+											? '3.75'
+											: '3.5'}rem;width: 100%; text-align:center;"
 							>
 								{emoji}
 							</div>
@@ -658,10 +658,10 @@
 								class=" {rmsLevel * 100 > 4
 									? ' size-[4.5rem]'
 									: rmsLevel * 100 > 2
-									? ' size-16'
-									: rmsLevel * 100 > 1
-									? 'size-14'
-									: 'size-12'}  transition-all rounded-full {(model?.info?.meta
+										? ' size-16'
+										: rmsLevel * 100 > 1
+											? 'size-14'
+											: 'size-12'}  transition-all rounded-full {(model?.info?.meta
 									?.profile_image_url ?? '/static/favicon.png') !== '/static/favicon.png'
 									? ' bg-cover bg-center bg-no-repeat'
 									: 'bg-black dark:bg-white'}  bg-black dark:bg-white"
@@ -691,10 +691,10 @@
 									style="font-size:{rmsLevel * 100 > 4
 										? '13'
 										: rmsLevel * 100 > 2
-										? '12'
-										: rmsLevel * 100 > 1
-										? '11.5'
-										: '11'}rem;width:100%;text-align:center;"
+											? '12'
+											: rmsLevel * 100 > 1
+												? '11.5'
+												: '11'}rem;width:100%;text-align:center;"
 								>
 									{emoji}
 								</div>
@@ -740,10 +740,10 @@
 									class=" {rmsLevel * 100 > 4
 										? ' size-52'
 										: rmsLevel * 100 > 2
-										? 'size-48'
-										: rmsLevel * 100 > 1
-										? 'size-[11.5rem]'
-										: 'size-44'}  transition-all rounded-full {(model?.info?.meta
+											? 'size-48'
+											: rmsLevel * 100 > 1
+												? 'size-[11.5rem]'
+												: 'size-44'}  transition-all rounded-full {(model?.info?.meta
 										?.profile_image_url ?? '/static/favicon.png') !== '/static/favicon.png'
 										? ' bg-cover bg-center bg-no-repeat'
 										: 'bg-black dark:bg-white'} "

src/lib/components/chat/MessageInput/Documents.svelte

@@ -27,7 +27,7 @@
 						title: $i18n.t('All Documents'),
 						collection_names: $documents.map((doc) => doc.collection_name)
 					}
-			  ]
+				]
 			: []),
 		...$documents
 			.reduce((a, e, i, arr) => {

src/lib/components/chat/MessageInput/Models.svelte

@@ -147,8 +147,8 @@
 					<div class="m-1 overflow-y-auto p-1 rounded-r-lg space-y-0.5 scrollbar-hidden">
 						{#each filteredModels as model, modelIdx}
 							<button
-								class=" px-3 py-1.5 rounded-xl w-full text-left {modelIdx === selectedIdx
-									? '  bg-gray-50 dark:bg-gray-850  selected-command-option-button'
+								class="px-3 py-1.5 rounded-xl w-full text-left {modelIdx === selectedIdx
+									? 'bg-gray-50 dark:bg-gray-850 selected-command-option-button'
 									: ''}"
 								type="button"
 								on:click={() => {
@@ -159,13 +159,18 @@
 								}}
 								on:focus={() => {}}
 							>
-								<div class=" font-medium text-black dark:text-gray-100 line-clamp-1">
+								<div class="flex font-medium text-black dark:text-gray-100 line-clamp-1">
+									<img
+										src={model?.info?.meta?.profile_image_url ?? '/static/favicon.png'}
+										alt={model?.name ?? model.id}
+										class="rounded-full size-6 items-center mr-2"
+									/>
 									{model.name}
 								</div>
 
 								<!-- <div class=" text-xs text-gray-600 line-clamp-1">
 								{doc.title}
-							</div> -->
+								</div> -->
 							</button>
 						{/each}
 					</div>

src/lib/components/chat/Messages.svelte

@@ -1,6 +1,6 @@
 <script lang="ts">
 	import { v4 as uuidv4 } from 'uuid';
-	import { chats, config, settings, user as _user, mobile } from '$lib/stores';
+	import { chats, config, settings, user as _user, mobile, currentChatPage } from '$lib/stores';
 	import { tick, getContext, onMount } from 'svelte';
 
 	import { toast } from 'svelte-sonner';
@@ -90,7 +90,8 @@
 			history: history
 		});
 
-		await chats.set(await getChatList(localStorage.token));
+		currentChatPage.set(1);
+		await chats.set(await getChatList(localStorage.token, $currentChatPage));
 	};
 
 	const confirmEditResponseMessage = async (messageId, content) => {
@@ -146,12 +147,14 @@
 
 		await tick();
 
-		const element = document.getElementById('messages-container');
-		autoScroll = element.scrollHeight - element.scrollTop <= element.clientHeight + 50;
+		if ($settings?.scrollOnBranchChange ?? true) {
+			const element = document.getElementById('messages-container');
+			autoScroll = element.scrollHeight - element.scrollTop <= element.clientHeight + 50;
 
-		setTimeout(() => {
-			scrollToBottom();
-		}, 100);
+			setTimeout(() => {
+				scrollToBottom();
+			}, 100);
+		}
 	};
 
 	const showNextMessage = async (message) => {
@@ -195,12 +198,14 @@
 
 		await tick();
 
-		const element = document.getElementById('messages-container');
-		autoScroll = element.scrollHeight - element.scrollTop <= element.clientHeight + 50;
+		if ($settings?.scrollOnBranchChange ?? true) {
+			const element = document.getElementById('messages-container');
+			autoScroll = element.scrollHeight - element.scrollTop <= element.clientHeight + 50;
 
-		setTimeout(() => {
-			scrollToBottom();
-		}, 100);
+			setTimeout(() => {
+				scrollToBottom();
+			}, 100);
+		}
 	};
 
 	const deleteMessageHandler = async (messageId) => {
@@ -300,7 +305,7 @@
 				{#each messages as message, messageIdx}
 					<div class=" w-full {messageIdx === messages.length - 1 ? ' pb-12' : ''}">
 						<div
-							class="flex flex-col justify-between px-5 mb-3 {$settings?.widescreenMode ?? null
+							class="flex flex-col justify-between px-5 mb-3 {($settings?.widescreenMode ?? null)
 								? 'max-w-full'
 								: 'max-w-5xl'} mx-auto rounded-lg group"
 						>
@@ -312,10 +317,10 @@
 									{message}
 									isFirstMessage={messageIdx === 0}
 									siblings={message.parentId !== null
-										? history.messages[message.parentId]?.childrenIds ?? []
-										: Object.values(history.messages)
+										? (history.messages[message.parentId]?.childrenIds ?? [])
+										: (Object.values(history.messages)
 												.filter((message) => message.parentId === null)
-												.map((message) => message.id) ?? []}
+												.map((message) => message.id) ?? [])}
 									{confirmEditMessage}
 									{showPreviousMessage}
 									{showNextMessage}

src/lib/components/chat/Messages/CitationsModal.svelte

@@ -60,8 +60,8 @@
 									href={document?.metadata?.file_id
 										? `/api/v1/files/${document?.metadata?.file_id}/content`
 										: document.source.name.includes('http')
-										? document.source.name
-										: `#`}
+											? document.source.name
+											: `#`}
 									target="_blank"
 								>
 									{document?.metadata?.name ?? document.source.name}

src/lib/components/chat/Messages/CodeBlock.svelte

@@ -1,17 +1,25 @@
 <script lang="ts">
-	import Spinner from '$lib/components/common/Spinner.svelte';
-	import { copyToClipboard } from '$lib/utils';
 	import hljs from 'highlight.js';
-	import 'highlight.js/styles/github-dark.min.css';
 	import { loadPyodide } from 'pyodide';
-	import { onMount, tick } from 'svelte';
+	import mermaid from 'mermaid';
+
+	import { getContext, getAllContexts, onMount } from 'svelte';
+	import { copyToClipboard } from '$lib/utils';
+
+	import 'highlight.js/styles/github-dark.min.css';
+
 	import PyodideWorker from '$lib/workers/pyodide.worker?worker';
 
+	const i18n = getContext('i18n');
+
 	export let id = '';
 
+	export let token;
 	export let lang = '';
 	export let code = '';
 
+	let mermaidHtml = null;
+
 	let highlightedCode = null;
 	let executing = false;
 
@@ -204,70 +212,102 @@ __builtins__.input = input`);
 	};
 
 	let debounceTimeout;
+
 	$: if (code) {
-		// Function to perform the code highlighting
-		const highlightCode = () => {
-			highlightedCode = hljs.highlightAuto(code, hljs.getLanguage(lang)?.aliases).value || code;
-		};
+		if (lang === 'mermaid' && (token?.raw ?? '').endsWith('```')) {
+			(async () => {
+				try {
+					const { svg } = await mermaid.render(`mermaid-${id}`, code);
+					mermaidHtml = svg;
+				} catch (error) {
+					console.error('Error:', error);
+				}
+			})();
+		} else {
+			// Function to perform the code highlighting
+			const highlightCode = () => {
+				highlightedCode = hljs.highlightAuto(code, hljs.getLanguage(lang)?.aliases).value || code;
+			};
 
-		// Clear the previous timeout if it exists
-		clearTimeout(debounceTimeout);
-
-		// Set a new timeout to debounce the code highlighting
-		debounceTimeout = setTimeout(highlightCode, 10);
+			// Clear the previous timeout if it exists
+			clearTimeout(debounceTimeout);
+			// Set a new timeout to debounce the code highlighting
+			debounceTimeout = setTimeout(highlightCode, 10);
+		}
 	}
+
+	onMount(async () => {
+		await mermaid.initialize({ startOnLoad: true });
+
+		if (lang === 'mermaid' && (token?.raw ?? '').endsWith('```')) {
+			try {
+				const { svg } = await mermaid.render(`mermaid-${id}`, code);
+				mermaidHtml = svg;
+			} catch (error) {
+				console.error('Error:', error);
+			}
+		}
+	});
 </script>
 
-<div class="mb-4" dir="ltr">
-	<div
-		class="flex justify-between bg-[#202123] text-white text-xs px-4 pt-1 pb-0.5 rounded-t-lg overflow-x-auto"
-	>
-		<div class="p-1">{@html lang}</div>
+<div class="my-2" dir="ltr">
+	{#if lang === 'mermaid'}
+		{#if mermaidHtml}
+			{@html mermaidHtml}
+		{:else}
+			<pre class=" mermaid-{id}">{code}</pre>
+		{/if}
+	{:else}
+		<div
+			class="flex justify-between bg-[#202123] text-white text-xs px-4 pt-1 pb-0.5 rounded-t-lg overflow-x-auto"
+		>
+			<div class="p-1">{@html lang}</div>
 
-		<div class="flex items-center">
-			{#if lang.toLowerCase() === 'python' || lang.toLowerCase() === 'py' || (lang === '' && checkPythonCode(code))}
-				{#if executing}
-					<div class="copy-code-button bg-none border-none p-1 cursor-not-allowed">Running</div>
-				{:else}
-					<button
-						class="copy-code-button bg-none border-none p-1"
-						on:click={() => {
-							executePython(code);
-						}}>Run</button
-					>
+			<div class="flex items-center">
+				{#if lang.toLowerCase() === 'python' || lang.toLowerCase() === 'py' || (lang === '' && checkPythonCode(code))}
+					{#if executing}
+						<div class="copy-code-button bg-none border-none p-1 cursor-not-allowed">Running</div>
+					{:else}
+						<button
+							class="copy-code-button bg-none border-none p-1"
+							on:click={() => {
+								executePython(code);
+							}}>{$i18n.t('Run')}</button
+						>
+					{/if}
 				{/if}
-			{/if}
-			<button class="copy-code-button bg-none border-none p-1" on:click={copyCode}
-				>{copied ? 'Copied' : 'Copy Code'}</button
-			>
+				<button class="copy-code-button bg-none border-none p-1" on:click={copyCode}
+					>{copied ? $i18n.t('Copied') : $i18n.t('Copy Code')}</button
+				>
+			</div>
 		</div>
-	</div>
 
-	<pre
-		class=" hljs p-4 px-5 overflow-x-auto"
-		style="border-top-left-radius: 0px; border-top-right-radius: 0px; {(executing ||
-			stdout ||
-			stderr ||
-			result) &&
-			'border-bottom-left-radius: 0px; border-bottom-right-radius: 0px;'}"><code
-			class="language-{lang} rounded-t-none whitespace-pre"
-			>{#if highlightedCode}{@html highlightedCode}{:else}{code}{/if}</code
-		></pre>
+		<pre
+			class=" hljs p-4 px-5 overflow-x-auto"
+			style="border-top-left-radius: 0px; border-top-right-radius: 0px; {(executing ||
+				stdout ||
+				stderr ||
+				result) &&
+				'border-bottom-left-radius: 0px; border-bottom-right-radius: 0px;'}"><code
+				class="language-{lang} rounded-t-none whitespace-pre"
+				>{#if highlightedCode}{@html highlightedCode}{:else}{code}{/if}</code
+			></pre>
 
-	<div
-		id="plt-canvas-{id}"
-		class="bg-[#202123] text-white max-w-full overflow-x-auto scrollbar-hidden"
-	/>
+		<div
+			id="plt-canvas-{id}"
+			class="bg-[#202123] text-white max-w-full overflow-x-auto scrollbar-hidden"
+		/>
 
-	{#if executing}
-		<div class="bg-[#202123] text-white px-4 py-4 rounded-b-lg">
-			<div class=" text-gray-500 text-xs mb-1">STDOUT/STDERR</div>
-			<div class="text-sm">Running...</div>
-		</div>
-	{:else if stdout || stderr || result}
-		<div class="bg-[#202123] text-white px-4 py-4 rounded-b-lg">
-			<div class=" text-gray-500 text-xs mb-1">STDOUT/STDERR</div>
-			<div class="text-sm">{stdout || stderr || result}</div>
-		</div>
+		{#if executing}
+			<div class="bg-[#202123] text-white px-4 py-4 rounded-b-lg">
+				<div class=" text-gray-500 text-xs mb-1">STDOUT/STDERR</div>
+				<div class="text-sm">Running...</div>
+			</div>
+		{:else if stdout || stderr || result}
+			<div class="bg-[#202123] text-white px-4 py-4 rounded-b-lg">
+				<div class=" text-gray-500 text-xs mb-1">STDOUT/STDERR</div>
+				<div class="text-sm">{stdout || stderr || result}</div>
+			</div>
+		{/if}
 	{/if}
 </div>

src/lib/components/chat/Messages/CompareMessages.svelte

@@ -118,47 +118,47 @@
 								currentMessageId = message.id;
 								let messageId = message.id;
 								console.log(messageId);
-
 								//
 								let messageChildrenIds = history.messages[messageId].childrenIds;
 								while (messageChildrenIds.length !== 0) {
 									messageId = messageChildrenIds.at(-1);
 									messageChildrenIds = history.messages[messageId].childrenIds;
 								}
-
 								history.currentId = messageId;
 								dispatch('change');
 							}
 						}}
 					>
-						<ResponseMessage
-							message={groupedMessages[model].messages[groupedMessagesIdx[model]]}
-							siblings={groupedMessages[model].messages.map((m) => m.id)}
-							isLastMessage={true}
-							{updateChatMessages}
-							{confirmEditResponseMessage}
-							showPreviousMessage={() => showPreviousMessage(model)}
-							showNextMessage={() => showNextMessage(model)}
-							{readOnly}
-							{rateMessage}
-							{copyToClipboard}
-							{continueGeneration}
-							regenerateResponse={async (message) => {
-								regenerateResponse(message);
-								await tick();
-								groupedMessagesIdx[model] = groupedMessages[model].messages.length - 1;
-							}}
-							on:save={async (e) => {
-								console.log('save', e);
+						{#key history.currentId}
+							<ResponseMessage
+								message={groupedMessages[model].messages[groupedMessagesIdx[model]]}
+								siblings={groupedMessages[model].messages.map((m) => m.id)}
+								isLastMessage={true}
+								{updateChatMessages}
+								{confirmEditResponseMessage}
+								showPreviousMessage={() => showPreviousMessage(model)}
+								showNextMessage={() => showNextMessage(model)}
+								{readOnly}
+								{rateMessage}
+								{copyToClipboard}
+								{continueGeneration}
+								regenerateResponse={async (message) => {
+									regenerateResponse(message);
+									await tick();
+									groupedMessagesIdx[model] = groupedMessages[model].messages.length - 1;
+								}}
+								on:save={async (e) => {
+									console.log('save', e);
 
-								const message = e.detail;
-								history.messages[message.id] = message;
-								await updateChatById(localStorage.token, chatId, {
-									messages: messages,
-									history: history
-								});
-							}}
-						/>
+									const message = e.detail;
+									history.messages[message.id] = message;
+									await updateChatById(localStorage.token, chatId, {
+										messages: messages,
+										history: history
+									});
+								}}
+							/>
+						{/key}
 					</div>
 				{/if}
 			{/each}

src/lib/components/chat/Messages/HTMLRenderer.svelte

@@ -0,0 +1,30 @@
+<script lang="ts">
+	import Image from '$lib/components/common/Image.svelte';
+	import CodeBlock from './CodeBlock.svelte';
+
+	/* The html content of the tag */
+	export let html; //: string;
+	let parsedHTML = [html];
+
+	export let images;
+	export let codes;
+
+	//  all images are in {{IMAGE_0}}, {{IMAGE_1}}....  format
+	// all codes are in {{CODE_0}}, {{CODE_1}}....  format
+
+	const rules = [];
+	rules.forEach((rule) => {
+		parsedHTML = parsedHTML.map((substr) => substr.split(rule.regex)).flat();
+	});
+</script>
+
+{#each parsedHTML as part}
+	{@const match = rules.find((rule) => rule.regex.test(part))}
+	{#if match}
+		<svelte:component this={match.component} {...match.props}>
+			{@html part}
+		</svelte:component>
+	{:else}
+		{@html part}
+	{/if}
+{/each}

src/lib/components/chat/Messages/KatexRenderer.svelte

@@ -0,0 +1,9 @@
+<script lang="ts">
+	import katex from 'katex';
+	import 'katex/contrib/mhchem';
+
+	export let content: string;
+	export let displayMode: boolean = false;
+</script>
+
+{@html katex.renderToString(content, { displayMode, throwOnError: false })}

src/lib/components/chat/Messages/MarkdownInlineTokens.svelte

@@ -0,0 +1,45 @@
+<script lang="ts">
+	import type { Token } from 'marked';
+	import { revertSanitizedResponseContent, unescapeHtml } from '$lib/utils';
+	import { onMount } from 'svelte';
+	import Image from '$lib/components/common/Image.svelte';
+
+	import KatexRenderer from './KatexRenderer.svelte';
+
+	export let id: string;
+	export let tokens: Token[];
+</script>
+
+{#each tokens as token}
+	{#if token.type === 'escape'}
+		{unescapeHtml(token.text)}
+	{:else if token.type === 'html'}
+		{@html token.text}
+	{:else if token.type === 'link'}
+		<a href={token.href} target="_blank" rel="nofollow" title={token.title}>{token.text}</a>
+	{:else if token.type === 'image'}
+		<Image src={token.href} alt={token.text} />
+	{:else if token.type === 'strong'}
+		<strong>
+			<svelte:self id={`${id}-strong`} tokens={token.tokens} />
+		</strong>
+	{:else if token.type === 'em'}
+		<em>
+			<svelte:self id={`${id}-em`} tokens={token.tokens} />
+		</em>
+	{:else if token.type === 'codespan'}
+		<code class="codespan">{revertSanitizedResponseContent(token.raw)}</code>
+	{:else if token.type === 'br'}
+		<br />
+	{:else if token.type === 'del'}
+		<del>
+			<svelte:self id={`${id}-del`} tokens={token.tokens} />
+		</del>
+	{:else if token.type === 'inlineKatex'}
+		{#if token.text}
+			<KatexRenderer content={revertSanitizedResponseContent(token.text)} displayMode={false} />
+		{/if}
+	{:else if token.type === 'text'}
+		{token.raw}
+	{/if}
+{/each}

src/lib/components/chat/Messages/MarkdownTokens.svelte

@@ -0,0 +1,132 @@
+<script lang="ts">
+	import { onMount } from 'svelte';
+	import type { Token } from 'marked';
+	import { revertSanitizedResponseContent, unescapeHtml } from '$lib/utils';
+
+	import CodeBlock from '$lib/components/chat/Messages/CodeBlock.svelte';
+	import MarkdownInlineTokens from '$lib/components/chat/Messages/MarkdownInlineTokens.svelte';
+	import KatexRenderer from './KatexRenderer.svelte';
+
+	export let id: string;
+	export let tokens: Token[];
+	export let top = true;
+
+	const headerComponent = (depth: number) => {
+		return 'h' + depth;
+	};
+</script>
+
+<!-- {JSON.stringify(tokens)} -->
+{#each tokens as token, tokenIdx}
+	{#if token.type === 'hr'}
+		<hr />
+	{:else if token.type === 'heading'}
+		<svelte:element this={headerComponent(token.depth)}>
+			<MarkdownInlineTokens id={`${id}-${tokenIdx}-h`} tokens={token.tokens} />
+		</svelte:element>
+	{:else if token.type === 'code'}
+		<CodeBlock
+			id={`${id}-${tokenIdx}`}
+			{token}
+			lang={token?.lang ?? ''}
+			code={revertSanitizedResponseContent(token?.text ?? '')}
+		/>
+	{:else if token.type === 'table'}
+		<table>
+			<thead>
+				<tr>
+					{#each token.header as header, headerIdx}
+						<th style={token.align[headerIdx] ? '' : `text-align: ${token.align[headerIdx]}`}>
+							<MarkdownInlineTokens
+								id={`${id}-${tokenIdx}-header-${headerIdx}`}
+								tokens={header.tokens}
+							/>
+						</th>
+					{/each}
+				</tr>
+			</thead>
+			<tbody>
+				{#each token.rows as row, rowIdx}
+					<tr>
+						{#each row ?? [] as cell, cellIdx}
+							<td style={token.align[cellIdx] ? '' : `text-align: ${token.align[cellIdx]}`}>
+								<MarkdownInlineTokens
+									id={`${id}-${tokenIdx}-row-${rowIdx}-${cellIdx}`}
+									tokens={cell.tokens}
+								/>
+							</td>
+						{/each}
+					</tr>
+				{/each}
+			</tbody>
+		</table>
+	{:else if token.type === 'blockquote'}
+		<blockquote>
+			<svelte:self id={`${id}-${tokenIdx}`} tokens={token.tokens} />
+		</blockquote>
+	{:else if token.type === 'list'}
+		{#if token.ordered}
+			<ol start={token.start || 1}>
+				{#each token.items as item, itemIdx}
+					<li>
+						<svelte:self
+							id={`${id}-${tokenIdx}-${itemIdx}`}
+							tokens={item.tokens}
+							top={token.loose}
+						/>
+					</li>
+				{/each}
+			</ol>
+		{:else}
+			<ul>
+				{#each token.items as item, itemIdx}
+					<li>
+						<svelte:self
+							id={`${id}-${tokenIdx}-${itemIdx}`}
+							tokens={item.tokens}
+							top={token.loose}
+						/>
+					</li>
+				{/each}
+			</ul>
+		{/if}
+	{:else if token.type === 'html'}
+		{@html token.text}
+	{:else if token.type === 'paragraph'}
+		<p>
+			<MarkdownInlineTokens id={`${id}-${tokenIdx}-p`} tokens={token.tokens ?? []} />
+		</p>
+	{:else if token.type === 'text'}
+		{#if top}
+			<p>
+				{#if token.tokens}
+					<MarkdownInlineTokens id={`${id}-${tokenIdx}-t`} tokens={token.tokens} />
+				{:else}
+					{unescapeHtml(token.text)}
+				{/if}
+			</p>
+		{:else if token.tokens}
+			<MarkdownInlineTokens id={`${id}-${tokenIdx}-p`} tokens={token.tokens ?? []} />
+		{:else}
+			{unescapeHtml(token.text)}
+		{/if}
+	{:else if token.type === 'inlineKatex'}
+		{#if token.text}
+			<KatexRenderer
+				content={revertSanitizedResponseContent(token.text)}
+				displayMode={token?.displayMode ?? false}
+			/>
+		{/if}
+	{:else if token.type === 'blockKatex'}
+		{#if token.text}
+			<KatexRenderer
+				content={revertSanitizedResponseContent(token.text)}
+				displayMode={token?.displayMode ?? false}
+			/>
+		{/if}
+	{:else if token.type === 'space'}
+		{''}
+	{:else}
+		{console.log('Unknown token', token)}
+	{/if}
+{/each}

src/lib/components/chat/Messages/UserMessage.svelte

@@ -62,8 +62,8 @@
 	{#if !($settings?.chatBubble ?? true)}
 		<ProfileImage
 			src={message.user
-				? $models.find((m) => m.id === message.user)?.info?.meta?.profile_image_url ?? '/user.png'
-				: user?.profile_image_url ?? '/user.png'}
+				? ($models.find((m) => m.id === message.user)?.info?.meta?.profile_image_url ?? '/user.png')
+				: (user?.profile_image_url ?? '/user.png')}
 		/>
 	{/if}
 	<div class="w-full overflow-hidden pl-1">
@@ -96,7 +96,7 @@
 			{#if message.files}
 				<div class="mt-2.5 mb-1 w-full flex flex-col justify-end overflow-x-auto gap-1 flex-wrap">
 					{#each message.files as file}
-						<div class={$settings?.chatBubble ?? true ? 'self-end' : ''}>
+						<div class={($settings?.chatBubble ?? true) ? 'self-end' : ''}>
 							{#if file.type === 'image'}
 								<img src={file.url} alt="input" class=" max-h-96 rounded-lg" draggable="false" />
 							{:else}
@@ -162,12 +162,12 @@
 				</div>
 			{:else}
 				<div class="w-full">
-					<div class="flex {$settings?.chatBubble ?? true ? 'justify-end' : ''} mb-2">
+					<div class="flex {($settings?.chatBubble ?? true) ? 'justify-end' : ''} mb-2">
 						<div
-							class="rounded-3xl {$settings?.chatBubble ?? true
+							class="rounded-3xl {($settings?.chatBubble ?? true)
 								? `max-w-[90%] px-5 py-2  bg-gray-50 dark:bg-gray-850 ${
 										message.files ? 'rounded-tr-lg' : ''
-								  }`
+									}`
 								: ''}  "
 						>
 							<pre id="user-message">{message.content}</pre>
@@ -175,7 +175,7 @@
 					</div>
 
 					<div
-						class=" flex {$settings?.chatBubble ?? true
+						class=" flex {($settings?.chatBubble ?? true)
 							? 'justify-end'
 							: ''}  text-gray-600 dark:text-gray-500"
 					>

src/lib/components/chat/ModelSelector/Selector.svelte

@@ -66,7 +66,7 @@
 	$: filteredItems = searchValue
 		? fuse.search(searchValue).map((e) => {
 				return e.item;
-		  })
+			})
 		: items.filter((item) => !item.model?.info?.meta?.hidden);
 
 	const pullModelHandler = async () => {

src/lib/components/chat/Settings/About.svelte

@@ -65,8 +65,8 @@
 							{updateAvailable === null
 								? $i18n.t('Checking for updates...')
 								: updateAvailable
-								? `(v${version.latest} ${$i18n.t('available!')})`
-								: $i18n.t('(latest)')}
+									? `(v${version.latest} ${$i18n.t('available!')})`
+									: $i18n.t('(latest)')}
 						</a>
 					</div>
 

src/lib/components/chat/Settings/Advanced/AdvancedParams.svelte

@@ -29,6 +29,7 @@
 		use_mmap: null,
 		use_mlock: null,
 		num_thread: null,
+		num_gpu: null,
 		template: null
 	};
 
@@ -864,6 +865,52 @@
 			{/if}
 		</div>
 
+		<div class=" py-0.5 w-full justify-between">
+			<div class="flex w-full justify-between">
+				<div class=" self-center text-xs font-medium">{$i18n.t('num_gpu (Ollama)')}</div>
+
+				<button
+					class="p-1 px-3 text-xs flex rounded transition flex-shrink-0 outline-none"
+					type="button"
+					on:click={() => {
+						params.num_gpu = (params?.num_gpu ?? null) === null ? 0 : null;
+					}}
+				>
+					{#if (params?.num_gpu ?? null) === null}
+						<span class="ml-2 self-center">{$i18n.t('Default')}</span>
+					{:else}
+						<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
+					{/if}
+				</button>
+			</div>
+
+			{#if (params?.num_gpu ?? null) !== null}
+				<div class="flex mt-0.5 space-x-2">
+					<div class=" flex-1">
+						<input
+							id="steps-range"
+							type="range"
+							min="0"
+							max="256"
+							step="1"
+							bind:value={params.num_gpu}
+							class="w-full h-2 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
+						/>
+					</div>
+					<div class="">
+						<input
+							bind:value={params.num_gpu}
+							type="number"
+							class=" bg-transparent text-center w-14"
+							min="0"
+							max="256"
+							step="1"
+						/>
+					</div>
+				</div>
+			{/if}
+		</div>
+
 		<!-- <div class=" py-0.5 w-full justify-between">
 			<div class="flex w-full justify-between">
 				<div class=" self-center text-xs font-medium">{$i18n.t('Template')}</div>

src/lib/components/chat/Settings/Chats.svelte

@@ -2,7 +2,7 @@
 	import fileSaver from 'file-saver';
 	const { saveAs } = fileSaver;
 
-	import { chats, user, settings } from '$lib/stores';
+	import { chats, user, settings, scrollPaginationEnabled, currentChatPage } from '$lib/stores';
 
 	import {
 		archiveAllChats,
@@ -62,7 +62,9 @@
 			}
 		}
 
-		await chats.set(await getChatList(localStorage.token));
+		currentChatPage.set(1);
+		await chats.set(await getChatList(localStorage.token, $currentChatPage));
+		scrollPaginationEnabled.set(true);
 	};
 
 	const exportChats = async () => {
@@ -77,7 +79,10 @@
 		await archiveAllChats(localStorage.token).catch((error) => {
 			toast.error(error);
 		});
-		await chats.set(await getChatList(localStorage.token));
+
+		currentChatPage.set(1);
+		await chats.set(await getChatList(localStorage.token, $currentChatPage));
+		scrollPaginationEnabled.set(true);
 	};
 
 	const deleteAllChatsHandler = async () => {
@@ -85,7 +90,10 @@
 		await deleteAllChats(localStorage.token).catch((error) => {
 			toast.error(error);
 		});
-		await chats.set(await getChatList(localStorage.token));
+
+		currentChatPage.set(1);
+		await chats.set(await getChatList(localStorage.token, $currentChatPage));
+		scrollPaginationEnabled.set(true);
 	};
 
 	const toggleSaveChatHistory = async () => {

src/lib/components/chat/Settings/Interface.svelte

@@ -22,6 +22,7 @@
 	let responseAutoCopy = false;
 	let widescreenMode = false;
 	let splitLargeChunks = false;
+	let scrollOnBranchChange = true;
 	let userLocation = false;
 
 	// Interface
@@ -33,12 +34,18 @@
 
 	let showEmojiInCall = false;
 	let voiceInterruption = false;
+	let hapticFeedback = false;
 
 	const toggleSplitLargeChunks = async () => {
 		splitLargeChunks = !splitLargeChunks;
 		saveSettings({ splitLargeChunks: splitLargeChunks });
 	};
 
+	const togglesScrollOnBranchChange = async () => {
+		scrollOnBranchChange = !scrollOnBranchChange;
+		saveSettings({ scrollOnBranchChange: scrollOnBranchChange });
+	};
+
 	const togglewidescreenMode = async () => {
 		widescreenMode = !widescreenMode;
 		saveSettings({ widescreenMode: widescreenMode });
@@ -64,6 +71,11 @@
 		saveSettings({ voiceInterruption: voiceInterruption });
 	};
 
+	const toggleHapticFeedback = async () => {
+		hapticFeedback = !hapticFeedback;
+		saveSettings({ hapticFeedback: hapticFeedback });
+	};
+
 	const toggleUserLocation = async () => {
 		userLocation = !userLocation;
 
@@ -141,9 +153,12 @@
 		chatBubble = $settings.chatBubble ?? true;
 		widescreenMode = $settings.widescreenMode ?? false;
 		splitLargeChunks = $settings.splitLargeChunks ?? false;
+		scrollOnBranchChange = $settings.scrollOnBranchChange ?? true;
 		chatDirection = $settings.chatDirection ?? 'LTR';
 		userLocation = $settings.userLocation ?? false;
 
+		hapticFeedback = $settings.hapticFeedback ?? false;
+
 		defaultModelId = $settings?.models?.at(0) ?? '';
 		if ($config?.default_models) {
 			defaultModelId = $config.default_models.split(',')[0];
@@ -318,6 +333,28 @@
 				</div>
 			</div>
 
+			<div>
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs">
+						{$i18n.t('Scroll to bottom when switching between branches')}
+					</div>
+
+					<button
+						class="p-1 px-3 text-xs flex rounded transition"
+						on:click={() => {
+							togglesScrollOnBranchChange();
+						}}
+						type="button"
+					>
+						{#if scrollOnBranchChange === true}
+							<span class="ml-2 self-center">{$i18n.t('On')}</span>
+						{:else}
+							<span class="ml-2 self-center">{$i18n.t('Off')}</span>
+						{/if}
+					</button>
+				</div>
+			</div>
+
 			<div>
 				<div class=" py-0.5 flex w-full justify-between">
 					<div class=" self-center text-xs">
@@ -409,6 +446,26 @@
 				</div>
 			</div>
 
+			<div>
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs">{$i18n.t('Haptic Feedback')}</div>
+
+					<button
+						class="p-1 px-3 text-xs flex rounded transition"
+						on:click={() => {
+							toggleHapticFeedback();
+						}}
+						type="button"
+					>
+						{#if hapticFeedback === true}
+							<span class="ml-2 self-center">{$i18n.t('On')}</span>
+						{:else}
+							<span class="ml-2 self-center">{$i18n.t('Off')}</span>
+						{/if}
+					</button>
+				</div>
+			</div>
+
 			<div class=" my-1.5 text-sm font-medium">{$i18n.t('Voice')}</div>
 
 			<div>

src/lib/components/chat/Tags.svelte

@@ -8,7 +8,13 @@
 		getTagsById,
 		updateChatById
 	} from '$lib/apis/chats';
-	import { tags as _tags, chats, pinnedChats } from '$lib/stores';
+	import {
+		tags as _tags,
+		chats,
+		pinnedChats,
+		currentChatPage,
+		scrollPaginationEnabled
+	} from '$lib/stores';
 	import { createEventDispatcher, onMount } from 'svelte';
 
 	const dispatch = createEventDispatcher();
@@ -46,11 +52,7 @@
 			tags: tags
 		});
 
-		console.log($_tags);
 		await _tags.set(await getAllChatTags(localStorage.token));
-
-		console.log($_tags);
-
 		if ($_tags.map((t) => t.name).includes(tagName)) {
 			if (tagName === 'pinned') {
 				await pinnedChats.set(await getChatListByTagName(localStorage.token, 'pinned'));
@@ -62,8 +64,11 @@
 				dispatch('close');
 			}
 		} else {
-			await chats.set(await getChatList(localStorage.token));
+			// if the tag we deleted is no longer a valid tag, return to main chat list view
+			currentChatPage.set(1);
+			await chats.set(await getChatList(localStorage.token, $currentChatPage));
 			await pinnedChats.set(await getChatListByTagName(localStorage.token, 'pinned'));
+			await scrollPaginationEnabled.set(true);
 		}
 	};
 

src/lib/components/common/Image.svelte

@@ -5,19 +5,21 @@
 	export let src = '';
 	export let alt = '';
 
-	let _src = '';
+	export let className = '';
 
+	let _src = '';
 	$: _src = src.startsWith('/') ? `${WEBUI_BASE_URL}${src}` : src;
 
 	let showImagePreview = false;
 </script>
 
-<ImagePreview bind:show={showImagePreview} src={_src} {alt} />
 <button
+	class={className}
 	on:click={() => {
-		console.log('image preview');
 		showImagePreview = true;
 	}}
 >
-	<img src={_src} {alt} class=" max-h-96 rounded-lg" draggable="false" data-cy="image" />
+	<img src={_src} {alt} class=" rounded-lg cursor-pointer" draggable="false" data-cy="image" />
 </button>
+
+<ImagePreview bind:show={showImagePreview} src={_src} {alt} />

src/lib/components/common/ImagePreview.svelte

@@ -7,6 +7,8 @@
 
 	let mounted = false;
 
+	let previewElement = null;
+
 	const downloadImage = (url, filename) => {
 		fetch(url)
 			.then((response) => response.blob())
@@ -34,14 +36,14 @@
 		mounted = true;
 	});
 
-	$: if (mounted) {
-		if (show) {
-			window.addEventListener('keydown', handleKeyDown);
-			document.body.style.overflow = 'hidden';
-		} else {
-			window.removeEventListener('keydown', handleKeyDown);
-			document.body.style.overflow = 'unset';
-		}
+	$: if (show && previewElement) {
+		document.body.appendChild(previewElement);
+		window.addEventListener('keydown', handleKeyDown);
+		document.body.style.overflow = 'hidden';
+	} else if (previewElement) {
+		window.removeEventListener('keydown', handleKeyDown);
+		document.body.removeChild(previewElement);
+		document.body.style.overflow = 'unset';
 	}
 </script>
 
@@ -49,9 +51,10 @@
 	<!-- svelte-ignore a11y-click-events-have-key-events -->
 	<!-- svelte-ignore a11y-no-static-element-interactions -->
 	<div
-		class="fixed top-0 right-0 left-0 bottom-0 bg-black text-white w-full min-h-screen h-screen flex justify-center z-50 overflow-hidden overscroll-contain"
+		bind:this={previewElement}
+		class="modal fixed top-0 right-0 left-0 bottom-0 bg-black text-white w-full min-h-screen h-screen flex justify-center z-[9999] overflow-hidden overscroll-contain"
 	>
-		<div class=" absolute left-0 w-full flex justify-between">
+		<div class=" absolute left-0 w-full flex justify-between select-none">
 			<div>
 				<button
 					class=" p-5"
@@ -95,6 +98,6 @@
 				</button>
 			</div>
 		</div>
-		<img {src} {alt} class=" mx-auto h-full object-scale-down" />
+		<img {src} {alt} class=" mx-auto h-full object-scale-down select-none" draggable="false" />
 	</div>
 {/if}

src/lib/components/common/Loader.svelte

@@ -0,0 +1,30 @@
+<script lang="ts">
+	import { createEventDispatcher, onMount } from 'svelte';
+	const dispatch = createEventDispatcher();
+
+	let loaderElement: HTMLElement;
+
+	onMount(() => {
+		const observer = new IntersectionObserver(
+			(entries, observer) => {
+				entries.forEach((entry) => {
+					if (entry.isIntersecting) {
+						dispatch('visible');
+						// observer.unobserve(loaderElement); // Stop observing until content is loaded
+					}
+				});
+			},
+			{
+				root: null, // viewport
+				rootMargin: '0px',
+				threshold: 0.1 // When 10% of the loader is visible
+			}
+		);
+
+		observer.observe(loaderElement);
+	});
+</script>
+
+<div bind:this={loaderElement}>
+	<slot />
+</div>

src/lib/components/common/Valves.svelte

@@ -27,7 +27,7 @@
 					on:click={() => {
 						valves[property] =
 							(valves[property] ?? null) === null
-								? valvesSpec.properties[property]?.default ?? ''
+								? (valvesSpec.properties[property]?.default ?? '')
 								: null;
 
 						dispatch('change');

src/lib/components/layout/Sidebar.svelte

@@ -11,7 +11,9 @@
 		showSidebar,
 		mobile,
 		showArchivedChats,
-		pinnedChats
+		pinnedChats,
+		scrollPaginationEnabled,
+		currentChatPage
 	} from '$lib/stores';
 	import { onMount, getContext, tick } from 'svelte';
 
@@ -34,6 +36,8 @@
 	import UserMenu from './Sidebar/UserMenu.svelte';
 	import ChatItem from './Sidebar/ChatItem.svelte';
 	import DeleteConfirmDialog from '$lib/components/common/ConfirmDialog.svelte';
+	import Spinner from '../common/Spinner.svelte';
+	import Loader from '../common/Loader.svelte';
 
 	const BREAKPOINT = 768;
 
@@ -50,6 +54,10 @@
 
 	let filteredChatList = [];
 
+	// Pagination variables
+	let chatListLoading = false;
+	let allChatsLoaded = false;
+
 	$: filteredChatList = $chats.filter((chat) => {
 		if (search === '') {
 			return true;
@@ -70,6 +78,29 @@
 		}
 	});
 
+	const enablePagination = async () => {
+		// Reset pagination variables
+		currentChatPage.set(1);
+		allChatsLoaded = false;
+		await chats.set(await getChatList(localStorage.token, $currentChatPage));
+
+		// Enable pagination
+		scrollPaginationEnabled.set(true);
+	};
+
+	const loadMoreChats = async () => {
+		chatListLoading = true;
+
+		currentChatPage.set($currentChatPage + 1);
+		const newChatList = await getChatList(localStorage.token, $currentChatPage);
+
+		// once the bottom of the list has been reached (no results) there is no need to continue querying
+		allChatsLoaded = newChatList.length === 0;
+		await chats.set([...$chats, ...newChatList]);
+
+		chatListLoading = false;
+	};
+
 	onMount(async () => {
 		mobile.subscribe((e) => {
 			if ($showSidebar && e) {
@@ -82,9 +113,8 @@
 		});
 
 		showSidebar.set(window.innerWidth > BREAKPOINT);
-
 		await pinnedChats.set(await getChatListByTagName(localStorage.token, 'pinned'));
-		await chats.set(await getChatList(localStorage.token));
+		await enablePagination();
 
 		let touchstart;
 		let touchend;
@@ -185,7 +215,11 @@
 				await tick();
 				goto('/');
 			}
-			await chats.set(await getChatList(localStorage.token));
+
+			allChatsLoaded = false;
+			currentChatPage.set(1);
+			await chats.set(await getChatList(localStorage.token, $currentChatPage));
+
 			await pinnedChats.set(await getChatListByTagName(localStorage.token, 'pinned'));
 		}
 	};
@@ -410,7 +444,10 @@
 						class="w-full rounded-r-xl py-1.5 pl-2.5 pr-4 text-sm bg-transparent dark:text-gray-300 outline-none"
 						placeholder={$i18n.t('Search')}
 						bind:value={search}
-						on:focus={() => {
+						on:focus={async () => {
+							// TODO: migrate backend for more scalable search mechanism
+							scrollPaginationEnabled.set(false);
+							await chats.set(await getChatList(localStorage.token)); // when searching, load all chats
 							enrichChatsWithContent($chats);
 						}}
 					/>
@@ -422,7 +459,7 @@
 					<button
 						class="px-2.5 text-xs font-medium bg-gray-50 dark:bg-gray-900 dark:hover:bg-gray-800 transition rounded-full"
 						on:click={async () => {
-							await chats.set(await getChatList(localStorage.token));
+							await enablePagination();
 						}}
 					>
 						{$i18n.t('all')}
@@ -431,12 +468,17 @@
 						<button
 							class="px-2.5 text-xs font-medium bg-gray-50 dark:bg-gray-900 dark:hover:bg-gray-800 transition rounded-full"
 							on:click={async () => {
+								scrollPaginationEnabled.set(false);
 								let chatIds = await getChatListByTagName(localStorage.token, tag.name);
 								if (chatIds.length === 0) {
 									await tags.set(await getAllChatTags(localStorage.token));
-									chatIds = await getChatList(localStorage.token);
+
+									// if the tag we deleted is no longer a valid tag, return to main chat list view
+									await enablePagination();
 								}
 								await chats.set(chatIds);
+
+								chatListLoading = false;
 							}}
 						>
 							{tag.name}
@@ -527,6 +569,21 @@
 						}}
 					/>
 				{/each}
+
+				{#if $scrollPaginationEnabled && !allChatsLoaded}
+					<Loader
+						on:visible={(e) => {
+							if (!chatListLoading) {
+								loadMoreChats();
+							}
+						}}
+					>
+						<div class="w-full flex justify-center py-1 text-xs animate-pulse items-center gap-2">
+							<Spinner className=" size-4" />
+							<div class=" ">Loading...</div>
+						</div>
+					</Loader>
+				{/if}
 			</div>
 		</div>
 

src/lib/components/layout/Sidebar/ChatItem.svelte

@@ -14,7 +14,7 @@
 		getChatListByTagName,
 		updateChatById
 	} from '$lib/apis/chats';
-	import { chatId, chats, mobile, pinnedChats, showSidebar } from '$lib/stores';
+	import { chatId, chats, mobile, pinnedChats, showSidebar, currentChatPage } from '$lib/stores';
 
 	import ChatMenu from './ChatMenu.svelte';
 	import ShareChatModal from '$lib/components/chat/ShareChatModal.svelte';
@@ -40,7 +40,9 @@
 			await updateChatById(localStorage.token, id, {
 				title: _title
 			});
-			await chats.set(await getChatList(localStorage.token));
+
+			currentChatPage.set(1);
+			await chats.set(await getChatList(localStorage.token, $currentChatPage));
 			await pinnedChats.set(await getChatListByTagName(localStorage.token, 'pinned'));
 		}
 	};
@@ -53,14 +55,18 @@
 
 		if (res) {
 			goto(`/c/${res.id}`);
-			await chats.set(await getChatList(localStorage.token));
+
+			currentChatPage.set(1);
+			await chats.set(await getChatList(localStorage.token, $currentChatPage));
 			await pinnedChats.set(await getChatListByTagName(localStorage.token, 'pinned'));
 		}
 	};
 
 	const archiveChatHandler = async (id) => {
 		await archiveChatById(localStorage.token, id);
-		await chats.set(await getChatList(localStorage.token));
+
+		currentChatPage.set(1);
+		await chats.set(await getChatList(localStorage.token, $currentChatPage));
 		await pinnedChats.set(await getChatListByTagName(localStorage.token, 'pinned'));
 	};
 
@@ -77,8 +83,8 @@
 			class=" w-full flex justify-between rounded-xl px-3 py-2 {chat.id === $chatId || confirmEdit
 				? 'bg-gray-200 dark:bg-gray-900'
 				: selected
-				? 'bg-gray-100 dark:bg-gray-950'
-				: 'group-hover:bg-gray-100 dark:group-hover:bg-gray-950'}  whitespace-nowrap text-ellipsis"
+					? 'bg-gray-100 dark:bg-gray-950'
+					: 'group-hover:bg-gray-100 dark:group-hover:bg-gray-950'}  whitespace-nowrap text-ellipsis"
 		>
 			<input
 				use:focusEdit
@@ -91,8 +97,8 @@
 			class=" w-full flex justify-between rounded-xl px-3 py-2 {chat.id === $chatId || confirmEdit
 				? 'bg-gray-200 dark:bg-gray-900'
 				: selected
-				? 'bg-gray-100 dark:bg-gray-950'
-				: ' group-hover:bg-gray-100 dark:group-hover:bg-gray-950'}  whitespace-nowrap text-ellipsis"
+					? 'bg-gray-100 dark:bg-gray-950'
+					: ' group-hover:bg-gray-100 dark:group-hover:bg-gray-950'}  whitespace-nowrap text-ellipsis"
 			href="/c/{chat.id}"
 			on:click={() => {
 				dispatch('select');
@@ -128,8 +134,8 @@
         {chat.id === $chatId || confirmEdit
 			? 'from-gray-200 dark:from-gray-900'
 			: selected
-			? 'from-gray-100 dark:from-gray-950'
-			: 'invisible group-hover:visible from-gray-100 dark:from-gray-950'}
+				? 'from-gray-100 dark:from-gray-950'
+				: 'invisible group-hover:visible from-gray-100 dark:from-gray-950'}
             absolute right-[10px] top-[6px] py-1 pr-2 pl-5 bg-gradient-to-l from-80%
 
               to-transparent"

src/lib/components/playground/Playground.svelte

@@ -121,7 +121,7 @@
 						? {
 								role: 'system',
 								content: system
-						  }
+							}
 						: undefined,
 					...messages
 				].filter((message) => message)

src/lib/components/workspace/Documents.svelte

@@ -88,7 +88,7 @@
 				tags?.length > 0
 					? {
 							tags: tags
-					  }
+						}
 					: null
 			).catch((error) => {
 				toast.error(error);

src/lib/components/workspace/Models.svelte

@@ -292,7 +292,7 @@
 			>
 				<div class=" self-start w-8 pt-0.5">
 					<div
-						class=" rounded-full bg-stone-700 {model?.info?.meta?.hidden ?? false
+						class=" rounded-full bg-stone-700 {(model?.info?.meta?.hidden ?? false)
 							? 'brightness-90 dark:brightness-50'
 							: ''} "
 					>
@@ -305,7 +305,7 @@
 				</div>
 
 				<div
-					class=" flex-1 self-center {model?.info?.meta?.hidden ?? false ? 'text-gray-500' : ''}"
+					class=" flex-1 self-center {(model?.info?.meta?.hidden ?? false) ? 'text-gray-500' : ''}"
 				>
 					<div class="  font-semibold line-clamp-1">{model.name}</div>
 					<div class=" text-xs overflow-hidden text-ellipsis line-clamp-1">

src/lib/components/workspace/Models/Knowledge/Selector.svelte

@@ -25,7 +25,7 @@
 							title: $i18n.t('All Documents'),
 							collection_names: $documents.map((doc) => doc.collection_name)
 						}
-				  ]
+					]
 				: []),
 			...$documents
 				.reduce((a, e, i, arr) => {