diff --git a/docs/getting-started/advanced-topics/scaling.md b/docs/getting-started/advanced-topics/scaling.md index c441cd0f8..26ddc4289 100644 --- a/docs/getting-started/advanced-topics/scaling.md +++ b/docs/getting-started/advanced-topics/scaling.md @@ -26,18 +26,25 @@ This is perfect for personal use, small teams, or evaluation. The scaling journe --- -## Step 1 — Switch to PostgreSQL +## Step 1 — Switch to an External SQL Database (PostgreSQL, MariaDB) **When:** You plan to run more than one Open WebUI instance, or you want better performance and reliability for your database. -SQLite stores everything in a single file and doesn't handle concurrent writes from multiple processes well. PostgreSQL is a production-grade database that supports many simultaneous connections. +SQLite stores everything in a single file and doesn't handle concurrent writes from multiple processes well. For production deployments, switch to an external SQL database such as **PostgreSQL** or **MariaDB**. **What to do:** -Set the `DATABASE_URL` environment variable to point to your PostgreSQL server: +Set the `DATABASE_URL` environment variable to point to your database server: ``` +# PostgreSQL DATABASE_URL=postgresql://user:password@db-host:5432/openwebui + +# MariaDB (recommended when using MariaDB specifically) +DATABASE_URL=mariadb+mariadbconnector://user:password@db-host:3306/openwebui + +# MySQL-compatible / PyMySQL alternative +DATABASE_URL=mysql+pymysql://user:password@db-host:3306/openwebui ``` **Key things to know:** @@ -48,7 +55,7 @@ DATABASE_URL=postgresql://user:password@db-host:5432/openwebui - If you skip this step and run multiple instances with SQLite, you will see `database is locked` errors and data corruption. See [Database Corruption / "Locked" Errors](/troubleshooting/multi-replica#4-database-corruption--locked-errors) for details. :::tip -A good starting point for tuning is `DATABASE_POOL_SIZE=15` and `DATABASE_POOL_MAX_OVERFLOW=20`. Keep the combined total per instance well below your PostgreSQL `max_connections` limit (default is 100). +A good starting point for tuning is `DATABASE_POOL_SIZE=15` and `DATABASE_POOL_MAX_OVERFLOW=20`. Keep the combined total per instance well below your database server's connection limit. E.g PostgreSQL `max_connections` default limit is 100. ::: --- @@ -89,7 +96,7 @@ For a complete step-by-step Redis setup (Docker Compose, Sentinel, Cluster mode, Open WebUI is stateless, so you can run as many instances as needed behind a **load balancer**. Each instance is identical and interchangeable. :::warning -Before running multiple instances, ensure you have completed **Steps 1 and 2** (PostgreSQL and Redis). You also need a shared `WEBUI_SECRET_KEY` across all replicas — without it, users will experience [login loops and 401 errors](/troubleshooting/multi-replica#1-login-loops--401-unauthorized-errors). For a full pre-flight checklist, see the [Core Requirements Checklist](/troubleshooting/multi-replica#core-requirements-checklist). +Before running multiple instances, ensure you have completed **Steps 1 and 2** (external SQL database and Redis). You also need a shared `WEBUI_SECRET_KEY` across all replicas — without it, users will experience [login loops and 401 errors](/troubleshooting/multi-replica#1-login-loops--401-unauthorized-errors). For a full pre-flight checklist, see the [Core Requirements Checklist](/troubleshooting/multi-replica#core-requirements-checklist). ::: ### Option A: Container Orchestration (Recommended) @@ -108,7 +115,7 @@ For simpler setups (e.g., a single powerful server), increase `UVICORN_WORKERS`: UVICORN_WORKERS=4 ``` -This spawns multiple application processes inside a single container. You still need PostgreSQL and Redis when using this approach. +This spawns multiple application processes inside a single container. You still need an external SQL database and Redis when using this approach. :::info Container orchestration is generally preferred because it provides automatic restarts, rolling updates, and more granular resource control. Multiple workers inside a single container is a simpler alternative when orchestration isn't available. @@ -164,6 +171,8 @@ Only PGVector and ChromaDB will be consistently maintained by the Open WebUI tea :::tip **PGVector** is the simplest choice if you're already running PostgreSQL for the main database — it adds vector search to the database you already have, with no additional infrastructure. +If you're standardizing on MariaDB for the primary database, **MariaDB Vector** provides a similar single-database-system deployment model for both application data and vector search. + For maximum scalability in self-hosted environments, **Milvus** and **Qdrant** both support **multitenancy mode** (`ENABLE_MILVUS_MULTITENANCY_MODE=True` / `ENABLE_QDRANT_MULTITENANCY_MODE=True`), which provides better resource sharing at scale. ::: @@ -354,7 +363,7 @@ ENABLE_DB_MIGRATIONS=false ## Quick Reference: When Do I Need What? -| Scenario | PostgreSQL | Redis | External Vector DB | Ext. Content Extraction | Ext. Embeddings | Shared Storage | +| Scenario | External SQL DB | Redis | External Vector DB | Ext. Content Extraction | Ext. Embeddings | Shared Storage | |---|:---:|:---:|:---:|:---:|:---:|:---:| | Single user / evaluation | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | Small team (< 50 users, single instance) | Recommended | ✗ | ✗ | Recommended | ✗ | ✗ | diff --git a/docs/reference/database-schema.md b/docs/reference/database-schema.md index 69050c5c0..97ffb21ac 100644 --- a/docs/reference/database-schema.md +++ b/docs/reference/database-schema.md @@ -880,7 +880,7 @@ To use SQLCipher with existing data, you must either: 1. **Start fresh** - Enable SQLCipher on a new installation and have users export/re-import their chats manually 2. **Manual database migration** - Use external SQLite/SQLCipher tools to export data from the unencrypted database and import it into a new encrypted database (advanced users only) 3. **Use filesystem-level encryption** - Consider alternatives like LUKS (Linux) or BitLocker (Windows) for at-rest encryption without database-level changes -4. **Switch to PostgreSQL** - For multi-user deployments, PostgreSQL with TLS provides encryption in transit and can be combined with encrypted storage +4. **Switch to an external SQL database** - For multi-user deployments, PostgreSQL or MariaDB with TLS provides encryption in transit and can be combined with encrypted storage ::: diff --git a/docs/reference/env-configuration.mdx b/docs/reference/env-configuration.mdx index 7ee7f08e6..35f793180 100644 --- a/docs/reference/env-configuration.mdx +++ b/docs/reference/env-configuration.mdx @@ -6098,14 +6098,20 @@ pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp :::info -**For PostgreSQL support, ensure you installed with `pip install open-webui[all]` instead of the basic installation.** -Supports SQLite, Postgres, and encrypted SQLite via SQLCipher. +**For PostgreSQL, MariaDB support, ensure you installed with `pip install open-webui[all]` instead of the basic installation.** +Supports SQLite, PostgreSQL, MariaDB databases, and encrypted SQLite via SQLCipher. **Changing the URL does not migrate data between databases.** Documentation on the URL scheme is available [here](https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls). If your database password contains special characters, please ensure they are properly URL-encoded. For example, a password like `p@ssword` should be encoded as `p%40ssword`. +Examples: +- SQLite: `sqlite:///${DATA_DIR}/webui.db` +- PostgreSQL: `postgresql://user:password@db-host:5432/openwebui` +- MariaDB (preferred): `mariadb+mariadbconnector://user:password@db-host:3306/openwebui` +- MySQL/MariaDB compatibility fallback: `mysql+pymysql://user:password@db-host:3306/openwebui` + For configuration using individual parameters or encrypted SQLite, see the relevant sections below. ::: @@ -6119,7 +6125,7 @@ For configuration using individual parameters or encrypted SQLite, see the relev :::warning **Required for Multi-Replica Setups** -For multi-replica or high-availability deployments (Kubernetes, Docker Swarm), you **MUST** use an external database (PostgreSQL) instead of SQLite. SQLite does not support concurrent writes from multiple instances and will result in database corruption or data inconsistency. +For multi-replica or high-availability deployments (Kubernetes, Docker Swarm), you **MUST** use an external database (PostgreSQL, MariaDB) instead of SQLite. SQLite does not support concurrent writes from multiple instances and will result in database corruption or data inconsistency. ::: @@ -6127,7 +6133,7 @@ For multi-replica or high-availability deployments (Kubernetes, Docker Swarm), y - Type: `str` - Default: `None` (automatically set to `sqlite` if `DATABASE_URL` uses default SQLite path) -- Description: Specifies the database type (e.g., `sqlite`, `postgresql`, `sqlite+sqlcipher`). This is used in conjunction with other individual parameters to construct the `DATABASE_URL` if a complete `DATABASE_URL` is not explicitly defined. +- Description: Specifies the database type (e.g., `sqlite`, `postgresql`, `sqlite+sqlcipher`). This is used in conjunction with other individual parameters to construct the `DATABASE_URL` if a complete `DATABASE_URL` is not explicitly defined. For MariaDB deployments, prefer setting the full `DATABASE_URL` explicitly. - Persistence: No #### `DATABASE_USER` diff --git a/docs/reference/index.md b/docs/reference/index.md index 91792df9f..9bde48818 100644 --- a/docs/reference/index.md +++ b/docs/reference/index.md @@ -21,7 +21,7 @@ Over 200 environment variables control authentication, model routing, storage, l | :--- | :--- | | 🔐 **Authentication & signup** | `ENABLE_SIGNUP`, `ENABLE_LOGIN_FORM`, `WEBUI_ADMIN_EMAIL`, OIDC/LDAP/SCIM | | 🤖 **Model connections** | Ollama, OpenAI, direct pipeline URLs, timeouts, load balancing | -| 💾 **Storage & databases** | SQLite, PostgreSQL, S3/GCS/Azure Blob, Redis | +| 💾 **Storage & databases** | SQLite, PostgreSQL, MariaDB, S3/GCS/Azure Blob, Redis | | 📊 **Logging & audit** | `GLOBAL_LOG_LEVEL`, JSON logging, audit log levels and paths | | 🧠 **RAG & retrieval** | Chunk size, overlap, embedding engines, reranking, vector DB selection | diff --git a/docs/troubleshooting/manual-database-migration.md b/docs/troubleshooting/manual-database-migration.md index fd527e1ac..4814bdaed 100644 --- a/docs/troubleshooting/manual-database-migration.md +++ b/docs/troubleshooting/manual-database-migration.md @@ -3,7 +3,7 @@ sidebar_position: 900 title: "Database Migration" sidebar_label: Manual Migration description: Complete guide for manually running Alembic database migrations when Open WebUI's automatic migration fails or requires direct intervention. -keywords: [alembic, migration, database, troubleshooting, sqlite, postgresql, docker] +keywords: [alembic, migration, database, troubleshooting, sqlite, postgresql, mariadb, mysql, docker] --- import Tabs from '@theme/Tabs'; @@ -19,7 +19,7 @@ You need manual migration only if: - Open WebUI logs show specific migration errors during startup - You're performing offline database maintenance - Automatic migration fails after a version upgrade -- You're migrating between database types (SQLite ↔ PostgreSQL) +- You're migrating between database types (SQLite ↔ PostgreSQL/MariaDB) - A developer has instructed you to run migrations manually ::: @@ -68,6 +68,11 @@ Database migrations cannot run while Open WebUI is active. You **must** stop all pg_dump -h localhost -U your_user -d open_webui_db > backup_$(date +%Y%m%d_%H%M%S).sql ``` + + ```bash title="Terminal" + mysqldump -h localhost -u your_user -p open_webui_db > backup_$(date +%Y%m%d_%H%M%S).sql + ``` + ### Verify Backup Integrity @@ -93,6 +98,13 @@ Database migrations cannot run while Open WebUI is active. You **must** stop all grep -c "CREATE TABLE" backup_*.sql ``` + + ```bash title="Terminal - Verify Backup" + # Verify backup file is not empty and contains SQL + head -n 20 backup_*.sql + grep -c "CREATE TABLE" backup_*.sql + ``` + :::tip Backup Storage @@ -168,6 +180,12 @@ export DATABASE_URL="sqlite:////app/backend/data/webui.db" # For PostgreSQL export DATABASE_URL="postgresql://user:password@localhost:5432/open_webui_db" +# For MariaDB (preferred) +export DATABASE_URL="mariadb+mariadbconnector://user:password@localhost:3306/open_webui_db" + +# MariaDB compatibility fallback +export DATABASE_URL="mysql+pymysql://user:password@localhost:3306/open_webui_db" + # Required: WEBUI_SECRET_KEY # Get from existing file in backend directory (NOT data directory) export WEBUI_SECRET_KEY=$(cat /app/backend/.webui_secret_key) @@ -198,6 +216,12 @@ export DATABASE_URL="sqlite:////full/path/to/webui.db" # For PostgreSQL export DATABASE_URL="postgresql://user:password@localhost:5432/open_webui_db" +# For MariaDB (preferred) +export DATABASE_URL="mariadb+mariadbconnector://user:password@localhost:3306/open_webui_db" + +# MariaDB compatibility fallback +export DATABASE_URL="mysql+pymysql://user:password@localhost:3306/open_webui_db" + # Required: WEBUI_SECRET_KEY # If using .env file, Alembic may not pick it up automatically - export manually export WEBUI_SECRET_KEY=$(cat ../data/.webui_secret_key) diff --git a/docs/troubleshooting/multi-replica.mdx b/docs/troubleshooting/multi-replica.mdx index 8a6114a24..96ea5443f 100644 --- a/docs/troubleshooting/multi-replica.mdx +++ b/docs/troubleshooting/multi-replica.mdx @@ -14,7 +14,7 @@ If you are setting up a scaled deployment for the first time, start with the [Sc Before troubleshooting specific errors, ensure your deployment meets these **absolute requirements** for a multi-replica setup. Missing any of these will cause instability, login loops, or data loss. 1. **Shared Secret Key:** [`WEBUI_SECRET_KEY`](/reference/env-configuration#webui_secret_key) **MUST** be identical on all replicas. -2. **External Database:** You **MUST** use an external PostgreSQL database (see [`DATABASE_URL`](/reference/env-configuration#database_url)). SQLite is **NOT** supported for multiple instances. +2. **External Database:** You **MUST** use an external SQL database such as PostgreSQL or MariaDB (see [`DATABASE_URL`](/reference/env-configuration#database_url)). SQLite is **NOT** supported for multiple instances. 3. **Redis for WebSockets:** [`ENABLE_WEBSOCKET_SUPPORT=True`](/reference/env-configuration#enable_websocket_support) and [`WEBSOCKET_MANAGER=redis`](/reference/env-configuration#websocket_manager) with a valid [`WEBSOCKET_REDIS_URL`](/reference/env-configuration#websocket_redis_url) are required. 4. **Shared Storage:** A persistent volume (RWX / ReadWriteMany if possible, or ensuring all replicas map to the same underlying storage for `data/`) is critical for RAG (uploads/vectors) and generated images. 5. **External Vector Database (Required):** The default ChromaDB uses a local SQLite-backed `PersistentClient` that is **not safe for multi-worker or multi-replica deployments**. SQLite connections are not fork-safe, and concurrent writes from multiple processes will crash workers instantly. You **must** use a dedicated external Vector DB (e.g., [PGVector](/reference/env-configuration#pgvector_db_url), [MariaDB Vector](/reference/env-configuration#mariadb_vector_db_url), Milvus, Qdrant) via [`VECTOR_DB`](/reference/env-configuration#vector_db), or run ChromaDB as a [separate HTTP server](/reference/env-configuration#chroma_http_host). @@ -100,10 +100,14 @@ REDIS_URL=redis://your-redis-host:6379/0 Using **SQLite** with multiple replicas. SQLite is a file-based database and does not support concurrent network writes from multiple containers. **Solution:** -Migrate to **PostgreSQL**. Update your connection string: +Migrate to an **external SQL database**. Update your connection string: ```bash +# PostgreSQL DATABASE_URL=postgresql://user:password@postgres-host:5432/openwebui + +# MariaDB (recommended when targeting MariaDB specifically) +DATABASE_URL=mariadb+mariadbconnector://user:password@db-host:3306/openwebui ``` ### 5. Uploaded Files or RAG Knowledge Inaccessible diff --git a/docs/troubleshooting/performance.md b/docs/troubleshooting/performance.md index cc6dc72b8..61cdaf8c2 100644 --- a/docs/troubleshooting/performance.md +++ b/docs/troubleshooting/performance.md @@ -17,7 +17,7 @@ This guide provides a comprehensive overview of strategies to optimize Open WebU 3. **High Scale for Many Users (e.g., Enterprise/Production)**: * *Goal*: Stability and concurrency. - * *Strategy*: Requires dedicated Vector DBs (Milvus/Qdrant), increased thread pools, caching to handle load, and **PostgreSQL** instead of SQLite. + * *Strategy*: Requires dedicated Vector DBs (Milvus/Qdrant), increased thread pools, caching to handle load, and an **external SQL database** (PostgreSQL or MariaDB) instead of SQLite. --- @@ -78,11 +78,14 @@ Drastically improves the speed of follow-up questions when chatting with large d For high-scale deployments, your database configuration is the single most critical factor for stability. -### PostgreSQL (Mandatory for Scale) -For any multi-user or high-concurrency setup, **PostgreSQL is mandatory**. SQLite (the default) is not designed for high concurrency and will become a bottleneck (database locking errors). +### External SQL Database (Mandatory for Scale) +For any multi-user or high-concurrency setup, an **external SQL database** is mandatory. SQLite (the default) is not designed for high concurrency and will become a bottleneck (database locking errors). - **Variable**: `DATABASE_URL` -- **Example**: `postgres://user:password@localhost:5432/webui` +- **Examples**: + - `postgresql://user:password@localhost:5432/webui` + - `mariadb+mariadbconnector://user:password@localhost:3306/webui` + - `mysql+pymysql://user:password@localhost:3306/webui` ### Chat Saving Strategy @@ -128,6 +131,7 @@ For multi-user setups, the choice of Vector DB matters. - **Recommendations**: * **Milvus** or **Qdrant**: Best for improved scale and performance. These are client-server databases, inherently safe for multi-process access. * **PGVector**: Excellent choice if you are already using PostgreSQL. Also fully multi-process safe. + * **MariaDB Vector**: Good fit if you are already using MariaDB as the primary database and want both application data and vector search on one database system. * **ChromaDB HTTP mode**: If you want to keep using ChromaDB, run it as a [separate server](/reference/env-configuration#chroma_http_host) so Open WebUI connects via HTTP instead of local SQLite. - **Multitenancy**: If using Milvus or Qdrant, enabling multitenancy offers better resource sharing. * `ENABLE_MILVUS_MULTITENANCY_MODE=True`