feat: add wildcard database template support#794
feat: add wildcard database template support#794hikionori wants to merge 9 commits intopgdogdev:mainfrom
Conversation
Support wildcard database templates (name="*") to dynamically route unknown database names to a real PostgreSQL server without explicit per-database configuration. Features: - Wildcard database templates: name="*" entry matches any unmapped database - Wildcard users: name="*" and/or database="*" for flexible auth routing - Dynamic pool creation: pools created on-demand during client login - Passthrough auth integration: client credentials correctly forwarded to backend in dynamically created pools Implementation includes: - exists_or_wildcard() API to check if pool exists or can be created - add_wildcard_pool(user, db, passthrough_password) for dynamic creation - Priority-based wildcard matching (explicit > user wildcard > db wildcard > both) - Integration tests for trust/passthrough auth, concurrency, error handling - Unit tests for wildcard matching and dynamic pool creation logic Fixes passthrough auth failures when creating wildcard pools by ensuring client passwords are propagated to the backend connection configuration.
|
Thank you for the PR! I'm currently OOO, I'll be back next week and I'll take a look. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Thanks for submitting this. I'll take a look. However, one important clarification:
This is not true. We support configuration reloading with no downtime. You can send a |
… creation Add two #[ignore] integration test modules for wildcard pool edge cases: - pool_cap_saturation: tests max_wildcard_pools limit enforcement, no-corruption of existing pools on cap rejection, and pool reuse after idle eviction. - concurrent_pool_creation: tests race-safe pool instantiation when multiple concurrent connections target the same or different wildcard databases. Also adds admin SET support for max_wildcard_pools and wildcard_pool_idle_timeout, enabling runtime configuration of these settings during tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wildcard users/databases (name="*" or database="*") were being passed to new_pool() in from_config(), creating static pools with literal "*" as the database name and username. When min_pool_size > 0, the pool monitor would immediately try to connect to Postgres using these literal values, causing "database * does not exist" and "password authentication failed for user *" errors. The fix: 1. Skip new_pool() for wildcard user entries in from_config() — these templates are stored in wildcard_db_templates/wildcard_users and pools are created lazily via add_wildcard_pool() when real clients connect. 2. Update find_wildcard_match() to resolve priorities from wildcard_users list instead of checking self.databases HashMap keys, since wildcard entries no longer exist as static pools. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wildcard template pools are created lazily on demand, so pre-warming with min_pool_size > 0 has no benefit and previously triggered the literal "*" connection bug. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When database_name is explicitly set to "*" in the wildcard template config, add_wildcard_pool() was not substituting it with the actual database name. Only database_name=None triggered substitution. Now treats database_name="*" the same as None — both get replaced with the client-requested database name. Also removes the redundant database_name="*" from the generated config template. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Added SQL script for tenant workload simulation. - Created a bash script to run production readiness tests with configurable parameters. - Implemented Python script to generate PgDog configuration. - Developed bash script to generate tenant databases and test users. - Added SQL initialization script for setting up the database schema and seed data. - Introduced memory monitoring and metrics validation scripts. - Implemented pool validation checks for PgDog. - Enhanced Rust integration tests for concurrent pool creation and connection handling. - Added functionality to remove wildcard pools for credential rotation. - Updated client authentication logic to handle password changes and pool recreation.
- Introduced a new script `pool_lifecycle.sh` to test the lifecycle of wildcard pools, validating eviction, memory stability, and recreation of pools. - Enhanced `scale_connect.sh` to include a minimum success rate parameter, allowing for better control over test success criteria. - Updated `run.sh` to integrate the new pool lifecycle test and added additional parameters for configuration, including idle timeout and lifecycle settings. - Implemented a new script `configure_toxiproxy.sh` to set up Toxiproxy for fault injection testing. - Modified `generate_tenants.sh` to support quiet mode for less verbose output during tenant creation.
|
Hello! I’m following up on this pull request to kindly ask if you’ve had a chance to review it. I understand you may be busy, so no rush — I just wanted to gently bring it back to your attention. Thank you for your time. @levkk |
Summary
Support wildcard database and user templates (
name = "*") to dynamically routeunknown
(user, database)pairs to a real PostgreSQL server without explicitper-database configuration and without a proxy restart.
Problem
PgDog has no way to connect users to databases that aren't explicitly listed in
pgdog.toml. Any user/database pair not in the config gets an immediate "nosuch database" error, forcing operators to restart the proxy every time a new
database is provisioned.
Solution
Add wildcard support for both
databasesandusersentries. A database oruser whose
namefield is"*"acts as a catch-all template. When a clientconnects to an unknown
(user, database)pair,add_wildcard_poollooks forthe best-matching wildcard template, creates a pool on demand, and registers it
in the global
DATABASESstore — all without a restart or config reload.Configuration
Minimal wildcard setup
With limits and automatic eviction
Implementation
Wildcard matching
Four match priorities (highest wins):
user/*)*/db)*/*)exists_or_wildcard()APINew method on
Databasesreturnstrueif a pool already exists or if awildcard template would match the
(user, database)pair. Used during theclient-login handshake to decide whether to proceed without triggering pool
creation yet.
Dynamic pool creation (
add_wildcard_pool)LOCKfor the whole operation to prevent duplicate pools from racingthreads.
ConfigAndUserssnapshot at construction time sowildcard-pool creation is immune to concurrent SIGHUP reloads.
new_pool.connection config so the pool can authenticate to PostgreSQL and the
proxy-level credential check succeeds — fixing a bug where passthrough auth
would fail silently when a wildcard pool was created dynamically.
Pool limits (
max_wildcard_pools)wildcard_pool_countis tracked insideDatabasesand reset to zero on everyconfig reload, so the limit applies per-epoch. Once the limit is reached,
further connections return
Ok(None)(no error, treated as "no such database")until a slot is freed by eviction or SIGHUP.
Pool eviction (
wildcard_pool_idle_timeout)Dynamically-created pools are tracked in
Databases::dynamic_pools: HashSet<User>.A background task started once in
init()(viaWILDCARD_EVICTION: Lazy<()>)periodically calls
evict_idle_wildcard_pools():dynamic_pools.Cluster::total_connections()— sum ofidle + checked_outacrossall shards.
cluster.shutdown(), removes fromdynamic_pools, decrementswildcard_pool_countso new pools can fill thefreed slot immediately, without a config reload.
DATABASES.When
wildcard_pool_idle_timeout = 0(default), automatic eviction isdisabled and pools only leave
DATABASESon SIGHUP or restart.SIGHUP / config-reload behaviour
reload_from_existingrebuildsDatabasesfrom the updated config file.Wildcard pools are absent from the new config, so
replace_databasesdropsthem, their connections drain naturally, and
wildcard_pool_countresets tozero. The next client connection recreates the pool from the (potentially
updated) wildcard template.
Tests
Unit tests (14, all passing —
cargo nextest run --test-threads=1 wildcard)test_wildcard_database_simple_querytest_wildcard_exists_or_wildcardexists_or_wildcardreturns true for wildcard-matched pairstest_wildcard_add_pool_dynamicadd_wildcard_poolcreates and registers a pool on demandtest_wildcard_nonexistent_pg_databasetest_max_wildcard_pools_limit_enforcedOk(None)) when limit=2test_max_wildcard_pools_zero_means_unlimitedtest_max_wildcard_pools_counter_resets_on_reloadtest_evict_idle_wildcard_pools_removes_idle_pooltest_evict_idle_wildcard_pools_decrements_counttest_evict_idle_wildcard_pools_noop_on_emptydynamic_poolsis emptytest_wildcard_db_templates_populatedwildcard_db_templatesbuilt correctly from*databasestest_wildcard_users_populatedwildcard_usersbuilt correctly from*userstest_find_wildcard_match_prioritytest_no_wildcard_when_absenthas_wildcard()is false when no*entries existIntegration tests (
integration/wildcard/)Cover end-to-end scenarios against a live Postgres instance:
Files changed
pgdog-config/src/general.rsmax_wildcard_pools,wildcard_pool_idle_timeoutfieldspgdog/src/backend/databases.rsDatabasesstruct,add_wildcard_pool,evict_idle_wildcard_pools,WILDCARD_EVICTIONstatic,from_config,reload_from_existingdocs,exists_or_wildcardpgdog/src/backend/pool/cluster.rsCluster::total_connections()pgdog/src/config/mod.rsload_test_wildcard,load_test_wildcard_with_limittest helperspgdog/src/frontend/client/query_engine/test/wildcard.rsintegration/wildcard/