Add pure-PHP SQLite engine compatibility for WordPress and WooCommerce#431
Draft
adamziel wants to merge 12 commits into
Draft
Add pure-PHP SQLite engine compatibility for WordPress and WooCommerce#431adamziel wants to merge 12 commits into
adamziel wants to merge 12 commits into
Conversation
## What? Adds a browser-runtime compatibility check for the `wp_mysql_parser` WASM side modules and wires it into both WASM workflows: - `wasm-spike.yml` - `publish-wasm-extension-artifact.yml` The check compares each side module's `env` function imports against the matching Playground web PHP.wasm runtime exports. It runs for PHP 8.0 through 8.5 so the published manifest cannot claim browser support for a PHP version that will crash during extension startup. Also updates the native extension README/demo URL to use the live `blueprint.json` location and documents PHP 8.0-8.5 browser-runtime coverage instead of the temporary PHP 8.3 pin. ## Why? The previous Node-based smoke test was insufficient: PHP 8.4 loaded in the Node/CLI runtime but crashed in the browser runtime because the web runtime did not export Zend symbols imported by the extension side module. This PR makes that class of failure visible in CI before publishing a WASM manifest. ## Dependency This depends on WordPress/wordpress-playground#3690 and updated Playground web PHP.wasm artifacts. Current Playground web builds are known to miss required exports for PHP 8.0, 8.1, 8.4, and 8.5; the new CI check is expected to pass once those runtime exports ship. ## Testing - `node --check packages/php-ext-wp-mysql-parser/wasm-spike/check-playground-web-compat.mjs` - `git diff --check` - Verified locally with the Playground checkout: - PHP 8.3 currently passes. - PHP 8.4 currently fails with missing `zend_declare_class_constant_ex` and `zend_register_internal_class_ex`, matching the browser crash root cause. --------- Co-authored-by: Jan Jakeš <jan@jakes.pro> Co-authored-by: Chloe Pomegranate <chloehoughtonjenkins+git@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Francesco Bigiarini <francesco.bigiarini@gmail.com> Co-authored-by: Wojtek Naruniec <wojtek@naruniec.me> Co-authored-by: wpfuse <113634078+wp-fuse@users.noreply.github.com> Co-authored-by: Ashish Kumar <ashfame@users.noreply.github.com> Co-authored-by: Jon Surrell <sirreal@users.noreply.github.com> Co-authored-by: John Blackbourn <john@johnblackbourn.com> Co-authored-by: Jonathan Desrosiers <359867+desrosj@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This reverts commit b70a9ff.
Introduce WP_PHP_Engine — an SQLite-compatible database engine written
entirely in PHP, with no dependency on the pdo_sqlite or sqlite3
extensions. Tables live in PHP arrays, with optional persistence to a
plain file.
The engine plugs in underneath the existing MySQL-on-SQLite driver via
a PDO-compatible facade. Because the driver already translates MySQL to
SQLite SQL and registers its MySQL function library as PHP callbacks,
the entire driver stack — translation, information schema, transactions,
session semantics — runs unchanged on top of the new engine:
$pdo = new WP_PHP_Engine_PDO( 'php-engine::memory:' );
$connection = new WP_SQLite_Connection( array( 'pdo' => $pdo ) );
$driver = new WP_SQLite_Driver( $connection, 'wp' );
The engine implements the SQLite dialect surface the driver emits, and
then some: SELECT with joins/subqueries/CTEs/window functions/compound
queries, INSERT (incl. upserts), UPDATE (incl. UPDATE ... FROM and
multi-column assignments), DELETE, full DDL incl. STRICT tables and
AUTOINCREMENT, AFTER triggers, foreign keys with CASCADE/SET NULL/SET
DEFAULT, transactions and savepoints (O(1) snapshots via PHP COW),
sqlite_master, the PRAGMA introspection interface, type affinities,
collations (BINARY/NOCASE/RTRIM), and the core/date-time/aggregate
function library.
Conformance was driven by differential testing: a corpus of 37,548
SQLite queries captured from the driver test suites replays identically
(0 diffs) against the real SQLite and the PHP engine, including result
values, column names, error messages, and PDO column metadata. See
tests/tools/php-engine-differential.php.
The existing driver test suites now run twice — once against SQLite and
once against the PHP engine — via a small create_pdo() factory hook in
the test classes. All 1,171 tests (1.44M assertions) pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Persist DDL statements (CREATE TABLE/INDEX/TRIGGER/VIEW, DROP, ALTER) for file-backed databases. Persistence is now centralized in the execute() entry point instead of sprinkled across handlers, so every write statement is durable once no transaction is open. - Prevent lost updates between concurrent connections. The database file now carries a generation header, and every connection accesses it through one persistent handle with flock()-based locking: write statements (and whole transactions, from BEGIN to COMMIT/ROLLBACK) hold an exclusive lock around their read-modify-write cycle and reload the state if another process saved in the meantime; reads take a shared lock and reload when stale. The engine also refuses to open files in unrecognized formats, so it can never overwrite a real SQLite database file. - Wire the engine into production as a fallback: when the pdo_sqlite driver is missing (or WP_SQLITE_FORCE_PHP_ENGINE is set), WP_SQLite_Connection now transparently uses WP_PHP_Engine_PDO, and the db.php drop-in bootstraps the new SQLite driver on top of it instead of dying. Plugin activation and admin notices now only require PDO itself; composer moves ext-pdo_sqlite to "suggest". - bindParam() now binds by reference and reads the variable at execute() time, like PDO. - INSERT ... ON CONFLICT DO NOTHING now reports 0 affected rows, and DO UPDATE ... WHERE that filters out the update also reports 0. New WP_PHP_Engine_Tests cover file-backed DDL/data/AUTOINCREMENT persistence across reopens, rollback durability, concurrent-connection lost-update prevention, cross-connection visibility, foreign-file refusal, upsert row counts, and bindParam reference semantics. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collaborator
Author
|
Fair local benchmark (same WordPress SQLite driver stack, only backend swapped):
Result: on this small in-memory WordPress-ish workload, the pure-PHP engine is ~3.6x slower than SQLite PDO. This is intended as an apples-to-apples backend comparison through the same driver/translation layer, not as a production throughput claim. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it does
Adds a pure-PHP SQLite-compatible engine path so the SQLite integration can run without
pdo_sqlite.This PR also expands compatibility for WordPress and WooCommerce-style SQL:
WP_PHP_Engine_PDOinto the MySQL-on-SQLite driver and install/bootstrap paths.information_schema, views, triggers, routines/events metadata, logical database metadata, CTAS, defaults, ALTER TABLE cases, and legacy SQLite fallbacks.Rationale
WordPress Playground and similar environments may not have native SQLite/PDO support available. A pure-PHP fallback makes the integration usable in those environments while preserving the existing PDO-based path where native SQLite is available.
Concurrency is intentionally conservative. The pure-PHP engine is not trying to emulate SQLite’s full pager, WAL mode, MVCC, or cross-process read/write scheduling. It serializes writes with a database-level lock and uses transaction snapshots to avoid lost updates. That is enough for the fallback runtime this PR targets, but it is not a high-concurrency database engine and should not be presented as one.
Implementation
The pure-PHP path stores database state as PHP arrays and persists it for file-backed databases.
WP_PHP_Engine_PDOexposes enough PDO behavior for the existing driver stack, including legacy PHP/PDO fetch quirks covered by the package matrix.Concurrency support is limited to correctness-oriented safeguards. For a file-backed database, overlapping writes behave like this:
flock()on the database file.PDO::ATTR_TIMEOUT, orPRAGMA busy_timeoutin milliseconds). If the lock is released in time, it reloads the just-written generation before applying its own changes. If not, it fails with SQLite-compatible code 5,database is locked.Explicit transactions hold that exclusive lock from
BEGINuntilCOMMITorROLLBACK. Reads from other file-backed connections take a shared lock and use the same busy-timeout behavior: they wait while a write transaction is open, then either reload the committed generation before reading or fail withdatabase is lockedif the timeout expires.ROLLBACKrestores the transaction snapshot and releases the lock without exposing intermediate writes.For
:memory:databases, this is just a per-connection in-memory engine. There is no cross-connection shared database and therefore no cross-connection write coordination.Known concurrency shortcomings:
Testing instructions
CI is expected to pass across the full package PHP/SQLite matrix, WordPress PHPUnit, WordPress e2e, code style, and proxy tests.