Skip to content

Commit 7c72fa5

Browse files
committed
feat: Implement database encryption and improve testing for WasmJs
- **Database Encryption**: - Migrate to `SQLite3MultipleCiphers` WASM to support encryption (AES) in the browser via OPFS. - Implement `encrypt`, `decrypt`, and `rekey` operations in `WebSafeRepo` using `PRAGMA key` and `PRAGMA rekey` statements. - Update `sqlite.worker.js` to attempt initialization with the `multipleciphers-opfs` VFS, falling back to standard `opfs` if unavailable. - Introduce a database readability check in `WebSafeRepo` to detect encrypted databases and handle incorrect keys. - **Testing Infrastructure**: - Add a comprehensive Karma configuration (`wasm-config.js`) to handle WASM file serving, middleware redirects, and extended timeouts for WasmGC compilation. - Implement `isKarmaTestRunner()` check in `main.kt` to prevent production app initialization during UI tests. - Auto-detect local Chrome/Chromium binaries for running Karma tests across different operating systems. - Expand `WebUiTests` with specific test cases for CRUD, encryption flows, and backup features. - Add `SimpleWasmTest` to verify basic Compose and arithmetic functionality on the WasmJs platform. - **Build & Documentation**: - Update `build.gradle.kts` to automate the download and extraction of `SQLite3MultipleCiphers` WASM artifacts. - Update project `README.md` and 8 `OPFS_IMPLEMENTATION.md` to reflect new encryption capabilities for the Web platform. - Synchronize `sqlite.worker.js` implementation across app and test source sets. - Standardize `WebSafeRepo` to use `PlatformSQLiteState` for tracking encryption status.
1 parent 1013e6b commit 7c72fa5

15 files changed

Lines changed: 558 additions & 74 deletions

File tree

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,15 +34,15 @@ Supported platforms:
3434

3535
![Architecture blueprint for this project](docs/diagrams/architecture.png)
3636

37-
## WORK IN PROGRESS 🛠
37+
## FEATURES ✨
3838

3939
| feature \ platform | Android | iOS | Desktop Java | Web |
4040
|:------------------:|:-------:|:---:|:------------:|:---:|
4141
| database |||||
42-
| encryption |||| |
42+
| encryption |||| |
4343
| backup |||||
4444

45-
Check out [CONTRIBUTING.md](/CONTRIBUTING.md) if you want to develop missing features.
45+
Interested in contributing new features or fixes? Check out [CONTRIBUTING.md](/CONTRIBUTING.md).
4646

4747
## CONTINUOUS INTEGRATION / DELIVERY ♻️
4848

app/web/README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -84,10 +84,10 @@ HTML entry point:
8484

8585
### Database
8686

87-
- **Official SQLite WASM**: Native SQLite compiled to WebAssembly
87+
- **SQLite3MultipleCiphers WASM**: SQLite with encryption support compiled to WebAssembly
8888
- **OPFS Storage**: Origin-Private FileSystem for persistent database storage
8989
- **Web Worker**: Database operations run off the main thread
90-
- **No encryption**: SQLCipher not available in browsers
90+
- **Encryption ready**: [SQLite3MultipleCiphers](https://github.com/utelle/SQLite3MultipleCiphers) provides cipher support in the browser
9191

9292
### Webpack
9393

@@ -124,7 +124,7 @@ build/dist/wasmJs/productionExecutable/
124124
├── composeApp.wasm # Application WebAssembly binary
125125
├── skiko.wasm # Skia graphics engine
126126
├── sqlite3.js # SQLite JavaScript
127-
├── sqlite3.wasm # Official SQLite WASM
127+
├── sqlite3.wasm # SQLite3MultipleCiphers WASM (with encryption)
128128
├── sqlite.worker.js # Custom OPFS worker
129129
├── coi-serviceworker.js # Service worker for headers
130130
└── sql-wasm.wasm # Legacy SQL.js (fallback)
@@ -280,7 +280,7 @@ fun isWasmSupported(): Boolean = js("""
280280

281281
### Current Limitations
282282

283-
1. **No encryption**: SQLCipher not available in browsers
283+
1. **Encryption**: [SQLite3MultipleCiphers](https://github.com/utelle/SQLite3MultipleCiphers) WASM provides encrypt/decrypt/rekey via `PRAGMA key`/`PRAGMA rekey`
284284
2. ✅ **Storage**: OPFS provides persistent database storage
285285
3. ⚠️ **File access**: Restricted browser file API
286286
4. ⚠️ **Performance**: Slower than native (improving)
@@ -309,7 +309,7 @@ The web app now uses OPFS (Origin-Private FileSystem) for persistent database st
309309
- ✅ **Persistent storage**: Survives browser sessions
310310
- ✅ **Better performance**: Direct file system access
311311
- ✅ **Larger capacity**: Not limited by IndexedDB quotas
312-
- ✅ **Real SQLite**: Uses official SQLite WASM build
312+
- ✅ **Real SQLite with encryption**: Uses SQLite3MultipleCiphers WASM build
313313

314314
### Browser Support
315315

@@ -506,7 +506,7 @@ When working with this module:
506506
3. **Size matters**: Minimize bundle size
507507
4. **Progressive enhancement**: Detect and use modern APIs gracefully
508508
5. **Testing**: Test in multiple browsers
509-
6. **Security**: No sensitive data without encryption
509+
6. **Security**: Encryption available via SQLite3MultipleCiphers
510510
7. **Performance**: Profile and optimize load time
511511
8. **Responsive**: Support mobile and desktop browsers
512512
9. **Accessibility**: Follow WCAG guidelines

app/web/build.gradle.kts

Lines changed: 39 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -16,27 +16,22 @@ kotlin {
1616
wasmJs {
1717
outputModuleName.set("composeApp")
1818
browser {
19-
val rootDirPath = project.rootDir.path
20-
val projectDirPath = project.projectDir.path
21-
testTask {
22-
useKarma {
23-
useChrome()
24-
useChromeHeadless()
25-
}
26-
}
19+
val rootDirPath: String = project.rootDir.path
20+
val projectDirPath: String = project.projectDir.path
2721
commonWebpackConfig {
2822
outputFileName = "composeApp.js"
2923
devServer = (devServer ?: KotlinWebpackConfig.DevServer()).apply {
30-
// Serve sources to debug inside browser
24+
// Serve sources to debug inside the browser
3125
static(rootDirPath)
3226
static(projectDirPath)
3327
}
3428
}
29+
testTask { useKarma { useChromeHeadless() } }
3530
}
3631
binaries.executable()
3732
}
3833
sourceSets {
39-
val wasmJsMain by getting {
34+
wasmJsMain {
4035
dependencies {
4136
implementation(projects.core.domain)
4237
implementation(projects.core.presentation)
@@ -49,37 +44,61 @@ kotlin {
4944
}
5045
resources.srcDir(layout.buildDirectory.dir("sqlite"))
5146
}
52-
val wasmJsTest by getting {
47+
wasmJsTest {
5348
dependencies {
5449
implementation(kotlin("test"))
5550
implementation(projects.ui.test)
5651
implementation(libs.compose.ui.test)
52+
implementation(libs.compose.material3)
5753
implementation(libs.compose.material.icons.extended)
5854
implementation(libs.androidx.lifecycle.runtime.compose)
5955
implementation(libs.androidx.lifecycle.runtime.testing)
56+
implementation(libs.androidx.paging.common)
6057
}
6158
resources.srcDir(layout.buildDirectory.dir("sqlite"))
6259
}
6360
}
6461
}
6562

66-
val chromeBinaryFromEnv = providers.environmentVariable("CHROME_BIN").orNull
67-
val hasChromeForTests = chromeBinaryFromEnv?.let { file(it).exists() } == true
63+
// Chrome binary for Karma tests.
64+
// Auto-detects Chrome on macOS/Linux/Windows when CHROME_BIN is not set.
65+
// NOTE: WasmGC module compilation in Chrome is very slow for large modules (~30 MB test WASM).
66+
// The main() function is guarded with isKarmaTestRunner() check to prevent the production app
67+
// from launching during tests (see main.kt).
68+
val defaultChromePaths: List<String> = listOf(
69+
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", // macOS
70+
"/usr/bin/google-chrome-stable", // Linux (stable)
71+
"/usr/bin/google-chrome", // Linux
72+
"/usr/bin/chromium-browser", // Linux (Chromium)
73+
"/usr/bin/chromium", // Linux (Chromium alt)
74+
"C:/Program Files/Google/Chrome/Application/chrome.exe", // Windows
75+
"C:/Program Files (x86)/Google/Chrome/Application/chrome.exe", // Windows x86
76+
)
77+
val chromeBinary: String? = providers.environmentVariable("CHROME_BIN").orNull
78+
?: defaultChromePaths.firstOrNull { file(it).exists() }
6879

6980
tasks.named<KotlinJsTest>("wasmJsBrowserTest").configure {
70-
enabled = hasChromeForTests
81+
enabled = chromeBinary != null
82+
if (chromeBinary != null) {
83+
environment("CHROME_BIN", chromeBinary)
84+
}
7185
}
7286

73-
val sqliteVersion = 3500400 // See https://sqlite.org/download.html for the latest wasm build version
74-
val sqliteDownload = tasks.register("sqliteDownload", Download::class.java) {
75-
src("https://sqlite.org/2025/sqlite-wasm-$sqliteVersion.zip")
87+
// SQLite3MultipleCiphers WASM build with encryption support
88+
// See https://github.com/utelle/SQLite3MultipleCiphers/releases for the latest version
89+
val sqlite3mcVersion = "2.2.7"
90+
val sqliteVersion = "3.51.2"
91+
val sqliteWasmVersion = "3510200"
92+
val sqlite3mcZip = "sqlite3mc-$sqlite3mcVersion-sqlite-$sqliteVersion-wasm.zip"
93+
val sqliteDownload: Provider<Download> = tasks.register("sqliteDownload", Download::class.java) {
94+
src("https://github.com/utelle/SQLite3MultipleCiphers/releases/download/v$sqlite3mcVersion/$sqlite3mcZip")
7695
dest(layout.buildDirectory.dir("tmp"))
7796
onlyIfModified(true)
7897
}
79-
val sqliteUnzip = tasks.register("sqliteUnzip", Copy::class.java) {
98+
val sqliteUnzip: TaskProvider<Copy> = tasks.register("sqliteUnzip", Copy::class.java) {
8099
dependsOn(sqliteDownload)
81-
from(zipTree(layout.buildDirectory.dir("tmp/sqlite-wasm-$sqliteVersion.zip"))) {
82-
include("sqlite-wasm-$sqliteVersion/jswasm/**")
100+
from(zipTree(layout.buildDirectory.dir("tmp/$sqlite3mcZip"))) {
101+
include("sqlite3mc-wasm-$sqliteWasmVersion/jswasm/**")
83102
exclude("**/*worker1*") // We use our own worker
84103
eachFile {
85104
relativePath = RelativePath(true, *relativePath.segments.drop(2).toTypedArray())
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
// Karma configuration for Compose WASM UI tests
2+
//
3+
// Key issues this config solves:
4+
// 1. WASM file serving: webpack bundles JS but not .wasm files. The bundled code
5+
// tries to fetch .wasm relative to import.meta.url (a webpack temp path).
6+
// We add a middleware to redirect .wasm requests to the correct location.
7+
// 2. Timeouts: WasmGC module (~30 MB) compilation blocks the main thread during init.
8+
// 3. Source-map performance: skipping source-map-loader for large WASM glue files.
9+
10+
const path = require('path');
11+
const fs = require('fs');
12+
13+
function findRepoRoot(startDir) {
14+
let current = startDir;
15+
while (current !== path.dirname(current)) {
16+
if (
17+
fs.existsSync(path.join(current, 'settings.gradle.kts')) ||
18+
fs.existsSync(path.join(current, 'settings.gradle'))
19+
) {
20+
return current;
21+
}
22+
current = path.dirname(current);
23+
}
24+
return startDir;
25+
}
26+
27+
const repoRoot = findRepoRoot(process.cwd());
28+
const composeResourcesAbsolutePath = path.resolve(
29+
repoRoot,
30+
'app',
31+
'web',
32+
'build',
33+
'processedResources',
34+
'wasmJs',
35+
'main',
36+
'composeResources'
37+
).replace(/\\/g, '/');
38+
39+
// Middleware to serve .wasm files from the correct location
40+
// Webpack sets import.meta.url to a temp directory, so fetch('x.wasm') goes to the
41+
// wrong path. This middleware intercepts .wasm requests and serves them from kotlin/.
42+
function wasmFileMiddleware(config) {
43+
const basePath = config.basePath;
44+
const composeResourcesRoot = path.resolve(
45+
repoRoot,
46+
'app',
47+
'web',
48+
'build',
49+
'processedResources',
50+
'wasmJs',
51+
'main'
52+
);
53+
return function(req, res, next) {
54+
const cleanUrl = req.url.split('?')[0];
55+
56+
if (cleanUrl.includes('/composeResources/')) {
57+
const composeIndex = cleanUrl.indexOf('/composeResources/');
58+
const relativePath = cleanUrl.substring(composeIndex + 1);
59+
const resourcePath = path.join(composeResourcesRoot, relativePath);
60+
if (fs.existsSync(resourcePath)) {
61+
res.setHeader('Content-Type', 'application/octet-stream');
62+
fs.createReadStream(resourcePath).pipe(res);
63+
return;
64+
}
65+
}
66+
if (cleanUrl.endsWith('.wasm') && !cleanUrl.endsWith('sql-wasm.wasm')) {
67+
const filename = path.basename(cleanUrl);
68+
const wasmPath = path.join(basePath, 'kotlin', filename);
69+
if (fs.existsSync(wasmPath)) {
70+
res.setHeader('Content-Type', 'application/wasm');
71+
res.setHeader('Cross-Origin-Embedder-Policy', 'require-corp');
72+
res.setHeader('Cross-Origin-Opener-Policy', 'same-origin');
73+
fs.createReadStream(wasmPath).pipe(res);
74+
return;
75+
}
76+
}
77+
next();
78+
};
79+
}
80+
81+
// Register middleware as an inline Karma plugin
82+
config.plugins = config.plugins || [];
83+
config.plugins.push({
84+
'middleware:wasmFileServer': ['factory', function() {
85+
return wasmFileMiddleware(config);
86+
}]
87+
});
88+
89+
config.set({
90+
beforeMiddleware: ['wasmFileServer'],
91+
middleware: ['wasmFileServer'],
92+
proxies: Object.assign({}, config.proxies, {
93+
'/composeResources/': `/absolute/${composeResourcesAbsolutePath}/`
94+
}),
95+
96+
// WasmGC compilation blocks the main thread for ~15-18 min on the first run.
97+
// pingTimeout must exceed compilation time so the socket stays alive.
98+
pingTimeout: 1200000, // 20 min
99+
browserSocketTimeout: 1200000, // 20 min
100+
browserNoActivityTimeout: 1500000, // 25 min
101+
browserDisconnectTimeout: 60000, // 1 min
102+
browserDisconnectTolerance: 1,
103+
captureTimeout: 60000, // 1 min
104+
processKillTimeout: 5000, // 5 sec
105+
106+
client: {
107+
mocha: {
108+
timeout: 60000 // 1 min
109+
}
110+
},
111+
112+
customLaunchers: {
113+
ChromeHeadlessForWasm: {
114+
base: 'ChromeHeadless',
115+
flags: [
116+
'--no-sandbox',
117+
'--enable-features=SharedArrayBuffer'
118+
]
119+
}
120+
},
121+
browsers: ['ChromeHeadlessForWasm']
122+
});
123+
124+
// Speed up webpack compilation by skipping source-map-loader for large WASM glue files.
125+
if (config.webpack && config.webpack.module && config.webpack.module.rules) {
126+
config.webpack.module.rules.forEach(function(rule) {
127+
if (rule.use && rule.use.indexOf('source-map-loader') !== -1) {
128+
rule.exclude = [/skiko\.m?js$/, /\.uninstantiated\.m?js$/];
129+
}
130+
});
131+
}

app/web/src/wasmJsMain/kotlin/com/softartdev/notedelight/main.kt

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
@file:OptIn(ExperimentalComposeUiApi::class)
1+
@file:OptIn(ExperimentalComposeUiApi::class, ExperimentalWasmJsInterop::class)
22

33
package com.softartdev.notedelight
44

@@ -13,6 +13,7 @@ import kotlinx.browser.document
1313
import org.koin.core.context.startKoin
1414

1515
fun main() {
16+
if (isKarmaTestRunner()) return
1617
Logger.setTag(DEFAULT_APP_LOG_TAG)
1718
startKoin {
1819
kermitLogger()
@@ -22,3 +23,6 @@ fun main() {
2223
App()
2324
}
2425
}
26+
27+
@JsFun("() => typeof window !== 'undefined' && window.__karma__ != null")
28+
private external fun isKarmaTestRunner(): Boolean

app/web/src/wasmJsMain/resources/sqlite.worker.js

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,22 @@ let db = null;
1313
async function createDatabase() {
1414
if (sqlite3Available) {
1515
const sqlite3 = await sqlite3InitModule();
16+
const capi = sqlite3.capi;
1617

17-
// This is the key part for OPFS support
18-
// It instructs SQLite to use the OPFS VFS.
18+
// Try opfs VFS with encryption wrapper (sqlite3mc).
19+
// The regular "opfs" VFS stores files in the OPFS root directory,
20+
// which is required for export/import to work via navigator.storage.getDirectory().
21+
try {
22+
const rc = capi.sqlite3mc_vfs_create("opfs", 0);
23+
if (rc === 0) {
24+
db = new sqlite3.oo1.DB("file:database.db?vfs=multipleciphers-opfs", "c");
25+
return;
26+
}
27+
} catch (error) {
28+
// multipleciphers-opfs not available
29+
}
30+
31+
// Fallback: try opfs without encryption
1932
try {
2033
db = new sqlite3.oo1.DB("file:database.db?vfs=opfs", "c");
2134
return;

0 commit comments

Comments
 (0)