Skip to content

Thread confinement for Appender (1.5)#585

Merged
staticlibs merged 1 commit intoduckdb:v1.5-variegatafrom
staticlibs:appender_owner_thread_15
Feb 28, 2026
Merged

Thread confinement for Appender (1.5)#585
staticlibs merged 1 commit intoduckdb:v1.5-variegatafrom
staticlibs:appender_owner_thread_15

Conversation

@staticlibs
Copy link
Collaborator

This is a backport of the PR #583 to v1.5-variegata stable branch.

Appender instances are not thread-safe. Only the close() method can be safely called from other threads concurrently with other operations.

append() and flush() operations operate on the same native buffer and cannot be called concurrently.

This PR implements a variant of a thread confinement for the Appender class intances. Only the thread that has created the Appender can call its methods (close() method still can be called from any thread). Method calls with throw SQLExeptions when called from other threads.

When it is necessary to use Appender from multiple threads, it is required to call unsafeBreakThreadConfinement() method first and use a Lock instance returned from it to synchronize the access to this Appender instance.

Example:

try (DuckDBAppender appender = connection.createAppender("tab1")) {
    Thread th = new Thread(() -> {
        // appender.flush(); // throws SQLException
        Lock appenderLock = appender.unsafeBreakThreadConfinement();
        appenderLock.lock();
        try {
            appender.flush();
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            appenderLock.unlock();
        }
    });
    th.start();
    th.join();
}

Testing: a concurrent test added that, without the patch, was crashing the JVM in about 1 of 10 runs.

Fixes: #582

This is a backport of the PR duckdb#583 to `v1.5-variegata` stable branch.

Appender instances are not thread-safe. Only the `close()` method can
be safely called from other threads concurrently with other operations.

`append()` and `flush()` operations operate on the same native buffer
and cannot be called concurrently.

This PR implements a variant of a thread confinement for the Appender
class intances. Only the thread that has created the Appender can call
its methods (`close()` method still can be called from any thread).
Method calls with throw `SQLExeption`s when called from other threads.

When it is necessary to use Appender from multiple threads, it is
required to call `unsafeBreakThreadConfinement()` method first and use a
`Lock` instance returned from it to synchronize the access to this
Appender instance.

Example:

```java
try (DuckDBAppender appender = connection.createAppender("tab1")) {
    Thread th = new Thread(() -> {
        // appender.flush(); // throws SQLException
        Lock appenderLock = appender.unsafeBreakThreadConfinement();
        appenderLock.lock();
        try {
            appender.flush();
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            appenderLock.unlock();
        }
    });
    th.start();
    th.join();
}
```

Testing: a concurrent test added that, without the patch, was crashing
the JVM in about 1 of 10 runs.

Fixes: duckdb#582
@staticlibs staticlibs merged commit 6841a5b into duckdb:v1.5-variegata Feb 28, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant