Skip to content

Thread confinement for Appender#583

Merged
staticlibs merged 1 commit intoduckdb:mainfrom
staticlibs:appender_owner_thread
Feb 28, 2026
Merged

Thread confinement for Appender#583
staticlibs merged 1 commit intoduckdb:mainfrom
staticlibs:appender_owner_thread

Conversation

@staticlibs
Copy link
Collaborator

Appender instances are not thread-safe. Only the close() method can be safely called from other threads concurrently with other operations.

append() and flush() operations operate on the same native buffer and cannot be called concurrently.

This PR implements a variant of a thread confinement for the Appender class intances. Only the thread that has created the Appender can call its methods (close() method still can be called from any thread). Method calls with throw SQLExeptions when called from other threads.

When it is necessary to use Appender from multiple threads, it is required to call unsafeBreakThreadConfinement() method first and use a Lock instance returned from it to synchronize the access to this Appender instance.

Example:

try (DuckDBAppender appender = connection.createAppender("tab1")) {
    Thread th = new Thread(() -> {
        // appender.flush(); // throws SQLException
        Lock appenderLock = appender.unsafeBreakThreadConfinement();
        appenderLock.lock();
        try {
            appender.flush();
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            appenderLock.unlock();
        }
    });
    th.start();
    th.join();
}

Testing: a concurrent test added that, without the patch, was crashing the JVM in about 1 of 10 runs.

Fixes: #582

Appender instances are not thread-safe. Only the `close()` method can
be safely called from other threads concurrently with other operations.

`append()` and `flush()` operations operate on the same native buffer
and cannot be called concurrently.

This PR implements a variant of a thread confinement for the Appender
class intances. Only the thread that has created the Appender can call
its methods (`close()` method still can be called from any thread).
Method calls with throw `SQLExeption`s when called from other threads.

When it is necessary to use Appender from multiple threads, it is
required to call `unsafeBreakThreadConfinement()` method first and use a
`Lock` instance returned from it to synchronize the access to this
Appender instance.

Example:

```java
try (DuckDBAppender appender = connection.createAppender("tab1")) {
    Thread th = new Thread(() -> {
        // appender.flush(); // throws SQLException
        Lock appenderLock = appender.unsafeBreakThreadConfinement();
        appenderLock.lock();
        try {
            appender.flush();
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            appenderLock.unlock();
        }
    });
    th.start();
    th.join();
}
```

Testing: a concurrent test added that, without the patch, was crashing
the JVM in about 1 of 10 runs.

Fixes: duckdb#582
@staticlibs staticlibs force-pushed the appender_owner_thread branch from 11a6876 to 9a82f69 Compare February 28, 2026 17:24
@staticlibs staticlibs merged commit d9686c3 into duckdb:main Feb 28, 2026
13 checks passed
@staticlibs staticlibs deleted the appender_owner_thread branch February 28, 2026 18:08
staticlibs added a commit to staticlibs/duckdb-java that referenced this pull request Feb 28, 2026
This is a backport of the PR duckdb#583 to `v1.5-variegata` stable branch.

Appender instances are not thread-safe. Only the `close()` method can
be safely called from other threads concurrently with other operations.

`append()` and `flush()` operations operate on the same native buffer
and cannot be called concurrently.

This PR implements a variant of a thread confinement for the Appender
class intances. Only the thread that has created the Appender can call
its methods (`close()` method still can be called from any thread).
Method calls with throw `SQLExeption`s when called from other threads.

When it is necessary to use Appender from multiple threads, it is
required to call `unsafeBreakThreadConfinement()` method first and use a
`Lock` instance returned from it to synchronize the access to this
Appender instance.

Example:

```java
try (DuckDBAppender appender = connection.createAppender("tab1")) {
    Thread th = new Thread(() -> {
        // appender.flush(); // throws SQLException
        Lock appenderLock = appender.unsafeBreakThreadConfinement();
        appenderLock.lock();
        try {
            appender.flush();
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            appenderLock.unlock();
        }
    });
    th.start();
    th.join();
}
```

Testing: a concurrent test added that, without the patch, was crashing
the JVM in about 1 of 10 runs.

Fixes: duckdb#582
staticlibs added a commit that referenced this pull request Feb 28, 2026
This is a backport of the PR #583 to `v1.5-variegata` stable branch.

Appender instances are not thread-safe. Only the `close()` method can
be safely called from other threads concurrently with other operations.

`append()` and `flush()` operations operate on the same native buffer
and cannot be called concurrently.

This PR implements a variant of a thread confinement for the Appender
class intances. Only the thread that has created the Appender can call
its methods (`close()` method still can be called from any thread).
Method calls with throw `SQLExeption`s when called from other threads.

When it is necessary to use Appender from multiple threads, it is
required to call `unsafeBreakThreadConfinement()` method first and use a
`Lock` instance returned from it to synchronize the access to this
Appender instance.

Example:

```java
try (DuckDBAppender appender = connection.createAppender("tab1")) {
    Thread th = new Thread(() -> {
        // appender.flush(); // throws SQLException
        Lock appenderLock = appender.unsafeBreakThreadConfinement();
        appenderLock.lock();
        try {
            appender.flush();
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            appenderLock.unlock();
        }
    });
    th.start();
    th.join();
}
```

Testing: a concurrent test added that, without the patch, was crashing
the JVM in about 1 of 10 runs.

Fixes: #582
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[1.4.4] siginfo: EXCEPTION_ACCESS_VIOLATION (0xc0000005), reading address 0x000001fd8f7d6ff0

1 participant