Support building with address sanitizer#581
Open
staticlibs wants to merge 5 commits intoduckdb:mainfrom
Open
Conversation
This PR adds support for building JDBC driver with the [address sanitizer](https://github.com/google/sanitizers/wiki/addresssanitizer) enabled: ``` mkdir build cd build cmake .. -DENABLE_ADDRESS_SANITIZER=ON make ``` To run the resulting binaries it is required to have the JVM built with address sanitizer enabled too. Example of such build of [OpenJDK 17](https://wiki.openjdk.org/spaces/JDKUpdates/pages/70320316/JDK+17u): ``` git clone https://github.com/openjdk/jdk17u.git cd jdk17u CC=/usr/bin/clang CXX=/usr/bin/clang++ bash ./configure \ --enable-asan \ --with-toolchain-type=clang \ --disable-warnings-as-errors ``` Example of running a Java app: ```java import java.sql.*; class AsanTest { public static void main(String[] args) throws Exception { DriverManager.getConnection("jdbc:duckdb:").close(); } } ``` ``` javac AsanTest.java path/to/jdk17u/build/linux-x86_64-server-release/images/jdk/bin/java \ -cp .:path/to/duckdb-java/build/duckdb_jdbc_nolib.jar \ AsanTest ``` If we add the following deliberately erratic snippet somewhere inside the JDBC driver: ```c++ int *x = reinterpret_cast<int*>(malloc(10 * sizeof(int*))); free(x); std::cout << x[5] << std::endl; ``` Then the execution will abort with something like the following: ``` ================================================================= ==114481==ERROR: AddressSanitizer: heap-use-after-free on address 0x7bf5410abf14 at pc 0x7b83372124ac bp 0x7f85413fe070 sp 0x7f85413fe068 READ of size 4 at 0x7bf5410abf14 thread T1 #0 0x7b83372124ab in _duckdb_jdbc_startup(JNIEnv_*, _jclass*, _jbyteArray*, unsigned char, _jobject*) /home/alex/projects/duck/duckdb-java/src/jni/duckdb_java.cpp:79:18 #1 0x7b8337239880 in Java_org_duckdb_DuckDBNative_duckdb_1jdbc_1startup /home/alex/projects/duck/duckdb-java/src/jni/functions.cpp:9:30 duckdb#2 0x7b852b3445d9 (<unknown module>) 0x7bf5410abf14 is located 20 bytes inside of 80-byte region [0x7bf5410abf00,0x7bf5410abf50) freed by thread T1 here: #0 0x0000004a3fea in free (/home/alex/projects/external/jdk17u/build/linux-x86_64-server-release/images/jdk/bin/java+0x4a3fea) (BuildId: b4e8ae1f23ee7414ebabfce60a8c985b06eaf2d8) #1 0x7b8337212340 in _duckdb_jdbc_startup(JNIEnv_*, _jclass*, _jbyteArray*, unsigned char, _jobject*) /home/alex/projects/duck/duckdb-java/src/jni/duckdb_java.cpp:78:6 duckdb#2 0x7b8337239880 in Java_org_duckdb_DuckDBNative_duckdb_1jdbc_1startup /home/alex/projects/duck/duckdb-java/src/jni/functions.cpp:9:30 duckdb#3 0x7b852b3445d9 (<unknown module>) duckdb#4 0x7b852b34025f (<unknown module>) duckdb#5 0x7b852b34025f (<unknown module>) duckdb#6 0x7b852b3406e5 (<unknown module>) duckdb#7 0x7b852b34025f (<unknown module>) duckdb#8 0x7b852b34025f (<unknown module>) duckdb#9 0x7b852b337cc8 (<unknown module>) duckdb#10 0x7b853c8ff879 in JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) /home/alex/projects/external/jdk17u/src/hotspot/share/runtime/javaCalls.cpp:429:7 duckdb#11 0x7b853cab0c0b in jni_invoke_static(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, JavaThread*) /home/alex/projects/external/jdk17u/src/hotspot/share/prims/jni.cpp:889:3 duckdb#12 0x7b853cabb4ad in jni_CallStaticVoidMethod /home/alex/projects/external/jdk17u/src/hotspot/share/prims/jni.cpp:1718:3 duckdb#13 0x7f854215457a in JavaMain /home/alex/projects/external/jdk17u/src/java.base/share/native/libjli/java.c:547:5 duckdb#14 0x7f8542159368 in ThreadJavaMain /home/alex/projects/external/jdk17u/src/java.base/unix/native/libjli/java_md.c:651:29 duckdb#15 0x0000004a1baa in asan_thread_start(void*) (/home/alex/projects/external/jdk17u/build/linux-x86_64-server-release/images/jdk/bin/java+0x4a1baa) (BuildId: b4e8ae1f23ee7414ebabfce60a8c985b06eaf2d8) previously allocated by thread T1 here: #0 0x0000004a4288 in malloc (/home/alex/projects/external/jdk17u/build/linux-x86_64-server-release/images/jdk/bin/java+0x4a4288) (BuildId: b4e8ae1f23ee7414ebabfce60a8c985b06eaf2d8) #1 0x7b8337212335 in _duckdb_jdbc_startup(JNIEnv_*, _jclass*, _jbyteArray*, unsigned char, _jobject*) /home/alex/projects/duck/duckdb-java/src/jni/duckdb_java.cpp:77:40 duckdb#2 0x7b8337239880 in Java_org_duckdb_DuckDBNative_duckdb_1jdbc_1startup /home/alex/projects/duck/duckdb-java/src/jni/functions.cpp:9:30 duckdb#3 0x7b852b3445d9 (<unknown module>) duckdb#4 0x7b852b34025f (<unknown module>) duckdb#5 0x7b852b34025f (<unknown module>) duckdb#6 0x7b852b3406e5 (<unknown module>) duckdb#7 0x7b852b34025f (<unknown module>) duckdb#8 0x7b852b34025f (<unknown module>) duckdb#9 0x7b852b337cc8 (<unknown module>) duckdb#10 0x7b853c8ff879 in JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) /home/alex/projects/external/jdk17u/src/hotspot/share/runtime/javaCalls.cpp:429:7 duckdb#11 0x7b853cab0c0b in jni_invoke_static(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, JavaThread*) /home/alex/projects/external/jdk17u/src/hotspot/share/prims/jni.cpp:889:3 duckdb#12 0x7b853cabb4ad in jni_CallStaticVoidMethod /home/alex/projects/external/jdk17u/src/hotspot/share/prims/jni.cpp:1718:3 duckdb#13 0x7f854215457a in JavaMain /home/alex/projects/external/jdk17u/src/java.base/share/native/libjli/java.c:547:5 duckdb#14 0x7f8542159368 in ThreadJavaMain /home/alex/projects/external/jdk17u/src/java.base/unix/native/libjli/java_md.c:651:29 duckdb#15 0x0000004a1baa in asan_thread_start(void*) (/home/alex/projects/external/jdk17u/build/linux-x86_64-server-release/images/jdk/bin/java+0x4a1baa) (BuildId: b4e8ae1f23ee7414ebabfce60a8c985b06eaf2d8) Thread T1 created by T0 here: #0 0x0000004883a5 in pthread_create (/home/alex/projects/external/jdk17u/build/linux-x86_64-server-release/images/jdk/bin/java+0x4883a5) (BuildId: b4e8ae1f23ee7414ebabfce60a8c985b06eaf2d8) #1 0x7f8542159275 in CallJavaMainInNewThread /home/alex/projects/external/jdk17u/src/java.base/unix/native/libjli/java_md.c:670:9 duckdb#2 0x7f85421576f5 in ContinueInNewThread /home/alex/projects/external/jdk17u/src/java.base/share/native/libjli/java.c:2287:16 duckdb#3 0x7f854215262c in JLI_Launch /home/alex/projects/external/jdk17u/src/java.base/share/native/libjli/java.c:340:12 duckdb#4 0x0000004e8e68 in main /home/alex/projects/external/jdk17u/src/java.base/share/native/launcher/main.c:224:12 duckdb#5 0x7f8541e2f574 in __libc_start_call_main (/lib64/libc.so.6+0x3574) (BuildId: 92b5376d35bb29c098175948cf3e7cbcae3aeae1) duckdb#6 0x7f8541e2f627 in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x3627) (BuildId: 92b5376d35bb29c098175948cf3e7cbcae3aeae1) duckdb#7 0x000000400774 in _start (/home/alex/projects/external/jdk17u/build/linux-x86_64-server-release/images/jdk/bin/java+0x400774) (BuildId: b4e8ae1f23ee7414ebabfce60a8c985b06eaf2d8) SUMMARY: AddressSanitizer: heap-use-after-free /home/alex/projects/duck/duckdb-java/src/jni/duckdb_java.cpp:79:18 in _duckdb_jdbc_startup(JNIEnv_*, _jclass*, _jbyteArray*, unsigned char, _jobject*) Shadow bytes around the buggy address: 0x7bf5410abc80: 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 0x7bf5410abd00: 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 00 00 0x7bf5410abd80: 00 fa fa fa fa fa 00 00 00 00 00 00 00 00 00 00 0x7bf5410abe00: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 fa fa 0x7bf5410abe80: fa fa 00 00 00 00 00 00 00 00 00 fa fa fa fa fa =>0x7bf5410abf00: fd fd[fd]fd fd fd fd fd fd fd fa fa fa fa fa fa 0x7bf5410abf80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7bf5410ac000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7bf5410ac080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7bf5410ac100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x7bf5410ac180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==114481==ABORTING ``` Note that on successful run the JVM may print a lot of internal "leak" warnings, it may be necessary to prepare a [suppression file](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions) to prevent this.
staticlibs
added a commit
to staticlibs/duckdb-java
that referenced
this pull request
Feb 28, 2026
This is a backport of the PR duckdb#581 to `v1.5-variegata` stable branch. Appender instances are not thread-safe. Only the `close()` method can be safely called from other threads concurrently with other operations. `append()` and `flush()` operations operate on the same native buffer and cannot be called concurrently. This PR implements a variant of a thread confinement for the Appender class intances. Only the thread that has created the Appender can call its methods (`close()` method still can be called from any thread). Method calls with throw `SQLExeption`s when called from other threads. When it is necessary to use Appender from multiple threads, it is required to call `unsafeBreakThreadConfinement()` method first and use a `Lock` instance returned from it to synchronize the access to this Appender instance. Example: ```java try (DuckDBAppender appender = connection.createAppender("tab1")) { Thread th = new Thread(() -> { // appender.flush(); // throws SQLException Lock appenderLock = appender.unsafeBreakThreadConfinement(); appenderLock.lock(); try { appender.flush(); } catch (SQLException e) { e.printStackTrace(); } finally { appenderLock.unlock(); } }); th.start(); th.join(); } ``` Testing: a concurrent test added that, without the patch, was crashing the JVM in about 1 of 10 runs. Fixes: duckdb#582
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for building JDBC driver with the address sanitizer enabled:
To run the resulting binaries it is required to have the JVM built with address sanitizer enabled too. Example of such build of OpenJDK 17:
Example of running a Java app:
If we add the following deliberately erratic snippet somewhere inside the JDBC driver:
Then the execution will abort with something like the following:
Note that on successful run the JVM may print a lot of internal "leak" warnings, it may be necessary to prepare a suppression file to prevent this.