Skip to content

fix(docker): relocate libtcnative-1 to /srv/native-libs for correct FIPS removal#36071

Open
dsilvam wants to merge 3 commits into
mainfrom
fix/34212-fips-remove-libtcnative-on-fips-host
Open

fix(docker): relocate libtcnative-1 to /srv/native-libs for correct FIPS removal#36071
dsilvam wants to merge 3 commits into
mainfrom
fix/34212-fips-remove-libtcnative-on-fips-host

Conversation

@dsilvam

@dsilvam dsilvam commented Jun 9, 2026

Copy link
Copy Markdown
Member

Summary

Supersedes PR #35777 which was merged but proved insufficient in customer testing.

The previous fix (removing AprLifecycleListener from server.xml at runtime) did not prevent the crash because setenv.sh sets -Djava.library.path=/usr/lib/<arch>-linux-gnu/ and Tomcat auto-detects and loads libtcnative-1 from that path regardless of server.xml configuration.

Root cause confirmed: On a FIPS-enabled kernel (fips_enabled=1), OpenSSL 3.x requires the FIPS provider module (fips.so) before allowing any cryptographic operation. Ubuntu 24.04 does not ship fips.so. The first OpenSSL call — EVP_MD_get0_provider for random number generation inside libtcnative-1 — segfaults with SIGSEGV in libcrypto.so.3. This occurs even with SSLEngine=off and even without AprLifecycleListener in server.xml.

Changes

  • Dockerfile: During image build (as root), move libtcnative-1.so.0* from /usr/lib/<arch>-linux-gnu/ to /srv/native-libs/ (owned by the dotcms user) and leave symlinks in /usr/lib. Works on both amd64 and arm64 via $(uname -m).
  • 15-detect-fips-and-set-ssl-engine.sh: When FIPS detected, rm /srv/native-libs/libtcnative-1.so.0* (writable by dotcms user). The symlinks in /usr/lib become dangling so dlopen() cannot load the library regardless of java.library.path or server.xml. Includes a post-condition check that logs loudly on failure.

Non-FIPS containers are unaffected: symlinks resolve to the real files normally and the library loads, preserving performance.

Test plan

  • Build image and start container on a non-FIPS hostlibtcnative-1 must still load (AprLifecycleListener - Loaded Apache Tomcat Native library), CMS_SSL_ENGINE=on
  • Start container on a FIPS-enabled host (/proc/sys/crypto/fips_enabled=1, e.g. RHEL 8 with CIS Level 2) — container must start without SIGSEGV, log must show libtcnative-1 removed from /srv/native-libs
  • Verify ls /usr/lib/<arch>-linux-gnu/libtcnative-1.so.0* on FIPS container shows dangling symlinks
  • Verify ls /srv/native-libs/ on FIPS container is empty after startup
  • Verify arm64 image build succeeds (symlink path uses aarch64-linux-gnu)

Related

Closes #34212
Related: #34067, PR #34213, PR #35777, Freshdesk ticket #34489

dsilvam added 3 commits May 20, 2026 19:27
SSLEngine=off alone is insufficient to prevent the OpenSSL 3.x crash
on FIPS-enabled hosts. libtcnative-1 still loads libcrypto.so.3 and
calls OpenSSL for non-SSL operations (e.g. random number generation),
which triggers EVP_MD_get0_provider+0x4 SIGSEGV when the FIPS provider
(fips.so) is missing from Ubuntu 24.04.

The fix removes libtcnative-1.so at container startup when FIPS mode is
detected, before Tomcat starts. This prevents the native library from
loading OpenSSL entirely. Tomcat falls back to pure Java NIO/JSSE.
libtcnative-1 remains installed by default for non-FIPS environments.

Fixes #34212
Related: #34067, PR #34213
The previous approach (rm -f libtcnative-1.so) was silently a no-op:
the container runs as UID 65001 (non-root) and cannot unlink root-owned
files under /usr/lib. The || true masked the EACCES, making the fix
invisible in logs while the JVM continued to crash.

Instead, remove the AprLifecycleListener entry from server.xml at
container startup when FIPS mode is detected. server.xml lives under
/srv/dotserver/tomcat/conf/ which is owned by the dotcms user, so no
root privileges are needed. Without the AprLifecycleListener, Tomcat
never calls Library.initialize() and libtcnative-1 is never loaded,
preventing libcrypto.so.3 from being touched entirely.

Also adds a post-condition grep to fail loudly if the sed did not
remove the listener, so failures are visible in container logs.

Fixes #34212
Related: #34067, PR #34213
The server.xml AprLifecycleListener approach was insufficient because
setenv.sh sets java.library.path=/usr/lib/<arch>-linux-gnu/ and Tomcat
auto-detects and loads libtcnative-1 from there regardless of server.xml.

Root cause: On a FIPS-enabled kernel, OpenSSL 3.x requires fips.so before
allowing any crypto operation. Ubuntu 24.04 does not ship fips.so, so the
first OpenSSL call (EVP_MD_get0_provider) segfaults at the native level.

Fix:
- Dockerfile: move libtcnative-1.so.0* to /srv/native-libs/ (dotcms-owned)
  and leave symlinks in /usr/lib/<arch>-linux-gnu/. This is done as root
  during the image build — no runtime root access required.
- FIPS script: when FIPS detected, rm /srv/native-libs/libtcnative-1.so.0*
  (writable by dotcms user). The symlinks in /usr/lib become dangling so
  dlopen() cannot load the library regardless of java.library.path.

Non-FIPS containers are unaffected: symlinks resolve normally, the library
loads and provides its usual performance benefits.

Fixes #34212
Related: #34067, PR #34213

@wezell wezell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, hate this complexity but looks like it could work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

fix: Add FIPS mode detection and auto-disable APR SSL Engine

2 participants