Permanent worker starvation after maximum execution time worker restarts

[LauJosefsen]

What happened?

Hi there.

We had an issue where both replicas of a service running frankenphp 1.11.2 stopped serving requests untill container restarts.

It seems just before the malfunction, the containers had a large burst of PHP Fatal error: Maximum execution time of 30 seconds exceeded, which of course is not a problem in FrankenPHP but in our application. However we would expect these workers to become available again afterwards.

Looking at metrics, we can see in flight requests and busy workers spiking afterwards. However ready workers and total workers were permanent both at 8.

During the 5 hour outage there were no logs at all from the frankenphp server, indicating that workers did not get scheduled at all (we would expect maximum execution time logs if it was an application problem). We even have a caddy middleware for access logging, which also did not log anything during the outage.

I have attached pictures of the inflight requests and busy worker metrics during the outage. I don't know what else to look for, but I believe it cannot be (entirely) an application issue that workers don't even try to process requests - at least that is my conclussion. - it is however suspicious that both webserver replicas has the same issue at the exact same time, but if the maximum execution time trigger is correct, then it also occurred on both replicas.

Explanation of the timeline in the images:

• 04:00 start of webservers not handling requests
• 07:45 we rollout a restart of the replicas, the 2 new replicas are semi-healthy, some throughput, but issue partially persist.
• 07:55 we scale replicas to 10, still 2 of the 10 replicas are semi-healthy
• 08:15 the 10 replicas are restarted, and all replicas become healthy

Build Type

Docker (Debian Trixie)

Worker Mode

Yes

Operating System

GNU/Linux

CPU Architecture

x86_64

PHP configuration

php -r "phpinfo();"
phpinfo()
PHP Version => 8.5.3

System => Linux webserver-844bc8b968-2ws6d 6.8.0-60-generic #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 19:04:15 UTC 2025 x86_64
Build Date => Feb 13 2026 18:30:38
Build System => Linux - Docker
Build Provider => https://github.com/docker-library/php
Configure Command =>  './configure'  '--build=x86_64-linux-gnu' '--sysconfdir=/usr/local/etc' '--with-config-file-path=/usr/local/etc/php' '--with-config-file-scan-dir=/usr/local/etc/php/conf.d' '--enable-option-checking=fatal' '--with-mhash' '--with-pic' '--enable-mbstring' '--enable-mysqlnd' '--with-password-argon2' '--with-sodium=shared' '--with-pdo-sqlite=/usr' '--with-sqlite3=/usr' '--with-curl' '--with-iconv' '--with-openssl' '--with-readline' '--with-zlib' '--enable-phpdbg' '--enable-phpdbg-readline' '--with-pear' '--with-libdir=lib/x86_64-linux-gnu' '--enable-embed' '--enable-zts' '--disable-zend-signals' 'build_alias=x86_64-linux-gnu' 'PHP_UNAME=Linux - Docker' 'PHP_BUILD_PROVIDER=https://github.com/docker-library/php'
Server API => Command Line Interface
Virtual Directory Support => enabled
Configuration File (php.ini) Path => /usr/local/etc/php
Loaded Configuration File => (none)
Scan this dir for additional .ini files => /usr/local/etc/php/conf.d
Additional .ini files parsed => /usr/local/etc/php/conf.d/docker-php-ext-apcu.ini,
/usr/local/etc/php/conf.d/docker-php-ext-bcmath.ini,
/usr/local/etc/php/conf.d/docker-php-ext-exif.ini,
/usr/local/etc/php/conf.d/docker-php-ext-gd.ini,
/usr/local/etc/php/conf.d/docker-php-ext-gettext.ini,
/usr/local/etc/php/conf.d/docker-php-ext-gmp.ini,
/usr/local/etc/php/conf.d/docker-php-ext-intl.ini,
/usr/local/etc/php/conf.d/docker-php-ext-opentelemetry.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pcntl.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pdo_mysql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pdo_pgsql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-protobuf.ini,
/usr/local/etc/php/conf.d/docker-php-ext-rdkafka.ini,
/usr/local/etc/php/conf.d/docker-php-ext-redis.ini,
/usr/local/etc/php/conf.d/docker-php-ext-soap.ini,
/usr/local/etc/php/conf.d/docker-php-ext-sodium.ini,
/usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini,
/usr/local/etc/php/conf.d/docker-php-ext-yaml.ini,
/usr/local/etc/php/conf.d/docker-php-ext-zip.ini,
/usr/local/etc/php/conf.d/php.ini

PHP API => 20250925
PHP Extension => 20250925
Zend Extension => 420250925
Zend Extension Build => API420250925,TS
PHP Extension Build => API20250925,TS
PHP Integer Size => 64 bits
Debug Build => no
Thread Safety => enabled
Thread API => POSIX Threads
Zend Signal Handling => disabled
Zend Memory Manager => enabled
Zend Multibyte Support => provided by mbstring
Zend Max Execution Timers => enabled
IPv6 Support => enabled
DTrace Support => disabled

Registered PHP Streams => https, ftps, compress.zlib, php, file, glob, data, http, ftp, phar, zip
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, tls, tlsv1.0, tlsv1.1, tlsv1.2, tlsv1.3
Registered Stream Filters => zlib.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, convert.*, consumed, dechunk

This program makes use of the Zend Scripting Language Engine:
Zend Engine v4.5.3, Copyright (c) Zend Technologies
    with Xdebug v3.5.0, Copyright (c) 2002-2025, by Derick Rethans
    with Zend OPcache v8.5.3, Copyright (c), by Zend Technologies


 _______________________________________________________________________


Configuration

apcu

APCu Support => Enabled
Version => 5.1.28
APCu Debugging => Disabled
MMAP Support => Enabled
MMAP File Mask =>  
Serialization Support => php
Build Date => Feb 15 2026 19:34:41

Directive => Local Value => Master Value
apc.coredump_unmap => Off => Off
apc.enable_cli => On => On
apc.enabled => On => On
apc.entries_hint => 0 => 0
apc.gc_ttl => 3600 => 3600
apc.mmap_file_mask => no value => no value
apc.mmap_hugepage_size => 0 => 0
apc.preload_path => no value => no value
apc.serializer => php => php
apc.shm_size => 32M => 32M
apc.slam_defense => Off => Off
apc.smart => 0 => 0
apc.ttl => 0 => 0
apc.use_request_time => Off => Off

bcmath

BCMath support => enabled

Directive => Local Value => Master Value
bcmath.scale => 0 => 0

Core

PHP Version => 8.5.3

Directive => Local Value => Master Value
allow_url_fopen => On => On
allow_url_include => Off => Off
arg_separator.input => & => &
arg_separator.output => & => &
auto_append_file => no value => no value
auto_globals_jit => On => On
auto_prepend_file => no value => no value
browscap => no value => no value
default_charset => UTF-8 => UTF-8
default_mimetype => text/html => text/html
disable_functions => no value => no value
display_errors => Off => Off
display_startup_errors => Off => Off
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => On => On
enable_post_data_reading => On => On
error_append_string => no value => no value
error_log => no value => no value
error_log_mode => 0644 => 0644
error_prepend_string => no value => no value
error_reporting => 22527 => 22527
expose_php => Off => Off
extension_dir => /usr/local/lib/php/extensions/no-debug-zts-20250925 => /usr/local/lib/php/extensions/no-debug-zts-20250925
fatal_error_backtraces => On => On
fiber.stack_size => no value => no value
file_uploads => On => On
hard_timeout => 2 => 2
highlight.comment => <span style="color: #FF8000">#FF8000</span> => <span style="color: #FF8000">#FF8000</span>
highlight.default => <span style="color: #0000BB">#0000BB</span> => <span style="color: #0000BB">#0000BB</span>
highlight.html => <span style="color: #000000">#000000</span> => <span style="color: #000000">#000000</span>
highlight.keyword => <span style="color: #007700">#007700</span> => <span style="color: #007700">#007700</span>
highlight.string => <span style="color: #DD0000">#DD0000</span> => <span style="color: #DD0000">#DD0000</span>
html_errors => Off => Off
ignore_repeated_errors => Off => Off
ignore_repeated_source => Off => Off
ignore_user_abort => Off => Off
implicit_flush => On => On
include_path => .:/usr/local/lib/php => .:/usr/local/lib/php
input_encoding => no value => no value
internal_encoding => no value => no value
log_errors => On => On
mail.add_x_header => Off => Off
mail.cr_lf_mode => crlf => crlf
mail.force_extra_parameters => no value => no value
mail.log => no value => no value
mail.mixed_lf_and_crlf => Off => Off
max_execution_time => 0 => 0
max_file_uploads => 40 => 40
max_input_nesting_level => 64 => 64
max_input_time => -1 => -1
max_input_vars => 1000 => 1000
max_memory_limit => -1 => -1
max_multipart_body_parts => -1 => -1
memory_limit => 128M => 128M
open_basedir => no value => no value
output_buffering => 0 => 0
output_encoding => no value => no value
output_handler => no value => no value
post_max_size => 100M => 100M
precision => 14 => 14
realpath_cache_size => 4096K => 4096K
realpath_cache_ttl => 120 => 120
register_argc_argv => Off => Off
report_memleaks => On => On
report_zend_debug => Off => Off
request_order => no value => no value
sendmail_from => no value => no value
sendmail_path => /usr/sbin/sendmail -t -i => /usr/sbin/sendmail -t -i
serialize_precision => -1 => -1
short_open_tag => Off => Off
SMTP => localhost => localhost
smtp_port => 25 => 25
sys_temp_dir => no value => no value
syslog.facility => LOG_USER => LOG_USER
syslog.filter => no-ctrl => no-ctrl
syslog.ident => php => php
unserialize_callback_func => no value => no value
upload_max_filesize => 100M => 100M
upload_tmp_dir => no value => no value
user_dir => no value => no value
user_ini.cache_ttl => 300 => 300
user_ini.filename => .user.ini => .user.ini
variables_order => EGPCS => EGPCS
xmlrpc_error_number => 0 => 0
xmlrpc_errors => Off => Off
zend.assertions => 1 => 1
zend.detect_unicode => On => On
zend.enable_gc => On => On
zend.exception_ignore_args => On => On
zend.exception_string_param_max_len => 15 => 15
zend.max_allowed_stack_size => 0 => 0
zend.multibyte => Off => Off
zend.reserved_stack_size => 0 => 0
zend.script_encoding => no value => no value

ctype

ctype functions => enabled

curl

cURL support => enabled
cURL Information => 8.14.1
Age => 11
Features
AsynchDNS => Yes
CharConv => No
Debug => No
GSS-Negotiate => No
IDN => Yes
IPv6 => Yes
krb4 => No
Largefile => Yes
libz => Yes
NTLM => Yes
NTLMWB => No
SPNEGO => Yes
SSL => Yes
SSPI => No
TLS-SRP => Yes
HTTP2 => Yes
GSSAPI => Yes
KERBEROS5 => Yes
UNIX_SOCKETS => Yes
PSL => Yes
HTTPS_PROXY => Yes
MULTI_SSL => No
BROTLI => Yes
ALTSVC => Yes
HTTP3 => Yes
UNICODE => No
ZSTD => Yes
HSTS => Yes
GSASL => No
Protocols => dict, file, ftp, ftps, gopher, gophers, http, https, imap, imaps, ldap, ldaps, mqtt, pop3, pop3s, rtmp, rtmpe, rtmps, rtmpt, rtmpte, rtmpts, rtsp, scp, sftp, smb, smbs, smtp, smtps, telnet, tftp, ws, wss
Host => x86_64-pc-linux-gnu
SSL Version => OpenSSL/3.5.4
ZLib Version => 1.3.1
libSSH Version => libssh2/1.11.1

Directive => Local Value => Master Value
curl.cainfo => no value => no value

date

date/time support => enabled
timelib version => 2022.15
"Olson" Timezone Database Version => 2025.3
Timezone Database => internal
Default timezone => UTC

Directive => Local Value => Master Value
date.default_latitude => 31.7667 => 31.7667
date.default_longitude => 35.2333 => 35.2333
date.sunrise_zenith => 90.833333 => 90.833333
date.sunset_zenith => 90.833333 => 90.833333
date.timezone => UTC => UTC

dom

DOM/XML => enabled
DOM/XML API Version => 20031129
libxml Version => 2.9.14
HTML Support => enabled
XPath Support => enabled
XPointer Support => enabled
Schema Support => enabled
RelaxNG Support => enabled

exif

EXIF Support => enabled
Supported EXIF Version => 0220
Supported filetypes => JPEG, TIFF
Multibyte decoding support using mbstring => enabled
Extended EXIF tag formats => Canon, Casio, Fujifilm, Nikon, Olympus, Samsung, Panasonic, DJI, Sony, Pentax, Minolta, Sigma, Foveon, Kyocera, Ricoh, AGFA, Epson

Directive => Local Value => Master Value
exif.decode_jis_intel => JIS => JIS
exif.decode_jis_motorola => JIS => JIS
exif.decode_unicode_intel => UCS-2LE => UCS-2LE
exif.decode_unicode_motorola => UCS-2BE => UCS-2BE
exif.encode_jis => no value => no value
exif.encode_unicode => ISO-8859-15 => ISO-8859-15

fileinfo

fileinfo support => enabled
libmagic => 5.46

filter

Input Validation and Filtering => enabled

Directive => Local Value => Master Value
filter.default => unsafe_raw => unsafe_raw
filter.default_flags => no value => no value

gd

GD Support => enabled
GD Version => bundled (2.1.0 compatible)
GIF Read Support => enabled
GIF Create Support => enabled
PNG Support => enabled
libPNG Version => 1.6.48
WBMP Support => enabled
XBM Support => enabled
BMP Support => enabled
TGA Read Support => enabled

Directive => Local Value => Master Value
gd.jpeg_ignore_warning => On => On

gettext

GetText Support => enabled

gmp

gmp support => enabled
GMP version => 6.3.0

hash

hash support => enabled
Hashing Engines => md2 md4 md5 sha1 sha224 sha256 sha384 sha512/224 sha512/256 sha512 sha3-224 sha3-256 sha3-384 sha3-512 ripemd128 ripemd160 ripemd256 ripemd320 whirlpool tiger128,3 tiger160,3 tiger192,3 tiger128,4 tiger160,4 tiger192,4 snefru snefru256 gost gost-crypto adler32 crc32 crc32b crc32c fnv132 fnv1a32 fnv164 fnv1a64 joaat murmur3a murmur3c murmur3f xxh32 xxh64 xxh3 xxh128 haval128,3 haval160,3 haval192,3 haval224,3 haval256,3 haval128,4 haval160,4 haval192,4 haval224,4 haval256,4 haval128,5 haval160,5 haval192,5 haval224,5 haval256,5 

MHASH support => Enabled
MHASH API Version => Emulated Support

iconv

iconv support => enabled
iconv implementation => glibc
iconv library version => 2.41

Directive => Local Value => Master Value
iconv.input_encoding => no value => no value
iconv.internal_encoding => no value => no value
iconv.output_encoding => no value => no value

intl

Internationalization support => enabled
ICU version => 76.1
ICU Data version => 76.1
ICU TZData version => 2024b
ICU Unicode version => 16.0

Directive => Local Value => Master Value
intl.default_locale => no value => no value
intl.error_level => 0 => 0
intl.use_exceptions => Off => Off

json

json support => enabled

lexbor

Lexbor support => active
Lexbor version => 2.5.0

libxml

libXML support => active
libXML Compiled Version => 2.9.14
libXML Loaded Version => 20914
libXML streams => enabled

mbstring

Multibyte Support => enabled
Multibyte string engine => libmbfl
HTTP input encoding translation => disabled
libmbfl version => 1.3.2

mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.

Multibyte (japanese) regex support => enabled
Multibyte regex (oniguruma) version => 6.9.9

Directive => Local Value => Master Value
mbstring.detect_order => no value => no value
mbstring.encoding_translation => Off => Off
mbstring.http_input => no value => no value
mbstring.http_output => no value => no value
mbstring.http_output_conv_mimetypes => ^(text/|application/xhtml\+xml) => ^(text/|application/xhtml\+xml)
mbstring.internal_encoding => no value => no value
mbstring.language => neutral => neutral
mbstring.regex_retry_limit => 1000000 => 1000000
mbstring.regex_stack_limit => 100000 => 100000
mbstring.strict_detection => Off => Off
mbstring.substitute_character => no value => no value

mysqlnd

mysqlnd => enabled
Version => mysqlnd 8.5.3
Compression => supported
core SSL => supported
extended SSL => supported
Command buffer size => 4096
Read buffer size => 32768
Read timeout => 86400
Collecting statistics => Yes
Collecting memory statistics => No
Tracing => n/a
Loaded plugins => mysqlnd,debug_trace,auth_plugin_mysql_native_password,auth_plugin_mysql_clear_password,auth_plugin_caching_sha2_password,auth_plugin_sha256_password
API Extensions => pdo_mysql

openssl

OpenSSL support => enabled
OpenSSL Library Version => OpenSSL 3.5.4 30 Sep 2025
OpenSSL Header Version => OpenSSL 3.5.4 30 Sep 2025
Openssl default config => /usr/lib/ssl/openssl.cnf

Directive => Local Value => Master Value
openssl.cafile => no value => no value
openssl.capath => no value => no value
openssl.libctx => custom => custom

opentelemetry

opentelemetry hooks => enabled
extension version => 1.2.1

Directive => Local Value => Master Value
opentelemetry.allow_stack_extension => Off => Off
opentelemetry.attr_hooks_enabled => On => On
opentelemetry.attr_post_handler_function => OpenTelemetry\API\Instrumentation\WithSpanHandler::post => OpenTelemetry\API\Instrumentation\WithSpanHandler::post
opentelemetry.attr_pre_handler_function => OpenTelemetry\API\Instrumentation\WithSpanHandler::pre => OpenTelemetry\API\Instrumentation\WithSpanHandler::pre
opentelemetry.conflicts => no value => no value
opentelemetry.display_warnings => Off => Off
opentelemetry.validate_hook_functions => On => On

pcntl

pcntl support => enabled

pcre

PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.44 2024-06-07
PCRE Unicode Version => 15.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)

Directive => Local Value => Master Value
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => On => On
pcre.recursion_limit => 100000 => 100000

PDO

PDO support => enabled
PDO drivers => sqlite, mysql, pgsql

pdo_mysql

PDO Driver for MySQL => enabled
Client API version => mysqlnd 8.5.3

Directive => Local Value => Master Value
pdo_mysql.default_socket => no value => no value

pdo_pgsql

PDO Driver for PostgreSQL => enabled
PostgreSQL(libpq) Version => 17.8

pdo_sqlite

PDO Driver for SQLite 3.x => enabled
SQLite Library => 3.46.1

Phar

Phar: PHP Archive support => enabled
Phar API version => 1.1.1
Phar-based phar archives => enabled
Tar-based phar archives => enabled
ZIP-based phar archives => enabled
gzip compression => enabled
bzip2 compression => disabled (install ext/bz2)
Native OpenSSL support => enabled


Phar based on pear/PHP_Archive, original concept by Davey Shafik.
Phar fully realized by Gregory Beaver and Marcus Boerger.
Portions of tar implementation Copyright (c) 2003-2009 Tim Kientzle.
Directive => Local Value => Master Value
phar.cache_list => no value => no value
phar.readonly => On => On
phar.require_hash => On => On

posix

POSIX support => enabled

protobuf

Version => 4.33.5

Directive => Local Value => Master Value
protobuf.keep_descriptor_pool_after_request => 0 => 0

random

Version => 8.5.3

rdkafka

rdkafka support => enabled
version => 6.0.5
build date => Feb 15 2026 19:34:23
librdkafka version (runtime) => 2.13.0
librdkafka version (build) => 2.13.0.255

readline

Readline Support => enabled
Readline library => 8.2

Directive => Local Value => Master Value
cli.pager => no value => no value
cli.prompt => \b \>  => \b \> 

redis

Redis Support => enabled
Redis Version => 6.3.0
Redis Sentinel Version => 1.0
Available serializers => php, json

Directive => Local Value => Master Value
redis.arrays.algorithm => no value => no value
redis.arrays.auth => no value => no value
redis.arrays.autorehash => 0 => 0
redis.arrays.connecttimeout => 0 => 0
redis.arrays.consistent => 0 => 0
redis.arrays.distributor => no value => no value
redis.arrays.functions => no value => no value
redis.arrays.hosts => no value => no value
redis.arrays.index => 0 => 0
redis.arrays.lazyconnect => 0 => 0
redis.arrays.names => no value => no value
redis.arrays.pconnect => 0 => 0
redis.arrays.previous => no value => no value
redis.arrays.readtimeout => 0 => 0
redis.arrays.retryinterval => 0 => 0
redis.clusters.auth => no value => no value
redis.clusters.cache_slots => 1 => 1
redis.clusters.persistent => 0 => 0
redis.clusters.read_timeout => 0 => 0
redis.clusters.seeds => no value => no value
redis.clusters.timeout => 0 => 0
redis.pconnect.connection_limit => 0 => 0
redis.pconnect.echo_check_liveness => 1 => 1
redis.pconnect.pool_detect_dirty => 0 => 0
redis.pconnect.pool_pattern => no value => no value
redis.pconnect.pool_poll_timeout => 0 => 0
redis.pconnect.pooling_enabled => 1 => 1
redis.session.compression => none => none
redis.session.compression_level => 3 => 3
redis.session.early_refresh => 0 => 0
redis.session.lock_expire => 0 => 0
redis.session.lock_failure_readonly => 0 => 0
redis.session.lock_retries => 100 => 100
redis.session.lock_wait_time => 20000 => 20000
redis.session.locking_enabled => 0 => 0

Reflection

Reflection => enabled

session

Session Support => enabled
Registered save handlers => files user redis rediscluster 
Registered serializer handlers => php_serialize php php_binary 

Directive => Local Value => Master Value
session.auto_start => Off => Off
session.cache_expire => 180 => 180
session.cache_limiter => nocache => nocache
session.cookie_domain => no value => no value
session.cookie_httponly => Off => Off
session.cookie_lifetime => 0 => 0
session.cookie_partitioned => Off => Off
session.cookie_path => / => /
session.cookie_samesite => no value => no value
session.cookie_secure => Off => Off
session.gc_divisor => 100 => 100
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.lazy_write => On => On
session.name => PHPSESSID => PHPSESSID
session.referer_check => no value => no value
session.save_handler => files => files
session.save_path => no value => no value
session.serialize_handler => php => php
session.sid_bits_per_character => 4 => 4
session.sid_length => 32 => 32
session.upload_progress.cleanup => On => On
session.upload_progress.enabled => On => On
session.upload_progress.freq => 1% => 1%
session.upload_progress.min_freq => 1 => 1
session.upload_progress.name => PHP_SESSION_UPLOAD_PROGRESS => PHP_SESSION_UPLOAD_PROGRESS
session.upload_progress.prefix => upload_progress_ => upload_progress_
session.use_cookies => On => On
session.use_only_cookies => On => On
session.use_strict_mode => Off => Off
session.use_trans_sid => Off => Off

SimpleXML

SimpleXML support => enabled
Schema support => enabled

soap

Soap Client => enabled
Soap Server => enabled

Directive => Local Value => Master Value
soap.wsdl_cache => 1 => 1
soap.wsdl_cache_dir => /tmp => /tmp
soap.wsdl_cache_enabled => On => On
soap.wsdl_cache_limit => 5 => 5
soap.wsdl_cache_ttl => 86400 => 86400

sodium

sodium support => enabled
libsodium headers version => 1.0.18
libsodium library version => 1.0.18

SPL

SPL support => enabled
Interfaces => OuterIterator, RecursiveIterator, SeekableIterator, SplObserver, SplSubject
Classes => AppendIterator, ArrayIterator, ArrayObject, BadFunctionCallException, BadMethodCallException, CachingIterator, CallbackFilterIterator, DirectoryIterator, DomainException, EmptyIterator, FilesystemIterator, FilterIterator, GlobIterator, InfiniteIterator, InvalidArgumentException, IteratorIterator, LengthException, LimitIterator, LogicException, MultipleIterator, NoRewindIterator, OutOfBoundsException, OutOfRangeException, OverflowException, ParentIterator, RangeException, RecursiveArrayIterator, RecursiveCachingIterator, RecursiveCallbackFilterIterator, RecursiveDirectoryIterator, RecursiveFilterIterator, RecursiveIteratorIterator, RecursiveRegexIterator, RecursiveTreeIterator, RegexIterator, RuntimeException, SplDoublyLinkedList, SplFileInfo, SplFileObject, SplFixedArray, SplHeap, SplMinHeap, SplMaxHeap, SplObjectStorage, SplPriorityQueue, SplQueue, SplStack, SplTempFileObject, UnderflowException, UnexpectedValueException

sqlite3

SQLite3 support => enabled
SQLite Library => 3.46.1

Directive => Local Value => Master Value
sqlite3.defensive => On => On
sqlite3.extension_dir => no value => no value

standard

Dynamic Library Support => enabled
Path to sendmail => /usr/sbin/sendmail -t -i

Directive => Local Value => Master Value
assert.active => On => On
assert.bail => Off => Off
assert.callback => no value => no value
assert.exception => On => On
assert.warning => On => On
auto_detect_line_endings => Off => Off
default_socket_timeout => 60 => 60
from => no value => no value
session.trans_sid_hosts => no value => no value
session.trans_sid_tags => a=href,area=href,frame=src,form= => a=href,area=href,frame=src,form=
unserialize_max_depth => 4096 => 4096
url_rewriter.hosts => no value => no value
url_rewriter.tags => form= => form=
user_agent => no value => no value

tokenizer

Tokenizer Support => enabled

uri

URI support => active
uriparser bundled version => 1.0.0

xdebug

__   __   _      _                 
\ \ / /  | |    | |                
 \ V / __| | ___| |__  _   _  __ _ 
  > < / _` |/ _ \ '_ \| | | |/ _` |
 / . \ (_| |  __/ |_) | |_| | (_| |
/_/ \_\__,_|\___|_.__/ \__,_|\__, |
                              __/ |
                             |___/ 

Version => 3.5.0
Support Xdebug on Patreon, GitHub, or as a business: https://xdebug.org/support

             Enabled Features (through 'xdebug.mode' setting)             
Feature => Enabled/Disabled
Development Helpers => ✘ disabled
Coverage => ✘ disabled
GC Stats => ✘ disabled
Profiler => ✘ disabled
Step Debugger => ✘ disabled
Tracing => ✘ disabled

                            Optional Features                            
Compressed File Support => yes (gzip)
Clock Source => clock_gettime
TSC Clock Source => available
'xdebug://gateway' pseudo-host support => yes
'xdebug://nameserver' pseudo-host support => yes
Systemd Private Temp Directory => not enabled

Directive => Local Value => Master Value
xdebug.auto_trace => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.cli_color => 0 => 0
xdebug.client_discovery_header => HTTP_X_FORWARDED_FOR,REMOTE_ADDR => HTTP_X_FORWARDED_FOR,REMOTE_ADDR
xdebug.client_host => localhost => localhost
xdebug.client_port => 9003 => 9003
xdebug.cloud_id => no value => no value
xdebug.collect_assignments => Off => Off
xdebug.collect_includes => (setting removed in Xdebug 3) => (setting removed in Xdebug 3)
xdebug.collect_params => On => On
xdebug.collect_return => Off => Off
xdebug.collect_vars => (setting removed in Xdebug 3) => (setting removed in Xdebug 3)
xdebug.connect_timeout_ms => 200 => 200
xdebug.control_socket => time: 25ms => time: 25ms
xdebug.coverage_enable => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.default_enable => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.discover_client_host => Off => Off
xdebug.dump.COOKIE => no value => no value
xdebug.dump.ENV => no value => no value
xdebug.dump.FILES => no value => no value
xdebug.dump.GET => no value => no value
xdebug.dump.POST => no value => no value
xdebug.dump.REQUEST => no value => no value
xdebug.dump.SERVER => no value => no value
xdebug.dump.SESSION => no value => no value
xdebug.dump_globals => On => On
xdebug.dump_once => On => On
xdebug.dump_undefined => Off => Off
xdebug.file_link_format => no value => no value
xdebug.filename_format => no value => no value
xdebug.force_display_errors => Off => Off
xdebug.force_error_reporting => 0 => 0
xdebug.gc_stats_enable => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.gc_stats_output_dir => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.gc_stats_output_name => gcstats.%p => gcstats.%p
xdebug.halt_level => 0 => 0
xdebug.idekey => no value => no value
xdebug.log => no value => no value
xdebug.log_level => 7 => 7
xdebug.max_nesting_level => 512 => 512
xdebug.max_stack_frames => -1 => -1
xdebug.mode => no value => no value
xdebug.output_dir => /tmp => /tmp
xdebug.overload_var_dump => (setting removed in Xdebug 3) => (setting removed in Xdebug 3)
xdebug.path_mapping => 0 => 0
xdebug.profiler_append => Off => Off
xdebug.profiler_enable => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.profiler_enable_trigger => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.profiler_enable_trigger_value => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.profiler_output_dir => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.profiler_output_name => cachegrind.out.%p => cachegrind.out.%p
xdebug.remote_autostart => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_connect_back => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_enable => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_host => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_log => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_log_level => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_mode => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_port => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.remote_timeout => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.scream => Off => Off
xdebug.show_error_trace => Off => Off
xdebug.show_exception_trace => Off => Off
xdebug.show_local_vars => Off => Off
xdebug.show_mem_delta => (setting removed in Xdebug 3) => (setting removed in Xdebug 3)
xdebug.start_upon_error => default => default
xdebug.start_with_request => default => default
xdebug.trace_enable_trigger => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.trace_enable_trigger_value => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.trace_format => 0 => 0
xdebug.trace_options => 0 => 0
xdebug.trace_output_dir => (setting renamed in Xdebug 3) => (setting renamed in Xdebug 3)
xdebug.trace_output_name => trace.%c => trace.%c
xdebug.trigger_value => no value => no value
xdebug.use_compression => 1 => 1
xdebug.var_display_max_children => 128 => 128
xdebug.var_display_max_data => 512 => 512
xdebug.var_display_max_depth => 3 => 3

xml

XML Support => active
XML Namespace Support => active
libxml2 Version => 2.9.14

xmlreader

XMLReader => enabled

xmlwriter

XMLWriter => enabled

yaml

LibYAML Support => enabled
Module Version => 2.3.0
LibYAML Version => 0.2.5

Directive => Local Value => Master Value
yaml.decode_binary => 0 => 0
yaml.decode_php => 0 => 0
yaml.decode_timestamp => 0 => 0
yaml.output_canonical => 0 => 0
yaml.output_indent => 2 => 2
yaml.output_width => 80 => 80

Zend OPcache

Opcode Caching => Up and Running
Optimization => Enabled
SHM Cache => Enabled
File Cache => Disabled
JIT => Disabled
Startup => OK
Shared memory model => mmap
Cache hits => 0
Cache misses => 0
Used memory => 9167944
Free memory => 125049784
Wasted memory => 0
Interned Strings Used memory => 2600352
Interned Strings Free memory => 5788256
Cached scripts => 0
Cached keys => 0
Max keys => 16229
OOM restarts => 0
Hash keys restarts => 0
Manual restarts => 0
Start time => 2026-02-19T09:38:10+0000
Last restart time => none
Last force restart time => none

Directive => Local Value => Master Value
opcache.blacklist_filename => no value => no value
opcache.dups_fix => Off => Off
opcache.enable => On => On
opcache.enable_cli => On => On
opcache.enable_file_override => Off => Off
opcache.error_log => no value => no value
opcache.file_cache => no value => no value
opcache.file_cache_consistency_checks => On => On
opcache.file_cache_only => Off => Off
opcache.file_cache_read_only => Off => Off
opcache.file_update_protection => 2 => 2
opcache.force_restart_timeout => 180 => 180
opcache.huge_code_pages => Off => Off
opcache.interned_strings_buffer => 8 => 8
opcache.jit => disable => disable
opcache.jit_bisect_limit => 0 => 0
opcache.jit_blacklist_root_trace => 16 => 16
opcache.jit_blacklist_side_trace => 8 => 8
opcache.jit_buffer_size => 64M => 64M
opcache.jit_debug => 0 => 0
opcache.jit_hot_func => 127 => 127
opcache.jit_hot_loop => 61 => 61
opcache.jit_hot_return => 8 => 8
opcache.jit_hot_side_exit => 8 => 8
opcache.jit_max_exit_counters => 8192 => 8192
opcache.jit_max_loop_unrolls => 8 => 8
opcache.jit_max_polymorphic_calls => 2 => 2
opcache.jit_max_recursive_calls => 2 => 2
opcache.jit_max_recursive_returns => 2 => 2
opcache.jit_max_root_traces => 1024 => 1024
opcache.jit_max_side_traces => 128 => 128
opcache.jit_max_trace_length => 1024 => 1024
opcache.jit_prof_threshold => 0.005 => 0.005
opcache.lockfile_path => /tmp => /tmp
opcache.log_verbosity_level => 1 => 1
opcache.max_accelerated_files => 10000 => 10000
opcache.max_file_size => 0 => 0
opcache.max_wasted_percentage => 5 => 5
opcache.memory_consumption => 128 => 128
opcache.opt_debug_level => 0 => 0
opcache.optimization_level => 0x7FFEBFFF => 0x7FFEBFFF
opcache.preferred_memory_model => no value => no value
opcache.preload => no value => no value
opcache.preload_user => no value => no value
opcache.protect_memory => Off => Off
opcache.record_warnings => Off => Off
opcache.restrict_api => no value => no value
opcache.revalidate_freq => 2 => 2
opcache.revalidate_path => Off => Off
opcache.save_comments => On => On
opcache.use_cwd => On => On
opcache.validate_permission => Off => Off
opcache.validate_root => Off => Off
opcache.validate_timestamps => On => On

zip

Zip => enabled
Zip version => 1.22.7
Libzip version => 1.11.3
BZIP2 compression => Yes
XZ compression => Yes
ZSTD compression => Yes
AES-128 encryption => Yes
AES-192 encryption => Yes
AES-256 encryption => Yes

zlib

ZLib Support => enabled
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
Compiled Version => 1.3.1
Linked Version => 1.3.1

Directive => Local Value => Master Value
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

Additional Modules

Module Name

Environment

<Redacted>

PHP Variables

<Redacted>

Relevant log output

[AlliBalliBaba]

This looks like workers were not able to recover after the maximum execution timeout. Do you know what caused the execution timeouts (e.g unresponsive DB)? Are you using a framework? Are you maybe storing the output of the container image somewhere (STDOUT)?

You can also try rolling back to 1.11.1, since 1.11.2 is known to have some minor issues.

[LauJosefsen]

This looks like workers were not able to recover after the maximum execution timeout. Do you know what caused the execution timeouts (e.g unresponsive DB)? Are you using a framework? Are you maybe storing the output of the container image somewhere (STDOUT)?

You can also try rolling back to 1.11.1, since 1.11.2 is known to have some minor issues.

The execution timeouts are from fighting for the same lock in a database. We are running Laravel Octane, and our logger is in stdout, but it is completely empty from incident start to first restart.

I just tried reproduce the issue on a staging environment, but cannot reproduce the state where workers completely stop. I can reproduce requests causing worker starvation due to lock contention in the database (as is expected and not a Frankenphp bug), but at least some request throughput is maintained due to max execution errors.

[LauJosefsen]

Also, am I misunderstand the busy worker metric? Why can it go above the total worker metric? And why does ready worker metric stay at 8 even though no workers are (pressumably?) available?

I dont know if this is relevated to the issue or just metric reporting errors.

[AlliBalliBaba]

I think busy_workers also counts the number of queued requests, I think this is intended though I'm not fully sure (@withinboredom)

If you don't want to queue these requests forever you can also set a max_wait_time, which helps the server recover faster in these cases since it rejects requests after some time if there is no worker available.

frankenphp {
    max_wait_time 15s
}

How are you deploying these instances, with Kubernetes? Workers becoming fully unresponsive without logs is indeed strange, would be nice to have a way to reproduce it.

[LauJosefsen]

Yes I'm very aware issues without reproduction are annoying. I'm mostly writing since as I figured we could help each other reproducing the issue.

I'm still leaning towards a bug present, since the available worker metric reports all workers to be ready, yet nothing is processing.

We will add the max wait time along with a write_timeout in caddy. But again this doesn't exactly solve why no workers was scheduled at all.

We are deploying in Kubernetes, yes. Is there any particular information you are looking for?

[withinboredom]

If you're stuck waiting for a db lock it's very possible the db library extension is still waiting for that lock, when it finally gets it, and the request is ended, it's possible that php is in a weird state.