-
Notifications
You must be signed in to change notification settings - Fork 898
Open
Labels
3.0This issue affects the HAProxy 3.0 stable branch.This issue affects the HAProxy 3.0 stable branch.3.2The issue affects the HAProxy 3.2 stable branch.The issue affects the HAProxy 3.2 stable branch.3.3The issue affects the HAProxy 3.3 stable branch.The issue affects the HAProxy 3.3 stable branch.status: fixedThis issue is a now-fixed bug.This issue is a now-fixed bug.type: bugThis issue describes a bug.This issue describes a bug.
Description
Detailed Description of the Problem
I started to roll out 3.3.1 yesterday (from 3.2.10 prior) and got two crashes over night, while it was running fine on 3.2. for quite some time. So I suspect a bug in 3.3.
Expected Behavior
No crash
Steps to Reproduce the Behavior
N/A
Do you have any idea what may have caused this?
Something LUA related and perhaps due to a reload that happened at the same time basically.
The LUA code basically parses data from a stick-table for rate calculation. It reads from stick-table and returns it.
-- Summieren von Sticktableeinträgen aller haproxy-Instanzen mit Hostnamensuffix
local function http_req_rate(txn, tablename)
-- Der Frontendname muss innerhalb der Konfiguration in die Variable req.fe_name geschrieben werden, z.B. via:
-- http-request set-var(req.fe_name) hdr(X-fe_name)
local fe_name = txn:get_var("req.fe_name")
local sticktable = core.proxies[tablename].stktable:dump({})
local rate = 0
-- Datenformat der gedumpten sticktable: { ["172.17.0.1"] = { ["gpt"] = 0,["http_req_cnt"] = 3,["http_req_rate"] = 3,} ,}
for k,v in pairs(sticktable) do
-- Vergleich des Frontendnamen ohne 'n095xxx|'-Prefix
-- Achtung: LUA Stringindizes beginnen bei 1!
if string.sub(k, 9) == fe_name then
rate = rate + v["http_req_rate"]
end
end
return rate
end
core.register_fetches("http_req_rate", http_req_rate);
Do you have an idea how to solve the issue?
No response
What is your configuration?
tune.lua.bool-sample-conversion normal
lua-load-per-thread /etc/haproxy/enodia-backend-counter.luaOutput of haproxy -vv
HAProxy version 3.3.1-9c24c11 2025/12/19 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2027.
Known bugs: http://www.haproxy.org/bugs/bugs-3.3.1.html
Running on: Linux 6.1.0-42-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.159-1 (2025-12-30) x86_64
Build options :
TARGET = linux-glibc
CC = cc
CFLAGS = -O2 -g -march=x86-64-v3 -mtune=znver1 -fwrapv -fvect-cost-model=very-cheap
OPTIONS = USE_LIBCRYPT=1 USE_OPENSSL_AWSLC=1 USE_LUA=1 USE_ZLIB= USE_SLZ=1 USE_NS= USE_PROMEX=1 USE_PCRE= USE_PCRE_JIT= USE_PCRE2=1 USE_PCRE2_JIT=
DEBUG = -DDEBUG_GLITCHES=1
Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ECH -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE +KTLS -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY +LUA +MATH -MEMORY_PROFILING +NETFILTER -NS -OBSOLETE_LINKER +OPENSSL +OPENSSL_AWSLC -OPENSSL_WOLFSSL -OT -PCRE +PCRE2 -PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX -PTHREAD_EMULATION -QUIC -QUIC_OPENSSL_COMPAT +RT +SHM_OPEN +SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL -ZLIB +ACME
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_TGROUPS=32, MAX_THREADS=1024, default=64).
Built with SSL library version : OpenSSL 1.1.1 (compatible; AWS-LC 1.66.2)
Running on SSL library version : AWS-LC 1.66.2
SSL library supports TLS extensions : yes
SSL library supports SNI : yes
SSL library FIPS mode : no
SSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.6
Built with the Prometheus exporter as a service
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.42 2022-12-11
PCRE2 library supports JIT : no (USE_PCRE2_JIT not set)
Encrypted password support via crypt(3): yes
Built with gcc compiler version 12.2.0
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTTP side=FE|BE mux=H2 flags=HTX|HOL_RISK|NO_UPG
<default> : mode=HTTP side=FE|BE mux=H1 flags=HTX
h1 : mode=HTTP side=FE|BE mux=H1 flags=HTX|NO_UPG
fcgi : mode=HTTP side=BE mux=FCGI flags=HTX|HOL_RISK|NO_UPG
<default> : mode=SPOP side=BE mux=SPOP flags=HOL_RISK|NO_UPG
spop : mode=SPOP side=BE mux=SPOP flags=HOL_RISK|NO_UPG
<default> : mode=TCP side=FE|BE mux=PASS flags=
none : mode=TCP side=FE|BE mux=PASS flags=NO_UPG
Available services : prometheus-exporter
Available filters :
[BWLIM] bwlim-in
[BWLIM] bwlim-out
[CACHE] cache
[COMP] compression
[FCGI] fcgi-app
[SPOE] spoe
[TRACE] trace
Last Outputs and Backtraces
Jan 12 22:38:08 n255027 systemd[1]: Reloaded haproxy.service - HAProxy Load Balancer.
Jan 12 22:38:08 n255027 haproxy[39149]: [NOTICE] (39149) : haproxy version is 3.3.1-9c24c11
Jan 12 22:38:08 n255027 haproxy[39149]: [NOTICE] (39149) : path to executable is /usr/sbin/haproxy
Jan 12 22:38:08 n255027 haproxy[39149]: [WARNING] (39149) : Former worker (132907) exited with code 0 (Exit)
Jan 12 22:38:16 n255027 haproxy[169678]: [ALERT] (169678) : proxy 'genlisten_25941-aa_schulung_app_static_delivery' has no server available!
Jan 12 22:38:16 n255027 haproxy[169678]: [ALERT] (169678) : proxy 'genlisten_34041-bbk_prod_www_headless_server' has no server available!
Jan 12 22:38:16 n255027 haproxy[169678]: [ALERT] (169678) : proxy 'genlisten_34241-bbk_stage_www_headless_server' has no server available!
Jan 12 22:38:17 n255027 haproxy[169678]: [ALERT] (169678) : proxy 'genlisten_aa_integration_www' has no server available!
Jan 12 23:12:34 n255027 haproxy[169678]: [ALERT] (169678) : proxy 'genlisten_17440-henkel_stage_www' has no server available!
Jan 12 23:12:44 n255027 haproxy[169678]: [ALERT] (169678) : proxy 'genlisten_62440-wtv_staging_www' has no server available!
Jan 12 23:12:51 n255027 haproxy[169678]: [ALERT] (169678) : proxy 'genlisten_24240-bmzin_staging_www' has no server available!
Jan 12 23:15:16 n255027 haproxy[169678]: call trace(13):
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d59d06d <02 00 00 e8 a3 8b f7 ff]: main+0x25a5bd > ha_backtrace_to_stderr
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d59e54d <44 89 e6 e8 43 e9 ff ff]: stats_fill_sv_line+0x2fd/0x127a > main+0x25a3e0
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d3c4022 <fe ff ff e8 2e a2 1d 00]: hlua_process_task+0x12612 > stats_fill_sv_line
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d3c8f39 <fe ff ff e8 57 ac ff ff]: hlua_process_task+0x17529 > hlua_process_task+0x12180
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d535e24 <40 48 8b 43 58 ff 50 18]: task_process_applet+0x344/0xa84
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d5dc2f9 <84 6a 02 00 00 41 ff d2]: run_tasks_from_lists+0x3d9/0x9fd
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d5dcd09 <4e 30 01 e8 17 f2 ff ff]: process_runnable_tasks+0x3e9/0xa3f > run_tasks_from_lists
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d52b5f5 <01 00 00 e8 2b 13 0b 00]: run_poll_loop+0x135/0x5a1 > process_runnable_tasks
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x55963d52bcd1 <00 00 00 e8 ef f7 ff ff]: run_thread_poll_loop+0x261/0x567 > run_poll_loop
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x7f0637f331f5 <00 00 ff 90 38 06 00 00]: libc:+0x891f5
Jan 12 23:15:16 n255027 haproxy[169678]: | 0x7f0637fb38dc <83 e4 f0 4c 89 c7 ff d2]: libc:+0x1098dc
Jan 12 23:15:16 n255027 haproxy[169678]: Hint: when reporting this bug to developers, please check if a core file was
Jan 12 23:15:16 n255027 haproxy[169678]: produced, open it with 'gdb', issue 'bt' to produce a backtrace for the
Jan 12 23:15:16 n255027 haproxy[169678]: current thread only, then join it with the bug report.
Jan 12 23:15:16 n255027 haproxy[39149]: [ALERT] (39149) : Current worker (169678) exited with code 132 (Illegal instruction)
Jan 12 23:15:16 n255027 haproxy[39149]: [WARNING] (39149) : A worker process unexpectedly died and this can only be explained by a bug in haproxy or its dependencies.
Jan 12 23:15:16 n255027 haproxy[39149]: Please check that you are running an up to date and maintained version of haproxy and open a bug report.
Jan 12 23:15:16 n255027 haproxy[39149]: HAProxy version 3.3.1-9c24c11 2025/12/19 - https://haproxy.org/
Jan 12 23:15:16 n255027 haproxy[39149]: Status: stable branch - will stop receiving fixes around Q1 2027.
Jan 12 23:15:16 n255027 haproxy[39149]: Known bugs: http://www.haproxy.org/bugs/bugs-3.3.1.html
Jan 12 23:15:16 n255027 haproxy[39149]: Running on: Linux 6.1.0-42-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.159-1 (2025-12-30) x86_64
Jan 12 23:15:16 n255027 haproxy[39149]: [ALERT] (39149) : exit-on-failure: killing every processes with SIGTERM
Jan 12 23:15:16 n255027 haproxy[39149]: [WARNING] (39149) : All workers exited. Exiting... (132)
Additional Information
No response
Metadata
Metadata
Assignees
Labels
3.0This issue affects the HAProxy 3.0 stable branch.This issue affects the HAProxy 3.0 stable branch.3.2The issue affects the HAProxy 3.2 stable branch.The issue affects the HAProxy 3.2 stable branch.3.3The issue affects the HAProxy 3.3 stable branch.The issue affects the HAProxy 3.3 stable branch.status: fixedThis issue is a now-fixed bug.This issue is a now-fixed bug.type: bugThis issue describes a bug.This issue describes a bug.