Bug Report
Describe the bug
out_prometheus_exporter parses the generic HTTP server configuration, but
prom_http_server_create() does not pass buffer_chunk_size or workers to
flb_http_server_init_with_options().
Consequently, http_server.workers is accepted without creating the requested worker
runtime. Under a large metrics workload, synchronous metrics decoding and Prometheus
text encoding in cb_prom_flush() can occupy the caller event loop and delay HTTP
response handling.
The omission is present in Fluent Bit 5.0.2, current 5.0, and current master.
To Reproduce
- Configure the exporter with explicit HTTP options:
service:
flush: 1
pipeline:
inputs:
- name: fluentbit_metrics
scrape_interval: 1
outputs:
- name: prometheus_exporter
match: "*"
host: 127.0.0.1
port: 2021
http_server.buffer_chunk_size: 64K
http_server.workers: 2
- Start Fluent Bit and inspect its threads.
- Observe that no HTTP worker runtime or listener worker threads are created.
- Apply the two missing assignments and repeat.
- Observe two HTTP worker threads and a non-null worker runtime.
A deterministic runtime test reproduces the defect without timing thresholds:
server.buffer_chunk_size == ins->http_server_config->buffer_chunk_size
server.workers == 2
server.runtime != NULL
All three checks fail as applicable before the fix and pass afterward.
Expected behavior
The parsed http_server.buffer_chunk_size and http_server.workers settings should
reach the embedded HTTP server.
Your Environment
- Version used: 5.0.2 and master at
fd5ea1f0d503
- Operating systems: embedded Linux arm64 and Debian 12 x86_64 test container
- Plugins:
process_exporter_metrics, fluentbit_metrics,
prometheus_exporter
Additional context
With approximately 1500 synthetic processes and a 2.47 MB metrics chunk, one
correlated reproduction showed:
request TTFB: 673.957 ms
cb_prom_flush total: 746.633 ms
cb_prom_flush decode: 740.716 ms
HTTP handler total: 8.083 ms
HTTP payload mutex wait: ~0.002-0.003 ms
After applying the option-propagation fix and configuring two HTTP workers:
requests: 762
worst TTFB: 46 ms
concurrent flush: ~954 ms
The fix does not reduce encoding CPU time. It makes the configured HTTP worker runtime
effective, allowing request handling to proceed independently.
A separate follow-up enhancement can default the exporter to one output worker so
cb_prom_flush() does not run on the main engine event loop even when HTTP workers are
not explicitly configured.
Bug Report
Describe the bug
out_prometheus_exporterparses the generic HTTP server configuration, butprom_http_server_create()does not passbuffer_chunk_sizeorworkerstoflb_http_server_init_with_options().Consequently,
http_server.workersis accepted without creating the requested workerruntime. Under a large metrics workload, synchronous metrics decoding and Prometheus
text encoding in
cb_prom_flush()can occupy the caller event loop and delay HTTPresponse handling.
The omission is present in Fluent Bit 5.0.2, current
5.0, and currentmaster.To Reproduce
A deterministic runtime test reproduces the defect without timing thresholds:
All three checks fail as applicable before the fix and pass afterward.
Expected behavior
The parsed
http_server.buffer_chunk_sizeandhttp_server.workerssettings shouldreach the embedded HTTP server.
Your Environment
fd5ea1f0d503process_exporter_metrics,fluentbit_metrics,prometheus_exporterAdditional context
With approximately 1500 synthetic processes and a 2.47 MB metrics chunk, one
correlated reproduction showed:
After applying the option-propagation fix and configuring two HTTP workers:
The fix does not reduce encoding CPU time. It makes the configured HTTP worker runtime
effective, allowing request handling to proceed independently.
A separate follow-up enhancement can default the exporter to one output worker so
cb_prom_flush()does not run on the main engine event loop even when HTTP workers arenot explicitly configured.