Skip to content

[exporter/clickhouse] Upgrade Logs Table (optimizations + full text search)#47720

Open
SpencerTorres wants to merge 6 commits intoopen-telemetry:mainfrom
SpencerTorres:upgrade_logs_table
Open

[exporter/clickhouse] Upgrade Logs Table (optimizations + full text search)#47720
SpencerTorres wants to merge 6 commits intoopen-telemetry:mainfrom
SpencerTorres:upgrade_logs_table

Conversation

@SpencerTorres
Copy link
Copy Markdown
Member

Description

Upgrades the default logs table schema for performance enhancements without breaking backwards compatability.

  • Version-aware indexes: on ClickHouse 26.2+, uses TYPE text(tokenizer = ...) full text search indexes; older versions fall back to bloom_filter / tokenbf_v1.
  • New schema: drops TimestampTime, adds 8 materialized __otel_materialized_* columns for common k8s/deployment resource attributes, and changes ORDER BY to (toStartOfFiveMinutes(Timestamp), ServiceName, Timestamp) to optimize time-bucketed range scans for Grafana/HyperDX workloads.
  • Templating: migrates the logs SQL files from positional fmt.Sprintf to Go text/template with named fields and an {{ident}} helper for backtick identifier quoting.

Existing tables are not modified, and the INSERTs are backwards compatible. The JSON pipeline gets the template changes but keeps its existing indexes since they don't apply to map types.

Technically this isn't a breaking change, but we can mark it as one so people are more aware of the changes. This should not affect existing deployments, these changes only apply to new deployments.

Testing

Added unit tests covering both index-rendering paths. Integration matrix updated to run against 26.2-alpine (full text search) and 25.8-alpine (bloom_filter fallback).

Documentation

Updated README to match new query recommendations, old queries are still functional.

@SpencerTorres SpencerTorres requested a review from a team as a code owner April 17, 2026 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants