feat!: use ClickHouse operator and OSS-licensed subcharts (v2.0.0) - #388
feat!: use ClickHouse operator and OSS-licensed subcharts (v2.0.0)#388cbeneke wants to merge 1 commit into
Conversation
| In case you run the Bitnami-based chart, please hold off from migrations for now, as the included ClickHouse version does not support Langfuse v4 yet. | ||
| We will publish updates in our [GitHub discussion](https://github.com/orgs/langfuse/discussions/12518) once a supported upgrade path is established. | ||
| You can also subscribe to [OSS release updates](https://langfuse.com/self-hosting/upgrade#release-notes) on our website. | ||
| In v2 the chart also auto-generates credential Secrets for Postgres, ClickHouse, Valkey, and SeaweedFS on first install (persisted across upgrades via `lookup`), so only the three Langfuse application secrets (`salt`, `encryptionKey`, `nextauth.secret`) need to be supplied. |
There was a problem hiding this comment.
Could we theoretically generate the langfuse secrets as well on first apply and make this a zero-config install? As long as there's sufficient randomness there's no strong advantage of having them be user-provided.
| In v2 the chart also auto-generates credential Secrets for Postgres, ClickHouse, Valkey, and SeaweedFS on first install (persisted across upgrades via `lookup`), so only the three Langfuse application secrets (`salt`, `encryptionKey`, `nextauth.secret`) need to be supplied. | ||
|
|
||
| ## Recommended Setup | ||
| There is **no automatic in-place `helm upgrade`** from v1. For a near-zero-downtime data migration (Postgres logical replication + MinIO/`mc mirror`, with ClickHouse online sync), see [`examples/upgrade-v1-to-v2`](./examples/upgrade-v1-to-v2/). |
There was a problem hiding this comment.
What happens if I try to run a helm upgrade? Can we prevent that from happening automatically on v1 -> v2? Or does helm protect against major upgrades using that command already?
| This is the most future-proof setup: your data stores are decoupled from chart releases, can be upgraded and scaled independently, and you always have access to the latest component versions. | ||
| Additionally, you can benefit from managed components that often offer automatic backups and other convenience functions. | ||
| It is also the setup that future improvements of this chart will build on. | ||
| For new installations that target [Langfuse v4](https://langfuse.com/docs/v4), prefer a current ClickHouse release (25.12 minimum, **26.4** recommended). The bundled ClickHouse CR defaults to `26.4`. You can also keep ClickHouse fully external — see [`examples/v4-installation`](./examples/v4-installation/) and [`examples/external-components`](./examples/external-components/). |
There was a problem hiding this comment.
I think the chart should always install Langfuse v4. I don't see a strong reason to have a v3-based version included.
Just having a docs example on how to overwrite the image should be sufficient.
-> I think this section can go away.
| cpu: "500m" | ||
| memory: "512Mi" | ||
|
|
||
| clickhouse: |
There was a problem hiding this comment.
Can we also preset the disk size for keeper to like 20GiB here and ClickHouse to 100GiB? I think that would help avoid problems on disk scale-ups shortly after an installation.
| shadowDatabaseUrl: postgres://my-username:my-password@my-external-postgres-server.com | ||
| ``` | ||
|
|
||
| ##### With an external ClickHouse cluster |
There was a problem hiding this comment.
Did this go away due to a merge conflict? IMO retaining details on how to connect a clickhouse cloud environments is desirable.
| helm install langfuse . -n langfuse \ | ||
| -f ../../examples/minimal-installation/values.yaml \ | ||
| -f ../../examples/external-components/external-postgres.yaml |
There was a problem hiding this comment.
I would add an explicit sentence that those overlays are additive, i.e. one can apply redis and postgres within the same apply (assuming that that's the case). This may read too much like "exactly one" can be applied.
| > Rehearse in staging. Take backups of **all** components first | ||
| > ([Backup Strategies](https://langfuse.com/self-hosting/configuration/backups)). | ||
| > Reuse the same Langfuse `salt`, `encryptionKey` and `nextauth.secret` on v2 — otherwise encrypted | ||
| > Postgres columns become undecryptable. Align Langfuse `appVersion` on v1 with the v2 chart target |
There was a problem hiding this comment.
I'd clarify that appVersion and any image.tag overwrites should remain the same.
| kubectl -n langfuse scale deploy/langfuse-web deploy/langfuse-worker --replicas=0 | ||
| # wait for the v1 worker Redis queue to drain |
There was a problem hiding this comment.
If we scale-down the worker the queue wouldn't drain, right? Would that be a web scaledown first and then a worker scaledown second?
|
|
||
| # ReplacingMergeTree data tables that carry an `event_ts` version column. These | ||
| # are safe to copy incrementally with overlap (dedup handles duplicates). | ||
| : "${INCREMENTAL_TABLES:=traces observations scores dataset_run_items dataset_run_items_rmt project_environments blob_storage_file_log}" |
There was a problem hiding this comment.
We can ignore dataset_run_items, project_environments, and event_log here. They're not used in the newer v3 versions. We could say that something like v3.224.1 (latest published) is a minimum version for this migration to simplify things here.
| if [ $FINAL -eq 1 ]; then | ||
| for t in $FINAL_ONLY_TABLES; do copy_final_only "$t"; done | ||
| echo "== OPTIMIZE FINAL to collapse duplicates ==" | ||
| for t in $INCREMENTAL_TABLES; do | ||
| echo ">> OPTIMIZE $t FINAL"; chq "OPTIMIZE TABLE $TARGET_DB.$t FINAL" || true | ||
| done | ||
| fi |
There was a problem hiding this comment.
I'd prefer if we don't do that. Optimize final has some problematic behaviours on large data volumes. We can document this as an option, but it shouldn't be the default. Users can run it themselves if they are aware of side effects (link to clickhouse docs) and they observe too many duplicates after running this transfer. From what I can tell, duplicates are unexpected anyway, so we shouldn't optimize for this scenario.
2 participants
Based off #347
Summary