Skip to content

chore: upgrade clickhouse-sink-connector#3940

Merged
vpol merged 1 commit into
mainfrom
codex/upgrade-clickhouse-sink-connector
Jun 11, 2026
Merged

chore: upgrade clickhouse-sink-connector#3940
vpol merged 1 commit into
mainfrom
codex/upgrade-clickhouse-sink-connector

Conversation

@vpol

@vpol vpol commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • migrate clickhouse sync Debezium JDBC storage config keys to the new scheme
  • pin storage table DDLs to explicit api tables and add the replica status view override
  • update clickhouse-sink-docker to e4f6c9b3a958c9a5a4ef4aaa9f29e2d223293ff8

Checks

  • ruby YAML parse for .infra/clickhouse-sync.yml
  • targeted deprecated-key scan

@pulumi

pulumi Bot commented Jun 11, 2026

Copy link
Copy Markdown

🍹 The Update (preview) for dailydotdev/api/prod (at 922cd47) was successful.

✨ Neo Code Review

Routine application deployment to the new image tag, plus a correctness fix to ClickHouse sync connector property names that causes a pod replacement with a brief CDC replication pause. 🟡 Moderate Risk

This PR does two things: deploys a new application version (922cd47e... → all API deployments and crons), and fixes Debezium connector property key names in the ClickHouse sync configuration.

The config key renames (offset.storage.jdbc.offset.table.nameoffset.storage.jdbc.table.name, same pattern for .ddl and .delete, plus schema.history.internal.jdbc.schema.history.table.*schema.history.internal.jdbc.table.*) correct the property names to match what the JDBC offset/history store actually expects. The DDL statements also now use hardcoded table names (api.replica_source_info, api.replicate_schema_history) instead of %s placeholders. A new replica.status.view block is added to create a monitoring view.

🟡 WarningClickHouse sync replication gap during replacement: The clickhouse-sync-clickhouse-sync Deployment and its backing Secret are both being replaced (not rolling-updated) due to the config data change. The Deployment uses strategy: Recreate, meaning the existing pod is terminated before the new one starts. During this window, CDC replication from Postgres to ClickHouse will pause. The connector will resume from its stored offset in api.replica_source_info, so no data should be permanently lost, but there will be a lag catch-up period after restart.

🔵 Info — The DDL table name change from %s to hardcoded names means the CREATE TABLE IF NOT EXISTS statements will always target the correct tables regardless of how the connector substitutes the placeholder — this is a correctness fix, not a behavioral change if the connector was already pointed at the right tables.

Resource Changes

    Name                                                    Type                                      Operation
~   vpc-native-clean-old-notifications-cron                 kubernetes:batch/v1:CronJob               update
~   vpc-native-post-analytics-history-day-clickhouse-cron   kubernetes:batch/v1:CronJob               update
~   vpc-native-update-source-public-threshold-cron          kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-zombie-users-cron                      kubernetes:batch/v1:CronJob               update
~   vpc-native-temporal-deployment                          kubernetes:apps/v1:Deployment             update
~   vpc-native-hourly-notification-cron                     kubernetes:batch/v1:CronJob               update
~   vpc-native-post-analytics-clickhouse-cron               kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-zombie-opportunities-cron              kubernetes:batch/v1:CronJob               update
-   vpc-native-api-db-migration-808a92c8                    kubernetes:batch/v1:Job                   delete
~   vpc-native-materialize-monthly-best-post-archives-cron  kubernetes:batch/v1:CronJob               update
~   vpc-native-worker-job-deployment                        kubernetes:apps/v1:Deployment             update
~   vpc-native-channel-digests-cron                         kubernetes:batch/v1:CronJob               update
~   vpc-native-post-lifecycle-state-clickhouse-cron         kubernetes:batch/v1:CronJob               update
+   vpc-native-api-db-migration-7001b9d9                    kubernetes:batch/v1:Job                   create
~   vpc-native-generic-referral-reminder-cron               kubernetes:batch/v1:CronJob               update
~   vpc-native-update-trending-cron                         kubernetes:batch/v1:CronJob               update
~   vpc-native-update-tag-materialized-views-cron           kubernetes:batch/v1:CronJob               update
~   vpc-native-update-views-cron                            kubernetes:batch/v1:CronJob               update
~   vpc-native-private-deployment                           kubernetes:apps/v1:Deployment             update
~   vpc-native-expire-super-agent-trial-cron                kubernetes:batch/v1:CronJob               update
+-  clickhouse-sync-clickhouse-sync                         kubernetes:apps/v1:Deployment             create-replacement
~   vpc-native-update-tags-str-cron                         kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-expired-better-auth-sessions-cron      kubernetes:batch/v1:CronJob               update
~   vpc-native-user-profile-analytics-clickhouse-cron       kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-zombie-images-cron                     kubernetes:batch/v1:CronJob               update
~   vpc-native-user-profile-updated-sync-cron               kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-channel-highlights-cron                kubernetes:batch/v1:CronJob               update
~   vpc-native-check-analytics-report-cron                  kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-zombie-user-companies-cron             kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-stale-user-transactions-cron           kubernetes:batch/v1:CronJob               update
+   vpc-native-api-clickhouse-migration-7001b9d9            kubernetes:batch/v1:Job                   create
~   vpc-native-channel-highlights-cron                      kubernetes:batch/v1:CronJob               update
~   vpc-native-rotate-daily-quests-cron                     kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-gifted-plus-cron                       kubernetes:batch/v1:CronJob               update
~   vpc-native-sync-subscription-with-cio-cron              kubernetes:batch/v1:CronJob               update
~   vpc-native-bg-deployment                                kubernetes:apps/v1:Deployment             update
~   vpc-native-rotate-weekly-quests-cron                    kubernetes:batch/v1:CronJob               update
~   vpc-native-daily-digest-cron                            kubernetes:batch/v1:CronJob               update
+-  clickhouse-sync-clickhouse-sync-config                  kubernetes:core/v1:Secret                 create-replacement
~   vpc-native-personalized-digest-cron                     kubernetes:batch/v1:CronJob               update
~   clickhouse-sync                                         dailydotdev:pulumi-common:ClickHouseSync  update
... and 16 other changes

@vpol vpol marked this pull request as ready for review June 11, 2026 10:05
@vpol vpol merged commit 4a266b7 into main Jun 11, 2026
8 checks passed
@vpol vpol deleted the codex/upgrade-clickhouse-sink-connector branch June 11, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant