feat: ai context compaction backend

[notgiorgi]

Closes:

Description:

[notgiorgi]

• feat: add ai context compaction ui #23277
• feat: ai context compaction backend #23276 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[graphite-app]

throw new Error('Failed to create thread compaction') uses a plain Error instead of a typed error class. Per the backend rules, errors should be categorised using the existing error classes in errors.ts (or a new one created if none fits). Use an appropriate class such as UnexpectedServerError or create a new one.

Suggested change

	if (row === undefined) {
	throw new Error('Failed to create thread compaction');
	}
	if (row === undefined) {
	throw new UnexpectedServerError('Failed to create thread compaction');
	}

Spotted by Graphite (based on custom rule: packages/backend rules)



Is this helpful? React 👍 or 👎 to let us know.

[mintlify]

Docs PR opened: lightdash/mintlify-docs#667

Added a new AI agents guide page documenting thread context compaction, including how it works and how to enable it.

[github-actions]

🧪 Test Selection

✅ Tests that will run

Test	Description
Preview Environment	Deploys a preview environment for testing
Frontend E2E Tests	Runs Cypress app tests
Backend API Tests	Runs Vitest API tests
CLI Tests	Runs CLI integration and dbt version tests

⏭️ Tests skipped (no relevant file changes detected)

Test	How to trigger manually
Timezone Tests	Add test-timezone to PR description

Tip: Add test-all to your PR description to run all tests.

[github-actions]

Your preview environment pr-23276 has been deployed.

Preview environment endpoints are available at:

• https://lightdash-pr-23276.lightdash.okteto.dev
• https://lightdash-preview-pr-23276.lightdash.okteto.dev

[graphite-app]

The down migration drops the ai_thread_compaction table AND drops the token_usage column from ai_prompt. Deleting columns and tables are explicitly listed as dangerous operations that must not appear in migrations, because they are destructive and cannot be safely run while old code is still live. Remove the column drop (and ideally the table drop) from the down function, or at minimum make the column nullable rather than dropping it.

Suggested change

	export async function down(knex: Knex): Promise<void> {
	await knex.schema.dropTableIfExists(AI_THREAD_COMPACTION_TABLE_NAME);
	if (await knex.schema.hasColumn(AI_PROMPT_TABLE_NAME, 'token_usage')) {
	await knex.schema.alterTable(AI_PROMPT_TABLE_NAME, (table) => {
	table.dropColumn('token_usage');
	});
	}
	export async function down(knex: Knex): Promise<void> {
	// Intentionally left as a no-op.
	// Dropping columns and tables are destructive operations that cannot be safely
	// run while old code is still live. The table and column are left in place to
	// avoid data loss and to prevent breaking running instances.

Spotted by Graphite (based on custom rule: Migration guide)



Is this helpful? React 👍 or 👎 to let us know.

[github-actions]

Preview Environment

🌐 URL: https://lightdash-preview-pr-23276.lightdash.okteto.dev

📋 Logs: View in GCP Console

🔧 SSH: ./scripts/okteto-ssh.sh 23276

[notgiorgi]

Merge activity

• May 20, 9:49 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• May 20, 9:49 AM UTC: @notgiorgi merged this pull request with Graphite.

[lightdash-bot]

🎉 This PR is included in version 0.2985.0 🎉

The release is available on:

• 0.2985.0
• GitHub release

Your semantic-release bot 📦🚀