automigration: run plan on migrated state before committing - #6486
Merged
Merged
Conversation
After convertTFStateToDirect writes the converted state to a temp file, open it for read and run a full CalculatePlan against the workspace before committing the migration. If the plan returns any errors, record direct_migrate_plan_error=true and abandon the migration (bundle stays on terraform). This only runs when we intend to commit (direct engine selected), so pure dry-run passes are unaffected. Co-authored-by: Isaac
The plan check runs in the commit path (real API calls, not a dry run), so the stopped notice should not call it a dry-run migration. Co-authored-by: Isaac
denik
marked this pull request as ready for review
September 2, 2026 12:59
andrewnester
approved these changes
Sep 2, 2026
pietern
approved these changes
Sep 2, 2026
CalculatePlan already returns a non-nil error when its context has errors, so the subsequent HasError check was dead code. Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 3, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 3, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Why To prevent automigration for bundles that cannot work on direct without changes to config, for example #6480 ## Changes Run a full `bundle plan` against the converted state (written to a temp file) before committing the automatic terraform→direct migration. If the plan fails — for example because a resource can't be read from the workspace — the migration is abandoned: the bundle stays on terraform, `direct_migrate_plan_error=true` is recorded in telemetry, and the user sees the planning errors as warnings. The plan check only runs when the migration would be committed (direct engine selected). Pure dry-run post-deploy migrations (with `engine: terraform` setting) are not running the plan.
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
To prevent automigration for bundles that cannot work on direct without changes to config, for example #6480
Changes
Run a full
bundle planagainst the converted state (written to a temp file) before committing the automatic terraform→direct migration.If the plan fails — for example because a resource can't be read from the workspace — the migration is abandoned: the bundle stays on terraform,
direct_migrate_plan_error=trueis recorded in telemetry, and the user sees the planning errors as warnings.The plan check only runs when the migration would be committed (direct engine selected). Pure dry-run post-deploy migrations (with
engine: terraformsetting) are not running the plan.