Skip to content

automigration: run plan on migrated state before committing - #6486

Merged
denik merged 5 commits into
mainfrom
denik/automigration-with-plan
Sep 2, 2026
Merged

denik merged 5 commits into
mainfrom
denik/automigration-with-plan

Conversation

@denik

@denik denik commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Why

To prevent automigration for bundles that cannot work on direct without changes to config, for example #6480

Changes

Run a full bundle plan against the converted state (written to a temp file) before committing the automatic terraform→direct migration.

If the plan fails — for example because a resource can't be read from the workspace — the migration is abandoned: the bundle stays on terraform, direct_migrate_plan_error=true is recorded in telemetry, and the user sees the planning errors as warnings.

The plan check only runs when the migration would be committed (direct engine selected). Pure dry-run post-deploy migrations (with engine: terraform setting) are not running the plan.

After convertTFStateToDirect writes the converted state to a temp file,
open it for read and run a full CalculatePlan against the workspace before
committing the migration. If the plan returns any errors, record
direct_migrate_plan_error=true and abandon the migration (bundle stays on
terraform).

This only runs when we intend to commit (direct engine selected), so pure
dry-run passes are unaffected.

Co-authored-by: Isaac
The plan check runs in the commit path (real API calls, not a dry run),
so the stopped notice should not call it a dry-run migration.

Co-authored-by: Isaac
@denik
denik marked this pull request as ready for review September 2, 2026 12:59
CalculatePlan already returns a non-nil error when its context has errors,
so the subsequent HasError check was dead code.

Co-authored-by: Isaac
@denik
denik added this pull request to the merge queue Sep 2, 2026
@denik
denik removed this pull request from the merge queue due to a manual request Sep 2, 2026
@denik
denik added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit 8804ab4 Sep 2, 2026
27 checks passed
@denik
denik deleted the denik/automigration-with-plan branch September 2, 2026 14:56
deco-sdk-tagging Bot added a commit that referenced this pull request Sep 3, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
deco-sdk-tagging Bot added a commit that referenced this pull request Sep 3, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Why

To prevent automigration for bundles that cannot work on direct without
changes to config, for example
#6480

## Changes

Run a full `bundle plan` against the converted state (written to a temp
file) before committing the automatic terraform→direct migration.

If the plan fails — for example because a resource can't be read from
the workspace — the migration is abandoned: the bundle stays on
terraform, `direct_migrate_plan_error=true` is recorded in telemetry,
and the user sees the planning errors as warnings.

The plan check only runs when the migration would be committed (direct
engine selected). Pure dry-run post-deploy migrations (with `engine:
terraform` setting) are not running the plan.
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants