Mask a Lakebase map field as a whole, not per entry - #6441
Merged
Merged
Conversation
denik
force-pushed
the
denik/update-mask-index
branch
from
August 31, 2026 14:58
f9b090d to
bcab355
Compare
Contributor
Author
Regression Test ReportTested commit: bcab355 Shorten the changelog entry and link the PR
TestAccept/bundle/resources/postgres_endpoints/update_pg_settings/DATABRICKS_BUNDLE_ENGINE=direct ✅ | main (0a8aae1) ❌ | latest ➖main (0a8aae1): TestAccept/bundle/resources/postgres_projects/update_default_endpoint_autoscaling/DATABRICKS_BUNDLE_ENGINE=direct ✅ | main (0a8aae1) ❌ | latest ➖main (0a8aae1): TestAccept/bundle/resources/postgres_projects/update_default_endpoint_suspend/DATABRICKS_BUNDLE_ENGINE=direct ✅ | main (0a8aae1) ❌ | latest ➖main (0a8aae1): |
pietern
approved these changes
Sep 1, 2026
denik
force-pushed
the
denik/update-mask-index
branch
from
September 1, 2026 09:39
bcab355 to
e4fba84
Compare
denik
force-pushed
the
denik/update-mask-leaf-only
branch
from
September 1, 2026 09:49
76bec58 to
964c675
Compare
denik
force-pushed
the
denik/update-mask-index
branch
from
September 1, 2026 09:50
e4fba84 to
ebac25f
Compare
The plan diffs maps entry by entry, so changing one pg_settings key produced the change path settings.pg_settings['statement_timeout'], and the mask repeated it verbatim. A map or repeated field is addressable only as a whole: the API answers the indexed form with "Unknown field path in update_mask" (probed against a real workspace on 2026-08-31, where spec.settings and spec.settings.pg_settings are both accepted). Drop everything from the first subscript on, and dedupe, since two changed entries of one map collapse onto the same path. update_pg_settings covers the edit. add_settings covers adding the block, which masks the message itself; that path was missing from the fake's allowed list, so a case the real API accepts failed locally. Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
denik
force-pushed
the
denik/update-mask-index
branch
from
September 1, 2026 10:34
ebac25f to
dedcaa0
Compare
denik
enabled auto-merge
September 1, 2026 10:41
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Sep 1, 2026
Stacked on #6441 (which is stacked on #6440) — review those first. Four changes a bundle can express could not be deployed at all: | resource | field | |---|---| | `postgres_branches` | `expire_time`, `ttl` | | `postgres_endpoints` | `suspend_timeout_duration` | | `postgres_projects` | `default_endpoint_settings.suspend_timeout_duration` | `expire_time` / `ttl` / `no_expiry` are one oneof and `suspend_timeout_duration` / `no_suspension` another, and the API accepts them in `update_mask` only under the group name — masking the field itself is answered with `Unknown field path in update_mask`. All four now apply and their tests drop `Badness`. The group names are in neither the OpenAPI spec nor the SDK doc comments, so each map is hand-written from what the backend accepts, probed on 2026-08-31. Two members of one group collapse onto a single mask entry. `remove_suspend_timeout` still fails, and the mask is no longer why: the API requires a masked field to be populated in the body, so a removal has nothing to send. An absent value and an explicit `null` are both rejected; the supported way to express it is `no_suspension: true`. That test keeps a `Badness` saying so. This pull request and its description were written by Isaac.
denik
added a commit
that referenced
this pull request
Sep 1, 2026
Three files moved under the branch. One needed a real decision: structaccess/get.go -- main replaced .Interface().([]string) with reflect.TypeAssert in code this branch had already deleted, so the branch's version stands and main's form is applied to the three sites the branch still has. Consistent with the same change elsewhere in libs/structs. Rebasing was the wrong shape here: the conflict lands in a function the branch rewrites several times over, so it recurs commit by commit with no intermediate state worth resolving. One golden moves, in the branch's favour. #6441 masks a Lakebase map field as a whole rather than per entry, which is exactly the bug the catalog had recorded: four postgres_projects custom_tags rows go from BASE_ERROR to OK. Co-authored-by: Isaac
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 3, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 3, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
Stacked on #6440 — review that one first. A bundle that changes one key of `postgres_endpoints.settings.pg_settings` cannot deploy: ``` Unknown field path in update_mask: 'spec.settings.pg_settings['statement_timeout']' ``` The plan diffs maps entry by entry, so the change path carries the map key and the mask repeated it verbatim. A map or repeated field is addressable only as a whole. Probed against a real endpoint on 2026-08-31: `spec.settings` and `spec.settings.pg_settings` are both accepted, the indexed form is not. So drop everything from the first subscript on, and dedupe — two changed entries of one map collapse onto the same path. Terraform is unaffected; it masks the whole spec. Two tests, both local and cloud: - `update_pg_settings` — edit a key. Fails without this change. - `add_settings` — add the whole block, which leaves one change path and masks the message itself. That path was missing from the fake's allowed list, so a case the real API accepts was failing locally. This pull request and its description were written by Isaac.
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
Stacked on #6441 (which is stacked on #6440) — review those first. Four changes a bundle can express could not be deployed at all: | resource | field | |---|---| | `postgres_branches` | `expire_time`, `ttl` | | `postgres_endpoints` | `suspend_timeout_duration` | | `postgres_projects` | `default_endpoint_settings.suspend_timeout_duration` | `expire_time` / `ttl` / `no_expiry` are one oneof and `suspend_timeout_duration` / `no_suspension` another, and the API accepts them in `update_mask` only under the group name — masking the field itself is answered with `Unknown field path in update_mask`. All four now apply and their tests drop `Badness`. The group names are in neither the OpenAPI spec nor the SDK doc comments, so each map is hand-written from what the backend accepts, probed on 2026-08-31. Two members of one group collapse onto a single mask entry. `remove_suspend_timeout` still fails, and the mask is no longer why: the API requires a masked field to be populated in the body, so a removal has nothing to send. An absent value and an explicit `null` are both rejected; the supported way to express it is `no_suspension: true`. That test keeps a `Badness` saying so. This pull request and its description were written by Isaac.
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #6440 — review that one first.
A bundle that changes one key of
postgres_endpoints.settings.pg_settingscannot deploy:The plan diffs maps entry by entry, so the change path carries the map key and the mask repeated it verbatim. A map or repeated field is addressable only as a whole. Probed against a real endpoint on 2026-08-31:
spec.settingsandspec.settings.pg_settingsare both accepted, the indexed form is not.So drop everything from the first subscript on, and dedupe — two changed entries of one map collapse onto the same path. Terraform is unaffected; it masks the whole spec.
Two tests, both local and cloud:
update_pg_settings— edit a key. Fails without this change.add_settings— add the whole block, which leaves one change path and masks the message itself. That path was missing from the fake's allowed list, so a case the real API accepts was failing locally.This pull request and its description were written by Isaac.