Skip to content

localenv: support TOML multiline strings during merge - #6445

Merged
rugpanov merged 3 commits into
mainfrom
fix/setup-local-multiline-toml
Aug 31, 2026
Merged

rugpanov merged 3 commits into
mainfrom
fix/setup-local-multiline-toml

Conversation

@rugpanov

@rugpanov rugpanov commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Allow the setup-local formatting-preserving merge to handle TOML multi-line strings without treating their contents as project structure.

Why

Valid pyproject.toml files commonly use multi-line project descriptions. The previous conservative guard rejected every such file, including strings unrelated to managed fields. Tracks DECO-28330.

Tests

  • GOTOOLCHAIN=local go test ./libs/localenv ./cmd/environments -count=1
  • Changed Go package lint: 0 issues
  • Full fmt/checks could not download ruff and golang.org/x/net because package network connections were refused; PR CI will run them

This PR was written with Codex.

@rugpanov
rugpanov marked this pull request as ready for review August 31, 2026 12:28
@rugpanov
rugpanov added this pull request to the merge queue Aug 31, 2026
Merged via the queue into main with commit bce2846 Aug 31, 2026
28 checks passed
@rugpanov
rugpanov deleted the fix/setup-local-multiline-toml branch August 31, 2026 14:02
hejcman-enverus pushed a commit to hejcman-enverus/databricks_cli that referenced this pull request Sep 3, 2026
Two changes to the four `golangci-lint run` invocations in
`Taskfile.yml`:

- Add `-j=4` to cap analyzer concurrency.
- Add `--allow-parallel-runners` and drop the per-task `TMPDIR`
workaround introduced in databricks#5050.

`./task lint`, fresh caches, same `tools/go.mod`:

| | Cold wall | Cold CPU (user+sys) | Warm wall |
|---|---|---|---|
| origin/main | 185s | 2282s | 4.1s |
| this PR | 76s | 327s | 4.0s |

2.4× faster cold wall, 7× less CPU. Warm runs are cache-bound and
unchanged.

## `-j=4`

golangci-lint scales poorly past 4–6 workers (upstream
[databricks#5149](golangci/golangci-lint#5149)). With
three sub-lints running in parallel under `lint-go`, the default
`-j=GOMAXPROCS` puts ~48 nominal workers on 16 cores — most of the 2282s
on origin/main is kernel scheduling contention. Capping at 4 lets the
three modules cooperate.

## `--allow-parallel-runners`

By default golangci-lint takes a system-wide lock and refuses to start
if another copy is already running. The per-task `TMPDIR` override
existed to give each invocation its own copy of that lock file so it
didn't collide; `--allow-parallel-runners` tells golangci-lint not to
take the lock in the first place. Same outcome, no `TMPDIR` plumbing.

Parallel runs happen when `lint-go` fans out to the three module
sub-lints, and when multiple worktrees lint at once.

Safe to skip the lock because golangci-lint v2.12.0 added
content-addressable cache keys ([PR
databricks#6445](golangci/golangci-lint#6445)). Before
that, two worktrees produced separate cache entries (keyed by absolute
path) — no sharing, but also no contention. Now the entries are shared
by content, so concurrent runs read and write the same files; the cache
uses OS-level file locks with atomic write semantics to keep that safe.
`origin/main` already pins v2.12.2.

## Trade-off

Two `task lint` runs in the *same* worktree no longer fail-fast with
`"parallel golangci-lint is running"` — they race instead, harmlessly
but wastefully (duplicate analysis work). Minor.

## Test plan

- [x] `task lint` clean
- [x] `task lint-q` clean
- [x] Parallel `task lint` across two worktrees both succeed
- [ ] CI lint job passes

This pull request and its description were written by Isaac.
deco-sdk-tagging Bot added a commit that referenced this pull request Sep 3, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
deco-sdk-tagging Bot added a commit that referenced this pull request Sep 3, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Changes
Allow the setup-local formatting-preserving merge to handle TOML
multi-line strings without treating their contents as project structure.

## Why
Valid pyproject.toml files commonly use multi-line project descriptions.
The previous conservative guard rejected every such file, including
strings unrelated to managed fields. Tracks
[DECO-28330](https://databricks.atlassian.net/browse/DECO-28330).

## Tests
- GOTOOLCHAIN=local go test ./libs/localenv ./cmd/environments -count=1
- Changed Go package lint: 0 issues
- Full fmt/checks could not download ruff and golang.org/x/net because
package network connections were refused; PR CI will run them

_This PR was written with Codex._

[DECO-28330]:
https://databricks.atlassian.net/browse/DECO-28330?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
janniklasrose pushed a commit that referenced this pull request Sep 15, 2026
## Release v1.15.0

### CLI

 * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457))
 * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445))

### Bundles

 * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486))
 * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418))
 * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377))
 * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489))
 * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510))
 * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440))
 * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441))
 * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443))
 * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408))
 * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378))
 * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442))

### Dependency Updates

 * Bump Go toolchain to 1.26.8. ([#6476](#6476))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants