Skip to content

Add support for breakglass for immutable folders - #6774

Open
andrewnester wants to merge 4 commits into
mainfrom
feat/immutable/breakglass
Open

andrewnester wants to merge 4 commits into
mainfrom
feat/immutable/breakglass

Conversation

@andrewnester

@andrewnester andrewnester commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Add support for breakglass for immutable folders

Why

Immutable folders can be "break-glassed" in an emergency. In this case, CLI should prevent a normal deploy if the previously used snapshot was broken. With --force, it should allow to deploy to a new suffixed path and the deployments after should continue as normal

Note: there is no REST API endpoint to check the status of folder yet so we use GraphQL one for now

Tests

Added an acceptance test

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

11 files changed
Suggested: @denik
Also eligible: @janniklasrose, @pietern, @shreyas-goenka, @anton-107, @lennartkats-db

/bundle/ - needs approval

7 files changed
Suggested: @denik
Also eligible: @janniklasrose, @pietern, @shreyas-goenka, @anton-107, @lennartkats-db

General files (require maintainer)

4 files changed
Based on git history:

  • @denik -- recent work in libs/testserver/, .nextchanges/bundles/, bundle/direct/dresources/

Any maintainer (@anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db, @rugpanov, @rclarey) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: f1b06de

Run: 36122702593

Env ✅​pass 🙈​skip Time
✅​ aws linux 276 21 5:12
✅​ aws windows 278 19 4:14
✅​ azure linux 275 21 5:20
✅​ gcp linux 276 21 5:23
✅​ gcp windows 278 19 3:56
Top 5 slowest tests (at least 2 minutes):
duration env testname
4:12 aws windows TestAccept
4:02 gcp linux TestAccept
3:56 azure linux TestAccept
3:55 gcp windows TestAccept
3:51 aws linux TestAccept

Comment thread .nextchanges/bundles/immutable-breakglass.md Outdated
return 0, nil
}

entry, ok := b.DeploymentBundle.StateDB.GetResourceEntry(resources.SnapshotKey)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads state and makes a determination. Shouldn't that live in a DoRead of the snapshot?

I.e. seeing a different generation, or a "broken_glass: true" on DoRead can trigger recreation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple of blockers why it is not in DoRead, mainly the --force flag, which is not available at the resource level

It's also conceptually similar to existing CheckDashboardsModifiedRemotely check we have

Comment thread libs/snapshot/client.go
// ContentSubdir is the fixed subfolder the snapshot API appends to a caller-supplied
// relative_path to form the immutable content path (e.g. "<root>/<relative_path>/snapshot").
// The content path is what the create API echoes back and what InspectSnapshot inspects.
const ContentSubdir = "snapshot"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this happen?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's related to the fact that when we upload zip, we create 2 folders: one snapshot which can be broken glass and one is .internal which is not and contains a break glass marker and some other metadata

Comment thread libs/snapshot/client.go Outdated
@andrewnester
andrewnester requested review from a team as code owners September 25, 2026 10:11
@github-actions github-actions Bot added the DABs DABs related issues label Sep 25, 2026
ServicePrincipalName: p.ServicePrincipalName,
})
}
return canManage

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran a test and found the API rejects an empty list.

We could always include the current user, I think?

Comment thread libs/snapshot/client.go
}

var resp inspectSnapshotResponse
err = c.client.Do(ctx, http.MethodGet, inspectSnapshotPath, headers, nil, nil, &resp, withJSONBody(payload))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to do a GET with a request payload. It should use a query param.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants