fix(linstor): add opt-in graceful satellite shutdown on Talos - #4292

Backport of #4292 to release-1.6.

Cozystack's Talos configuration removes the systemd DRBD shutdown guard
without a replacement. talos.gracefulShutdown.enabled, disabled by default,
releases unused Secondary DRBD resources before the satellite stops so their
backing devices, including ZFS zvols, can be released during normal shutdown.
The hook skips Primary resources and open devices, rechecks each candidate,
and stops on ambiguous status, command failure or timeout. It uses no force,
demotion, Kubernetes API calls or extra permissions.

Three differences from main, all from the same cause: release-1.6 has no
drbd.enabled flag, since the DRBD-less-substrate switch landed after 1.6 was
cut. The hook is therefore gated on talos.gracefulShutdown.enabled alone, the
values block adds only that key, and the case asserting the hook is omitted on
a substrate without DRBD is left out along with the flag it sets. The README
line naming both flags names talos.enabled instead. Nothing else differs.

The automated backport in #4320 stopped at the same values.yaml hunk and
pushed the conflict markers as its only commit, which is why this exists.

Verified on this branch: 11 Helm tests and the 20 behavioural Python tests
pass.

Assisted-by: LLM
Signed-off-by: Yan Bondarenko <[email protected]>

Backport of #4292 to release-1.6.

Cozystack's Talos configuration removes the systemd DRBD shutdown guard
without a replacement. talos.gracefulShutdown.enabled, disabled by default,
releases unused Secondary DRBD resources before the satellite stops so their
backing devices, including ZFS zvols, can be released during normal shutdown.
The hook skips Primary resources and open devices, rechecks each candidate,
and stops on ambiguous status, command failure or timeout. It uses no force,
demotion, Kubernetes API calls or extra permissions.

Three differences from main, all from the same cause: release-1.6 has no
drbd.enabled flag, since the DRBD-less-substrate switch landed after 1.6 was
cut. The hook is therefore gated on talos.gracefulShutdown.enabled alone, the
values block adds only that key, and the case asserting the hook is omitted on
a substrate without DRBD is left out along with the flag it sets. The README
line naming both flags names talos.enabled instead. Nothing else differs.

The automated backport in #4320 stopped at the same values.yaml hunk and
pushed the conflict markers as its only commit, which is why this exists.

Verified on this branch: 11 Helm tests and the 20 behavioural Python tests
pass.

Assisted-by: LLM
Signed-off-by: Yan Bondarenko <[email protected]>
Signed-off-by: Myasnikov Daniil <[email protected]>

Backport of #4292 to release-1.6.

Cozystack's Talos configuration removes the systemd DRBD shutdown guard
without a replacement. talos.gracefulShutdown.enabled, disabled by default,
releases unused Secondary DRBD resources before the satellite stops so their
backing devices, including ZFS zvols, can be released during normal shutdown.
The hook skips Primary resources and open devices, rechecks each candidate,
and stops on ambiguous status, command failure or timeout. It uses no force,
demotion, Kubernetes API calls or extra permissions.

Three differences from main, all from the same cause: release-1.6 has no
drbd.enabled flag, since the DRBD-less-substrate switch landed after 1.6 was
cut. The hook is therefore gated on talos.gracefulShutdown.enabled alone, the
values block adds only that key, and the case asserting the hook is omitted on
a substrate without DRBD is left out along with the flag it sets. The README
line naming both flags names talos.enabled instead. Nothing else differs.

The automated backport in #4320 stopped at the same values.yaml hunk and
pushed the conflict markers as its only commit, which is why this exists.

Verified on this branch: 11 Helm tests and the 20 behavioural Python tests
pass.

Assisted-by: LLM
Signed-off-by: Yan Bondarenko <[email protected]>
Signed-off-by: Myasnikov Daniil <[email protected]>

…utdown on Talos (#4401)

This reopens a fork backport from a branch in this repository so its CI
can run. The commit is the one from #4338 by @yankawai, unchanged. On
release-1.6 the pull-request CI pushes the images it builds, and a run
from a fork has no registry credentials, so #4338 stopped at the build
jobs and never reached e2e. The description below is the author's.

## What this PR does

Manual backport of #4292 to `release-1.6`.

The automated backport, #4320, stopped on a conflict in
`packages/system/linstor/values.yaml` and pushed the markers as its only
commit, so it is red on DCO and would ship conflict markers in a values
file if merged.

The conflict is `drbd.enabled`, which does not exist on this line: the
DRBD-less-substrate switch landed on main after 1.6 was cut, and the
cherry-pick pulled its values block in with the hunk it needed.
Everything else in the change applies unchanged, so three differences
follow from that one cause:

- the preStop hook is gated on `talos.gracefulShutdown.enabled` alone
rather than on it and `drbd.enabled`;
- the values block adds `talos.gracefulShutdown` only;
- the case asserting the hook is omitted on a Talos substrate without
DRBD goes with the flag it sets, and the README line that named both
flags names `talos.enabled`.

Verified on this branch: 11 Helm tests and the 20 behavioural Python
tests pass. The file set and line counts match #4292 apart from those
three points.

Feature summary, unchanged from #4292: Cozystack's Talos configuration
removes the systemd DRBD shutdown guard without a replacement.
`talos.gracefulShutdown.enabled`, disabled by default, releases unused
Secondary DRBD resources before the satellite stops so their backing
devices, including ZFS zvols, can be released during normal shutdown.
The hook skips Primary resources and open devices, rechecks each
candidate, and stops on ambiguous status, command failure or timeout. It
never forces, demotes, calls the Kubernetes API or asks for extra
permissions. It is opt-in because preStop also runs on ordinary
satellite restarts, and Piraeus has one DaemonSet per node with no
cluster-wide rollout barrier.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff: this touches one system chart,
its hook script and its tests. No downstream repository restates the
linstor chart values.

- [x] No downstream repository is affected by this change

### Release note

```release-note
fix(linstor): add `talos.gracefulShutdown.enabled`, off by default, so a satellite releases unused Secondary DRBD resources before it stops and their backing devices, ZFS zvols included, can be released during a normal Talos shutdown. Enable it only where satellite and node restarts are serialized: the hook also runs on ordinary pod restarts, and Piraeus has one DaemonSet per node.
```

The audit started from labels and dropped every backport PR whose
original was not a candidate for the branch. On release-1.6 that hid
nine hand backports of unlabelled main PRs (#4431 to #4435, #4437,
#4438, #4456 and #4475), and it had no way to notice two open backport
PRs for the same originals: #4421 sat open next to #4456 for #3938 and
#4280, and nothing reported it.

Read the PRs on the branch from the side of the originals they claim as
well, and report two more sections per branch.

UNLABELLED lists each original claimed by backport PRs on the branch
that is not a candidate for it, with every backport PR claiming it and
its state, and says whether they backported it, only claim it with a PR
still open, or were all closed. It never moves the exit code: the gate
answers whether everything labelled landed, and an unlabelled backport
can only add to a branch, never leave a labelled change off it.

DUPLICATE lists each original claimed by two open backport PRs, or by
an open one after another already merged, labelled or not. Both make
the audit exit 1. One of the PRs is redundant, or the open one is the
rest of a split backport; either way someone has to decide before the
cut, and no verdict can say so, since a verdict settles on the first
merged backport or reports the first open one as pending. A closed PR
next to an open one is how a conflicting bot backport gets redone by
hand and is not flagged.

The titles of originals that no listing carries come from one GraphQL
request for the whole run. A failed lookup costs the titles and nothing
else: the URL is derived locally and the exit code is already settled.

--json now emits an object per branch holding candidates, unlabelled
and duplicates arrays, in place of the bare array of verdicts.

On release-1.6 today this lists 13 unlabelled originals, the nine above
among them, and three duplicates that are open right now: the bot's
conflict drafts for #3936, #4254 and #4292 were left open next to their
hand backports, and #4254 and #4292 each also have a fork PR open next
to its reopening from a branch in this repository. #4421 is closed and
shows up only as a closed claim on #4280.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <[email protected]>

A candidate counted as backported as soon as its backport PR had merged,
or as soon as any one of its commits showed up on the branch by subject
or -x reference. That is exactly the shape of the backport bot's
conflict drafts: they stop at the first commit that does not apply and
drop the rest, so merging one reads as a finished backport. A two-commit
PR with only its first commit on the branch and no backport PR linked
reported clean and exited 0.

Judge delivery per commit. Every commit the PR contributed has to be on
the branch -- reachable, named by an -x reference, or present under its
own subject -- skipping merge commits and commits that change nothing,
which the bot drops as well (merge_commits: skip; an empty cherry-pick
applies nothing). Nothing weaker counts: not an -x reference to the
PR's merge commit, which survives a cherry-pick later amended to drop a
commit, and not matching lines, which an unrelated line of the same
text satisfies. Subjects are matched one to one, so two commits both
called "fix tests" need two branch commits of that name, and a branch
commit whose -x reference names another commit is evidence only through
that reference. A wrong "backported" is the one answer a release gate
must not give, while a false alarm costs a look.

Some commits carried is partial, listing the missing ones, whatever the
linked backport PR says. A merged backport PR settles a PR of one commit
alone, as before; for a PR of several with none of them found it proves
only that something merged, and the candidate is unverified. Both fail
the gate.

Backports squashed by hand would then keep a branch red for good, so a
maintainer who has checked one can attest it on the merged backport PR
with a comment that is "backport-audit: complete" and nothing else.
That lifts partial or unverified to confirmed, which passes; the report
names who and links the comment. It counts only as the whole comment,
so it needs no reading of markdown: any other text -- a sentence
mentioning it, a quote, a code block, an explanation -- and it is not an
attestation. It counts only on a PR that merged, only from an author
GitHub associates with the repository as OWNER, MEMBER or COLLABORATOR,
and never from automation, whether its login ends in [bot] or is a
review or dependency bot gh reports without the suffix. A review bot
quoting the marker, a maintainer writing "do not post backport-audit:
complete yet", or a contributor vouching for their own backport
confirms nothing. The audit never infers it.

A git read a verdict depends on that fails, a merge commit missing from
the local repository for instance, now stops the audit with exit 2
instead of being read as a PR with no commits to check.

Unlabelled originals are never checked commit by commit, so their merged
claims read "backport PR merged" rather than "backported here".

On release-1.6 this reports five partials. #3955 and #3968 are real
gaps: a labeler mapping and a comment fix never reached the branch.
#4254 and #4292 were squashed by hand and their remaining changes are on
the branch; #3936's backport cherry-picked the PR's merge commit. Those
three are what an attestation is for, once someone has checked them.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <[email protected]>