[Backport release-1.6] fix(kubernetes): retry the tenant CNI install instead of uninstalling it - #4033

The tenant cilium HelmRelease carried no install strategy, so Flux
applied its default, RemediateOnFailure, which uninstalls the release
between attempts. Cilium is the tenant cluster's only CNI, so every
cycle takes it away and gives it back when the next install applies the
manifests again, and retries: -1 lets that repeat without end.

What sets a cycle off is an install that outruns its budget, which can
happen while the tenant still has no registered node, since the operator
Deployment cannot become Available until one exists. Remediation neither
causes that delay nor is fixed by this change; what it adds is a
repeated teardown of the CNI on top of a delay that resolves itself once
the node arrives. This removes the amplifier, not the cause.

RetryOnFailure keeps the applied manifests in place and retries a failed
install as an upgrade, so the rollout completes on its own once a worker
registers. Both actions carry the strategy because the retry of a failed
install runs as an upgrade, which would otherwise fall back to upgrade
remediation. It is the strategy and retry interval cozystack-api and
cozystack-operator already set on every HelmRelease they generate,
except that they leave remediation nil where this release keeps it. The
other tenant addons the chart renders still carry the remediation-only
shape; this one is where the teardown removes the cluster's only CNI
rather than churning a single component.

The cost is that a genuine upgrade failure no longer rolls back either,
it retries on the failed manifests instead. That is forced, since the
controller cannot tell a retried install from a real upgrade, and what
it replaces was an unbounded rollback-and-retry flap rather than a
stable safety net.

The remediation blocks stay, since they cover the branches no strategy
reaches: an absent release, which consults the install block, and an
out-of-sync one, which consults upgrade.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
(cherry picked from commit 80a2090)
Signed-off-by: Myasnikov Daniil <[email protected]>