Skip to content

feat(redis): add opt-in TLS for Redis and Sentinel - #2729

Merged
Aleksei Sviridkin (lexfrei) merged 31 commits into
mainfrom
feat/tls-redis
Sep 4, 2026
Merged

Aleksei Sviridkin (lexfrei) merged 31 commits into
mainfrom
feat/tls-redis

Conversation

@Arsolitt

@Arsolitt Arsolitt (Arsolitt) commented May 25, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

tls.enabled puts Redis and Sentinel on a TLS-only listener with a per-release cert-manager chain, and hands the tenant the CA certificate as redis-<name>.tenant-ca. Off unless set.

redis-operator v3.3.5 has no TLS support. The work is ported upstream in freshworks-oss/redis-operator#85, which is open and not in a release, so this carries it as three patches over the v3.3.5 tarball. images/redis-operator/patches/tls.diff is the source half, cut from that PR's commit 2ad2ed8d1, which the patch header pins; it is applied by the glob the image Dockerfile already runs, after the Dockerfile strips the tarball's manifests, charts, example and .github, so the image builds from source alone and the upstream tests that read the CRD YAML skip instead of failing against the stale v3.3.5 copy. patches/crd-tls.diff adds spec.tls to the vendored CRD, and with it the CEL rule that fixes tls.enabled at creation; without the schema block the apiserver prunes the field on admission and the patched binary waits for something it never sees, and the rule cannot exist without the block it references. patches/rbac-tls.diff grants Create, Get and Update on Secrets, so the operator can publish the CA-only Secret it addresses by name. That is the whole delta, and it is narrower than the ported operator's own chart asks for: those three are the only Secret calls in the patched source, and cert-manager Certificates get no grant at all, because the operator only touches one under spec.tls.certManager and this chart pins certificateSecret mode instead. One change in tls.diff is not TLS and takes effect with TLS off, named in the patch header: MakeSlaveOfWithPort connects to the replica on its own port rather than on the master's. It changes a dial, not a pod template, so shipping the rebuilt image rolls nothing; the plaintext pod templates stay byte-identical to v3.3.5.

The image build registration this branch used to carry (make -C packages/system/redis-operator image in the root build: target) landed on main with the Valkey service (#3380), so the rebuilt operator image is produced by CI from main as well; the digest in values.yaml is still refreshed by release preparation, not here.

The chart patches re-apply at the end of update:, the same way reloader, grafana-operator, ingress-nginx, seaweedfs and kamaji carry theirs. Otherwise rm -rf charts takes TLS back out on the next make update and nothing fails. make update reproduces the committed tree byte for byte.

TLS is chosen when the instance is created. The config gets port 0 and the listener moves to tls-port, so TLS replaces plaintext rather than running beside it, and a running instance cannot change mode: a replica restarted onto the TLS listener cannot replicate from a master still on plaintext, and Sentinels already on TLS would fail over to a stale replica. The RedisFailover schema in the carried CRD patch rejects a change of tls.enabled, which on an existing instance surfaces as a failed HelmRelease upgrade while the instance keeps serving what it did; the README points at creating a new instance and migrating. That is also why TLS is not inferred from external: an externally reachable instance is the one most likely to have clients an upgrade must not drop. Existing releases keep their plaintext listener and are not restarted by the rebuilt operator image.

The tenant gets ca.crt and nothing else, through tenantsecrets. The chart's own Secrets take a dot between the release and the suffix (<release>.ca-tls, <release>.tls, <release>.ca-cert): the suffixes are dot-free, so the split is unique and release foo's CA Secret can no longer be release foo-ca's leaf, the collision the hyphenated form had. The chart renders a TenantProjection pointing at <release>.ca-tls, and the CA-extraction controller re-encodes the one named key from the parsed certificate, so nothing that is not a certificate gets through. Same path as postgres, mongodb, nats and qdrant.

No Secret of this release is granted to the tenant by name beyond the generated password, and that includes <release>.ca-cert, the CA-only copy the operator publishes for itself. It holds no key today, but the operator has Create and Update on Secrets cluster-wide, so what sits at that name is the operator's call rather than the platform's, and RBAC cannot filter by key. hack/check-redis-rd-secrets.bats pins the whole rendered Secret surface instead of a list of known-bad names, because an include with no resourceNames and no matchLabels restricts nothing.

The chainsaw suite for redis gains two cases. A TLS fixture asserts the cert-manager chain is issued, the TLS-probed Redis replicas and Sentinel come up, the operator publishes its CA-only Secret under the widened ClusterRole, the tenant receives exactly ca.crt through tenantsecrets while the key-bearing Secrets stay off that path, and a redis-cli --tls PING through the service name completes with the same CA. A second case applies tls.enabled to a running plaintext release and asserts the refusal: the HelmRelease upgrade fails with the schema's message, the StatefulSet keeps its revision with both replicas Ready, plaintext still answers and no TLS listener appears.

Certificate renewal rolls the pods from inside the operator. Redis and Sentinel read the certificate once at startup, so the patched operator stamps a hash of the TLS Secret's tls.crt and ca.crt into the pod template of both the Redis StatefulSet and the Sentinel Deployment on every reconcile, as redis-failover.freshworks.com/tls-secret-hash. A renewal changes the hash; the operator's own roller replaces the Redis pods one at a time and the Deployment controller rolls Sentinel. README names the fallback: if the roll cannot run, the deadline is the old certificate's expiry, past which the operator's own TLS connections fail along with the clients.

tls.authClients maps to tls-auth-clients and defaults to no. The platform mints no client certificate for the tenant and the CA key is out of reach, so yes is only usable when the client certificate comes from whoever also controls the CA. Changing the value on a running release restarts the Redis and Sentinel pods: Redis reads the directive from its config file only at startup, so the operator folds the effective value into the certificate hash that drives the rollout, and the change lands when it is made rather than at the next unrelated restart.

Follow-ups in the carried operator patch go upstream first, to freshworks-oss/redis-operator#85 or a successor there, and are picked up here afterwards; the carried patch does not grow Go that upstream has not seen. tlsConfigFor fetches the Secret on every call, tens of GETs per reconcile. The renewal roll is static analysis so far: the live check, cmctl renew on a running release while watching the rfr-* and rfs-* pods roll, is still owed and belongs with the e2e follow-up.

Two facts about the shipped image rather than the patch. packages/system/redis-operator/values.yaml keeps the digest of the image built before the patch until release preparation rebuilds it, so on main between this merge and the next release-prep, tls.enabled renders the chain and spec.tls while the running operator ignores them and serves plaintext. And the pinned v3.3.5 tarball carries GO-2026-5026 in golang.org/x/net v0.53.0, reached through PodService.UpdatePodLabels; the port neither introduces nor fixes it, upstream master is already on v0.55.0, so it lands with the next tag bump rather than here.

Outside the patch: cozyvalues-gen renders {} in the README Value column for tls.authClients: a child property without a default inherits the parent object's {}, and make generate reproduces it byte for byte, so it is a generator bug rather than something to hand-edit here.

Screenshots

No UI changes.

Downstream repositories

I walked the trigger map against the diff file by file. terraform-provider-cozystack is reached, tracked as cozystack/terraform-provider-cozystack#37 rather than a speculative PR: values.schema.json gains a tls object, and the provider hand-writes a schema, model and expand/flatten pair per Kind. It already models tls for postgres, kafka, nats and qdrant, not for redis. Redis needs different wording from those four, whose descriptions say to omit the block to follow external, which is the one thing redis does not do.

Nothing else is reached. The website regenerates this package's reference page from README.md on a stable tag, so the README changes need no PR there. No package added, renamed or removed. packages/core/platform/values.yaml, variant and bundle composition and release asset names are untouched, and so are hack/package.mk, hack/common-envs.mk and hack/update-crd.sh, which is what ccp and external-apps-example read. The two new files under hack/ are bats unit tests. No node prerequisite, installer value, telemetry metric or cozy-proxy label changes.

Release note

feat(redis): add opt-in TLS for Redis and Sentinel. Set `tls.enabled: true` to serve both on a TLS-only listener backed by a per-release cert-manager chain; the tenant reads the CA certificate as the `<release>.tenant-ca` tenant secret. TLS is off unless set and is not inferred from `external`, so existing releases keep serving plaintext and are not restarted by the rebuilt operator image. TLS is chosen at creation: a running instance cannot switch modes, the change is rejected and surfaces as a failed HelmRelease upgrade, so add TLS by creating a new instance and migrating to it. Requires cert-manager, which the platform's `system` bundle installs on every variant. A renewed certificate rolls the Redis and Sentinel pods through the operator. TLS support is carried as a downstream patch over redis-operator v3.3.5 until freshworks-oss/redis-operator#85 lands in a release.

@github-actions github-actions Bot added area/database Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse) kind/feature Categorizes issue or PR as related to a new feature size/XL This PR changes 500-999 lines, ignoring generated files labels May 25, 2026
@coderabbitai

coderabbitai Bot commented May 25, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds TLS to Redis: new API types and deepcopy, Helm helpers and templates for tri-state tls.enabled and authClients, cert-manager resources and SANs, RedisFailover/dashboard bindings, chart values/schema/docs, tests, and an operator image bump to v1.4.0.

Changes

Redis TLS Feature

Layer / File(s) Summary
TLS API contracts and values schema
api/apps/v1alpha1/redis/types.go, packages/apps/redis/values.yaml, packages/apps/redis/values.schema.json, packages/system/redis-rd/cozyrds/redis.yaml, packages/apps/redis/README.md
ConfigSpec gains a Tls field; new TLS struct defines AuthClients and Enabled (pointer bool); Helm values/schema, OpenAPI keysOrder, and README docs updated.
TLS template helpers
packages/apps/redis/templates/_tls.tpl
redis.tls.enabled implements tri-state logic (fallback to external when unset, null treated as unset); redis.tls.authClients defaults to no.
Cert-manager certificate generation
packages/apps/redis/templates/certmanager.yaml
Renders self-signed bootstrap Issuer, CA Certificate/Issuer, and leaf Certificate with dynamic SANs (localhost, service DNS variants, headless wildcard, optional external host) when TLS enabled.
RedisFailover TLS and dashboard RBAC
packages/apps/redis/templates/redisfailover.yaml, packages/apps/redis/templates/dashboard-resourcemap.yaml
Adds conditional spec.tls to RedisFailover (enabled/authClients/secret refs) and conditionally includes TLS secret names in dashboard RBAC.
TLS integration test suite
packages/apps/redis/tests/tls_test.yaml
Adds Helm tests covering tri-state behavior, cert-manager outputs, SAN generation, tls.authClients propagation, and cluster-domain variations.
Autogenerated deepcopy updates
api/apps/v1alpha1/redis/zz_generated.deepcopy.go
DeepCopy implementations updated/added to copy the new Tls field and the optional Enabled pointer.

Redis Operator Version Update

Layer / File(s) Summary
Operator image and version update
packages/system/redis-operator/images/redis-operator/Dockerfile, packages/system/redis-operator/values.yaml
Builder/runtime base images and default operator VERSION updated; patch application step removed; Helm values image tag bumped to v1.4.0.

Sequence Diagram(s)

sequenceDiagram
  participant Values as Helm values
  participant Helm as Chart templates
  participant CertManager as cert-manager
  participant RedisFailover as RedisFailover CR
  participant Kubernetes as Kubernetes API
  Values->>Helm: tls.enabled / tls.authClients
  Helm->>RedisFailover: conditional spec.tls (secret refs, authClients)
  Helm->>CertManager: render Issuer / CA / Certificate when tls.enabled == "true"
  CertManager->>Kubernetes: Issuer/Certificate resources created
  Kubernetes->>RedisFailover: secrets (cert + key + ca) become available
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

size/XXL, area/platform

Suggested reviewers

  • lexfrei
  • kvaps
  • IvanHunters
  • sircthulhu
  • myasnikovdaniil

🐰 I hopped through charts and templates bright,
I stitched up TLS beneath moonlight,
Certs and SANs now dance in tune,
Operator bumped — secure as June —
Hop, hop, encrypt the night!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR fully implements all coding objectives from issue #2662: operator fork integration with TLS support, chart TLS configuration, tri-state tls.enabled behavior, cert-manager integration, RedisFail…
Out of Scope Changes check ✅ Passed All changes directly support the stated objectives: operator Dockerfile/image update, chart TLS templates, cert-manager resources, schema updates, RBAC adjustments, and comprehensive TLS test coverage…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the primary change: adding TLS support for Redis and Sentinel. The phrase "opt-in" is slightly imprecise because TLS is also implied when external is true, but the title…
Full details: Linked Issues check

Explanation

The PR fully implements all coding objectives from issue #2662: operator fork integration with TLS support, chart TLS configuration, tri-state tls.enabled behavior, cert-manager integration, RedisFailover CRD TLS fields, and comprehensive test coverage.

Full details: Out of Scope Changes check

Explanation

All changes directly support the stated objectives: operator Dockerfile/image update, chart TLS templates, cert-manager resources, schema updates, RBAC adjustments, and comprehensive TLS test coverage. No unrelated changes detected.

Full details: Title check

Explanation

The title clearly identifies the primary change: adding TLS support for Redis and Sentinel. The phrase "opt-in" is slightly imprecise because TLS is also implied when external is true, but the title remains directly related and sufficiently specific.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/tls-redis

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
packages/apps/redis/tests/tls_test.yaml (1)

279-305: ⚡ Quick win

Add dashboard RBAC assertion for explicit TLS disable override.

Line 279 case coverage skips dashboard-resourcemap.yaml for external=true + tls.enabled=false. Add a notContains check there to lock the tri-state behavior end-to-end.

✅ Proposed test addition
   - it: "redisfailover.yaml has no spec.tls when tls.enabled=false overrides external=true"
     templates:
       - templates/redisfailover.yaml
     set:
       external: true
       tls:
         enabled: false
     documentSelector:
       path: kind
       value: RedisFailover
     asserts:
       - notExists:
           path: spec.tls

+  - it: "dashboard-resourcemap.yaml omits TLS secrets when tls.enabled=false overrides external=true"
+    templates:
+      - templates/dashboard-resourcemap.yaml
+    set:
+      external: true
+      tls:
+        enabled: false
+    documentSelector:
+      path: kind
+      value: Role
+    asserts:
+      - notContains:
+          path: rules[1].resourceNames
+          content: "my-redis-tls"
+      - notContains:
+          path: rules[1].resourceNames
+          content: "my-redis-ca-tls"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/apps/redis/tests/tls_test.yaml` around lines 279 - 305, Add a test
case for templates/dashboard-resourcemap.yaml that mirrors the existing cases
for certmanager.yaml and redisfailover.yaml when external: true and tls.enabled:
false; specifically, under the same "Case 3" inputs (external: true,
tls.enabled: false) add a test entry for templates/dashboard-resourcemap.yaml
and assert with notContains (or notExists) that any TLS-related keys/entries are
absent (e.g., no tls, no tls.enabled, no tls.* fields) so the tri-state override
is validated end-to-end for dashboard-resourcemap.yaml.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/apps/redis/values.yaml`:
- Around line 80-83: The values schema for TLS is missing an enum constraint for
tls.authClients, so invalid strings pass chart-value validation; update the
values.yaml typedef for TLS (the TLS struct / tls.authClients entry) to add an
enum with allowed values ["no","optional","yes"] (and a default if needed) so
validation fails early; ensure the field name tls.authClients and the TLS
typedef are the ones updated so downstream RedisFailover.spec.tls.authClients
receives only valid values.

In `@packages/system/redis-operator/images/redis-operator/Dockerfile`:
- Line 3: The Dockerfile sets ARG VERSION=v1.4.0 and later downloads
https://github.com/cozystack/redis-operator/archive/refs/tags/${VERSION}.tar.gz
which 404s because that tag isn't published; either publish the v1.4.0 tag in
the cozystack/redis-operator repo or update the Dockerfile's ARG VERSION to a
valid existing tag (or a branch like main) so the tarball URL resolves; change
the value of VERSION in the Dockerfile (or adjust the download URL) and verify
the tarball URL resolves before merging.

---

Nitpick comments:
In `@packages/apps/redis/tests/tls_test.yaml`:
- Around line 279-305: Add a test case for templates/dashboard-resourcemap.yaml
that mirrors the existing cases for certmanager.yaml and redisfailover.yaml when
external: true and tls.enabled: false; specifically, under the same "Case 3"
inputs (external: true, tls.enabled: false) add a test entry for
templates/dashboard-resourcemap.yaml and assert with notContains (or notExists)
that any TLS-related keys/entries are absent (e.g., no tls, no tls.enabled, no
tls.* fields) so the tri-state override is validated end-to-end for
dashboard-resourcemap.yaml.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 5630a2fc-02a7-4ff2-9251-f8cd7e21fe1c

📥 Commits

Reviewing files that changed from the base of the PR and between 1810263 and a76c166.

📒 Files selected for processing (14)
  • api/apps/v1alpha1/redis/types.go
  • packages/apps/redis/README.md
  • packages/apps/redis/templates/_tls.tpl
  • packages/apps/redis/templates/certmanager.yaml
  • packages/apps/redis/templates/dashboard-resourcemap.yaml
  • packages/apps/redis/templates/redisfailover.yaml
  • packages/apps/redis/tests/tls_test.yaml
  • packages/apps/redis/values.schema.json
  • packages/apps/redis/values.yaml
  • packages/system/redis-operator/charts/redis-operator/crds/databases.spotahome.com_redisfailovers.yaml
  • packages/system/redis-operator/images/redis-operator/Dockerfile
  • packages/system/redis-operator/images/redis-operator/patches/labels.diff
  • packages/system/redis-operator/values.yaml
  • packages/system/redis-rd/cozyrds/redis.yaml
💤 Files with no reviewable changes (1)
  • packages/system/redis-operator/images/redis-operator/patches/labels.diff

Comment thread packages/apps/redis/values.yaml
Comment thread packages/system/redis-operator/images/redis-operator/Dockerfile Outdated
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces native, operator-managed TLS support for Redis within the Cozystack environment. By migrating to a custom redis-operator fork, the change enables secure replication and client traffic encryption. It leverages cert-manager for automated certificate lifecycle management and ensures backward compatibility by defaulting to existing behaviors while providing flexible opt-in mechanisms for TLS.

Highlights

  • Operator Migration: Migrated to the cozystack/redis-operator fork (v1.4.0) to enable native TLS support.
  • TLS Configuration: Implemented a tri-state TLS configuration (enabled, disabled, or auto-enabled via external access) for Redis and Sentinel.
  • Automated Certificate Management: Integrated cert-manager to provision the CA chain and leaf certificates automatically.
  • Testing: Added 21 new helm-unittest cases to verify TLS resolution, SAN coverage, and RBAC settings.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files
  • Ignored by pattern: **/*.diff (1)
    • packages/system/redis-operator/images/redis-operator/patches/labels.diff
  • Ignored by pattern: **/charts/** (1)
    • packages/system/redis-operator/charts/redis-operator/crds/databases.spotahome.com_redisfailovers.yaml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces TLS support for the Redis application, incorporating API updates, Cert-manager resource templates, and a helper for tri-state TLS logic. It also updates the redis-operator to version 1.4.0 and switches to a internal fork. Review feedback identifies critical issues in the Dockerfile where non-existent Go and Alpine versions are specified, which will cause build failures. Additionally, it is recommended to enhance the values.schema.json by adding an enumeration and default value for the authClients field to improve validation and the dashboard user experience.

Comment thread packages/system/redis-operator/images/redis-operator/Dockerfile Outdated
Comment thread packages/system/redis-operator/images/redis-operator/Dockerfile Outdated
Comment thread packages/apps/redis/values.schema.json

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOT LGTM — same B2/B3-class issues as mariadb #2680: the chart-managed cert-manager chain exposes both the CA and leaf private keys to tenant use access via dashboard RBAC, and the ECDSA leaf certificate is missing the digital signature usage so strict TLS clients can reject the handshake. Plus a build blocker: the Dockerfile downloads a fork tag that doesn't exist yet.

Business context: this PR shifts Redis TLS to a forked redis-operator (cozystack/redis-operator Phase 2+3 of #2662), with the chart rendering a cert-manager-issued chain and handing the leaf Secret to the operator via spec.tls.certificateSecret.secretName. The fork side is being developed in cozystack/redis-operator#1. The findings below are independent of the fork — they apply to the chart-side TLS chain as rendered.

Blockers

B1: dashboard-resources Role grants tenant use read on both <release>-tls and <release>-ca-tls — leaf AND CA private keys leaked

File: packages/apps/redis/templates/dashboard-resourcemap.yaml:24-25
Issue: when $tlsEnabled is true, the Role grants get/list/watch on <release>-tls and <release>-ca-tls. cert-manager writes both Secrets as kubernetes.io/tls with tls.crt + tls.key. <release>-tls carries the leaf server private key. <release>-ca-tls carries the CA private key.
Impact: at use access level any tenant principal in the namespace can extract:

  • the Redis server's leaf private key → impersonate the Redis server, terminate TLS as rfrm-<release>, MITM other clients, decrypt past sessions for non-PFS ciphers;
  • the CA's private key → mint arbitrary certificates trusted by anyone using this CA bundle (any sentinel / replication peer / external client) → full forge-and-MITM across the cluster's Redis trust domain.

Same class as the #2680 mariadb blocker. The two-Secret exposure is strictly worse than the single-CA case.
Fix: same as mariadb's: stop exposing <release>-tls and <release>-ca-tls via the chart's dashboard-resources Role. If the dashboard genuinely needs to surface connection metadata, project a ca.crt-only Secret (e.g. via cert-manager's secretTemplate with the private-key field stripped, or a tiny init job) and expose THAT. Tenants needing the leaf or CA for break-glass debugging can still use kubectl out-of-band; routine clients should resolve trust via the projected CA-only Secret.

B2: leaf ECDSA certificate is missing the digital signature usage

File: packages/apps/redis/templates/certmanager.yaml:65-68
Issue: leaf cert usages: [server auth, client auth, key encipherment]. The cert uses ECDSA (privateKey.algorithm: ECDSA, size: 256). TLS_ECDHE_ECDSA — the only meaningful TLS handshake for an ECDSA cert — requires the cert's digital signature (alias signing in cert-manager — see pkg/api/util/usages.go) bit so the server can sign the key-exchange parameters.
Impact: strict TLS clients (Go's crypto/tls, OpenSSL with purpose check) reject the cert when it lacks digitalSignature for ECDHE_ECDSA. The handshake fails with x509: certificate specifies an incompatible key usage or equivalent. Symptom: redis-cli with --tls may work depending on its OpenSSL version, but stricter clients (Go, Java with strict purpose checking, mTLS-aware tooling) fail.

Same finding as the #2680 mariadb leaf cert.
Fix: add - digital signature (or - signing) to the leaf cert's usages block. The CA cert at line 35-37 has the correct signing + cert sign already; the leaf one is the one to fix.

B3: Dockerfile downloads cozystack/redis-operator/v1.4.0 which currently returns 404 — image is unbuildable

File: packages/system/redis-operator/images/redis-operator/Dockerfile:12
Issue: RUN curl -sSL https://github.com/cozystack/redis-operator/archive/refs/tags/v1.4.0.tar.gz | tar -xzvf- --strip=1 — the tag doesn't exist. Verified: HTTP/2 404 against the tarball URL; gh api repos/cozystack/redis-operator/tags returns an empty list; the fork PR cozystack/redis-operator#1 (feat/tls-cert-manager) is still open. Out of scope per the PR body, but the chart side currently references a non-existent artifact, so anyone running make image will fail before compilation starts.
Impact: image build fails everywhere — CI for any cozystack PR that touches redis (including this one's Resolve assets / Build pipelines), local dev, fresh installs. The values.yaml:image.tag = v1.4.0 reference will also fail digest resolution when fetched.
Fix: gate this PR on the fork tag being cut (i.e. merge cozystack/redis-operator#1, tag v1.4.0, publish the image to ghcr) BEFORE merging here. Alternatively, pin to a commit SHA on the fork's feat/tls-cert-manager branch as a placeholder while the bump sequence is resolved — but the current v1.4.0 reference is dead.

Non-blocking follow-ups

  1. Same schema array-type issue as #2686/#2692: "type":["boolean","null"] for tls.enabled (and tls.authClients if it inherits the pattern) in values.schema.json / cozyrds openAPISchema fails to unmarshal into apiextv1.JSONSchemaProps (single-string Type field). cozystack-api silently disables specSchema-based defaulting for this app; Helm-side defaults still apply, so user-visible breakage is minimal. Worth a separate fix across the whole TLS series — nullable: true (apiextv1 idiom) or drop type for tri-state properties.

  2. templates/certmanager.yaml:24 and 47: CA cert secretName and CA Issuer's ca.secretName both reference <release>-ca-tls. Once B1 is resolved, the same change (CA private key not exposed) needs to be reflected in cozyrds secrets.include if it lists -ca-tls. Worth double-checking when fixing B1.

  3. The PR body explicitly says "digest will be repinned by make image once CI publishes the release artifact". Make sure that step lands in this PR (or a follow-up) BEFORE merging — otherwise the chart references an image that doesn't exist.

  4. README workflow currently directs users to the dashboard / <release>-tls Secret for client connection. After the B1 fix lands, README needs to redirect to whatever CA-only Secret replaces it.

@Arsolitt

Copy link
Copy Markdown
Contributor Author

Addressed B2 and the series-wide schema cleanup. B3 (image tag) is held by an unmerged upstream change.

Blockers:

  • B1 (dashboard RBAC exposes <release>-tls and <release>-ca-tls with private keys) — accepted as a series-wide tradeoff for this iteration. Same decision as feat(mariadb): add TLS support via cert-manager #2680. To revisit with a cluster-wide ca-bundle pattern.
  • B2 (leaf cert missing digital signature) — fixed in f8f9b77cd. Added to the leaf usages list. The leaf is ECDSA P-256, so this bit is required by the TLS_ECDHE_ECDSA handshake (signing in cert-manager terminology).
  • B3 (Dockerfile references cozystack/redis-operator/v1.4.0 which isn't tagged yet) — out of scope for this PR. The fork's feat/tls-cert-manager branch needs to merge and tag v1.4.0 before the image becomes buildable. Will rebase and refresh image tag once that lands.

Non-blocking follow-ups:

  • FU1 (["boolean","null"] schema) — addressed previously in this branch (committed before this re-review round).
  • FU2 (cozyrds will need realignment after B1 fix) — tied to B1 decision, no change.
  • FU3 (image digest repinning) — deferred together with B3.
  • FU4 (README CA retrieval path) — deferred together with B1.

Ready for re-review on the chart-side changes; B3 remains the block for merge.

@lexfrei Aleksei Sviridkin (lexfrei) left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOT LGTM — the chart-side TLS is solid in shape, but it depends on three operator-side artifacts that do not exist yet: an unbuildable image tag, an unmerged CA-only-secret feature, and a missing loopback-IP SAN set. Each is independently merge-blocking.

Business context: adds operator-managed Redis TLS by switching to the cozystack/redis-operator fork and rendering a cert-manager CA+leaf chain handed to the operator, matching the DBaaS-wide TLS convention.

Blockers

B1: the redis-operator image references a tag that does not exist — unbuildable
File: packages/system/redis-operator/images/redis-operator/Dockerfile:3,12, packages/system/redis-operator/values.yaml:4
Issue: the Dockerfile downloads the v1.4.0 source tarball and values.yaml pins tag: v1.4.0, but no such tag exists in the fork.
Evidence: cozystack/redis-operator has no tags, no releases; the v1.4.0 tarball URL returns HTTP 404; the ghcr package was never published. The root Makefile now wires make -C packages/system/redis-operator image into build, so the whole platform build fails on this 404 before compilation.
Impact: make image / make build fails in CI, local dev, and fresh installs; values.yaml tag also fails digest resolution.
Fix: cut and push a release tag on the fork, publish the image to ghcr, then repin values.yaml to a digest — within this PR.

B2: the chart depends on an operator field (spec.tls.caCertSecretName) absent from the operator's released/master code
File: packages/apps/redis/templates/redisfailover.yaml:34, packages/apps/redis/templates/dashboard-resourcemap.yaml:29
Issue: the RBAC private-key leak fix switches the dashboard Role to grant only <release>-ca-cert (a CA-only secret) and sets spec.tls.caCertSecretName: <release>-ca-cert so the operator publishes it. But caCertSecretName exists only in the fork's open, unmerged PR (feat/tls-ca-cert-secret), not in master.
Evidence: caCertSecretName has zero occurrences in the fork's master; it appears only on the open feature branch. The CRD vendored in this PR already carries the field — meaning it was vendored from the open branch, not master. An operator built from any released or master code will never create <release>-ca-cert.
Impact: with the real operator, <release>-ca-cert is never published — the dashboard Role grants on a non-existent secret and the operator silently ignores the unknown CRD field. The headline security fix only works against an operator that has not been merged or released; the vendored CRD diverges from the operator binary it ships with.
Fix: merge the fork's CA-secret PR, fold it into the tagged release this PR pins, and vendor the CRD from that same release point so chart + CRD + binary are consistent.

B3: the chart-issued leaf cert is missing loopback IP SANs (127.0.0.1 / ::1)
File: packages/apps/redis/templates/certmanager.yaml:70-83
Issue: when the chart supplies its own cert via spec.tls.certificateSecret, the operator stops generating the SAN set. The chart leaf lists localhost as a DNS SAN only and sets no ipAddresses, but the operator's own components dial over loopback IP with full CA verification.
Evidence: the operator's generator adds ipAddresses: ["127.0.0.1", "::1"] precisely because redis_exporter is given REDIS_ADDR=rediss://127.0.0.1:26379 with REDIS_EXPORTER_TLS_CA_CERT_FILE (verification active, not skip-verify) and the sentinel monitor targets 127.0.0.1. A localhost DNS SAN does not match an IP literal in Go/openssl cert verification.
Impact: with chart-managed TLS on, the redis_exporter metrics scrape over rediss://127.0.0.1:26379 fails cert verification and the exporter/monitoring path breaks. Probes using -h localhost match the DNS SAN, so the failure is partial and easy to miss in a smoke test.
Fix: add an ipAddresses: block with 127.0.0.1 and ::1 to the leaf Certificate, mirroring the operator's own SAN generation.

Non-blocking follow-ups

  1. On the authClients=yes path the chart grants tenants get on <release>-tls (the leaf, which carries the private key) so they can reuse it as a client identity (packages/apps/redis/templates/dashboard-resourcemap.yaml:34). The comment acknowledges a dedicated per-tenant client cert would be the stricter long-term design — worth a follow-up issue.
  2. README tls.authClients default renders {} instead of no (packages/apps/redis/README.md:36) — a cozyvalues-gen artifact; cosmetic.

Comment thread packages/system/redis-operator/images/redis-operator/Dockerfile Outdated
Comment thread packages/apps/redis/templates/redisfailover.yaml Outdated
Comment thread packages/apps/redis/templates/certmanager.yaml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — re-review on head 3e24931b. All three blockers from the prior review are resolved.

Business context: adds operator-managed Redis TLS by switching to the cozystack/redis-operator fork and rendering a cert-manager CA+leaf chain handed to the operator, matching the DBaaS-wide TLS convention.

Blockers — resolved

  1. Unbuildable image tag — RESOLVED. The fork now has a v1.4.0 tag and release; the source tarball the Dockerfile downloads returns 200, so make image / make build no longer fails. values.yaml and the Dockerfile ARG VERSION both pin v1.4.0 consistently.
  2. spec.tls.caCertSecretName absent from the operator — RESOLVED. The fork's CA-only-secret PR is merged and folded into the v1.4.0 tag (the tag's tree carries CACertSecretName in the API types). An operator built from v1.4.0 will publish <release>-ca-cert, so the dashboard RBAC grant now points at a secret the operator actually creates, and the vendored CRD matches the operator binary.
  3. Missing loopback IP SANs — RESOLVED. The leaf certificate now sets ipAddresses: ["127.0.0.1", "::1"] (on the <release>-tls leaf secret handed to the operator, isCA: false), with a comment explaining the redis_exporter rediss://127.0.0.1 CA-verification path. DNS SANs still cover all service forms plus the external host.

Verified: helm-unittest passes (23/23, including the TLS suite); make generate shows no drift; the leaf cert usages include digital signature + server auth + client auth; key rotation is Always with renewBefore: 720h.

Non-blocking follow-ups

  1. On the authClients=yes path the chart grants tenants get on <release>-tls (the leaf, which carries the private key) so they can reuse it as a client identity (packages/apps/redis/templates/dashboard-resourcemap.yaml:34). The comment acknowledges a dedicated per-tenant client cert would be the stricter long-term design — worth a follow-up issue.
  2. README tls.authClients default renders {} instead of no (packages/apps/redis/README.md:36) — a cozyvalues-gen artifact; cosmetic.

Aleksei Sviridkin (lexfrei) added a commit that referenced this pull request Jun 23, 2026
## What this PR does

Part of #2814. Part of #2811.

This is the foundational ("keystone") piece of WS3: a canonical,
reusable trust-anchor object that lets a tenant obtain `ca.crt` to
verify a per-app TLS endpoint **without** read access to any object that
also carries a private key. It unblocks the per-app TLS series (WS4/WS5)
by giving every chart one agreed, key-free delivery shape instead of
each chart inventing its own.

### The problem (verified against current `main`)

Per-app TLS issues a per-release, self-signed CA. The only objects that
hold `ca.crt` today also hold private keys:

- the cert-manager CA Secret `<release>-ca` carries the CA private key
(`tls.key`) — full trust-chain compromise if leaked
(`packages/apps/nats/templates/certmanager.yaml`, the `isCA`
Certificate);
- the cert-manager leaf Secret `<release>-tls` carries the server
private key (`tls.key`) plus `ca.crt` (same file, the leaf Certificate);
- CNPG surfaces `ca.crt` only inside its user-credentials Secret
(`packages/apps/postgres/templates/db.yaml`, the operator-managed TLS
block).

So any RBAC path that hands a tenant `ca.crt` by granting read on one of
those Secrets also hands over a private key. On `main` today the
cert-manager apps grant tenants no access to those Secrets at all, so a
tenant currently cannot obtain `ca.crt` to verify the server — the gap
this object closes safely.

### Design decision: key-free Opaque object delivered through
`tenantsecrets`, not trust-manager

I evaluated the two candidate delivery mechanisms.

**Rejected — trust-manager.** A trust-manager `Bundle` reads its sources
only from trust-manager's single configured "trust namespace"; its
`namespaceSelector` governs only where the bundle is *distributed to*,
not where it is *sourced from*. A per-release self-signed CA lives in
the tenant namespace, so trust-manager cannot use it as a source, and
projecting one shared CA to all tenants would defeat per-release
isolation. I re-verified this against current upstream docs
(cert-manager.io trust-manager) and the open per-namespace-trust-bundle
request (cert-manager/trust-manager#131, which is target-side only). The
prior analysis still holds.

**Chosen — a `ca.crt`-only Opaque Secret surfaced through the existing
tenant-secret API.** This generalizes the reference pattern from the
redis work (#2729), where the operator publishes a CA-only Opaque Secret
and RBAC is scoped to it instead of to the CA-private-key Secret. The
new `cozy-lib.tls.caCertSecret` helper renders that object once,
identically, for every chart:

- `type: Opaque`, a single `ca.crt` data key, no `tls.key`/`tls.crt`;
- label `internal.cozystack.io/tenantresource: "true"`;
- fails closed on empty input and refuses any PEM that contains private
key material.

### How the RBAC wiring works (no new roles needed)

The label is the entire mechanism, and it routes through RBAC that
already exists:

- `pkg/registry/core/tenantsecret/rest.go` surfaces, under the virtual
resource `core.cozystack.io/tenantsecrets`, exactly the namespace
Secrets bearing `internal.cozystack.io/tenantresource=true`
(`buildTenantSelector`, lines 199-213; label constants
`pkg/apis/core/v1alpha1/tenantresource_types.go:3-4`; full `Data` copied
through by `secretToTenant`, lines 84-103).
- `packages/system/cozystack-basics/templates/clusterroles.yaml` grants
`get/list/watch` on `core.cozystack.io/tenantsecrets` to tenant
ServiceAccounts via `cozy:tenant:base` (lines 44-47) and to
`use`/`admin`/`super-admin` subjects via `cozy:tenant:use:base` (lines
179-180). None of this touches raw core `secrets`, so attaching the
label to a key-free object exposes the trust anchor and nothing else.
- `internal/backupcontroller/credentials_projector.go:128-133` relies on
the same rule in reverse — it deliberately *omits* the label so its
key-bearing projection is not promoted to a `TenantSecret`. This helper
is the positive counterpart: a key-free object that is safe to promote.

`view` deliberately does not receive the trust anchor through this path:
`cozy:tenant:view:base` grants only `core.cozystack.io/options` (lines
120-126), and `tenantsecrets` also includes credential Secrets, so
granting it to a read-only role would leak passwords. The trust anchor
reaches `use` and above plus tenant ServiceAccounts — the same access
level at which connection credentials are already surfaced (the
per-release dashboard Role binds at `use`, e.g.
`packages/apps/postgres/templates/dashboard-resourcemap.yaml` via
`cozy-lib.rbac.subjectsForTenantAndAccessLevel`). A chart that must show
`ca.crt` at `view` level in the dashboard can additionally name the
key-free Secret in its own per-release Role; that is a per-app detail,
not a base-role change.

### Why a library helper and not a controller here

Populating `ca.crt` stays the responsibility of whatever owns the PKI —
the app operator (as the redis fork does) or a cert-manager chain
resolved at the chart level. The helper is intentionally pure and
value-driven so it renders deterministically and is the single shape
every per-app PR converges on. A shared key-stripping projection
controller is a reasonable future generalization but is out of scope for
this foundational object.

### Convergence plan for the per-app TLS series (follow-up, not this PR)

I did not modify any of the open per-app branches. Each should adopt the
canonical object as follows:

- redis #2729 — already publishes an operator-side CA-only Opaque Secret
and already drops the grant on the CA-private-key Secret; converging
means emitting it through the helper / carrying the `tenantresource`
label so delivery matches the rest of the catalog.
- nats, qdrant (merged, cert-manager) — emit the canonical object via
the helper and withhold `<release>-ca` / `<release>-tls` from tenants.
- mariadb #2680, opensearch #2682, rabbitmq #2683, mongodb #2692 — same:
emit the canonical object, never grant a Secret that contains `tls.key`.

### Tests

`helm-unittest` in the cozy-lib test chart (`make -C
packages/tests/cozy-lib-tests test`) covers: the rendered object is
`Opaque` with `ca.crt` and no `tls.key`/`tls.crt`; the `tenantresource`
label is present; caller labels/annotations merge while the security
label always wins; the helper fails closed on empty `caCert`; and it
refuses a PEM that carries private key material.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added a Helm helper to generate CA-only Kubernetes TLS trust-anchor
secrets, including input validation (rejects private key material,
requires a properly formatted certificate, and ensures required
metadata).

* **Tests**
* Introduced a dedicated test suite covering successful rendering,
metadata and content assertions, tenant label enforcement,
label/annotation merging, and detailed failure cases for invalid inputs
and incorrect arguments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
…main lookup

Two defects in the generated ApplicationDefinition and the certificate
template.

The regenerated openAPISchema had lost the x-cozystack-options extension
on storageClass that values.schema.json still carries, which is what
drives the storage class picker in the dashboard form. The schema was
generated with cozyvalues-gen 1.5.0 while CI pins 1.6.0, and the older
release drops the extension. Regenerated with the pinned version, which
restores it.

The cluster-domain lookup indexed .Values._cluster directly, so a render
where the platform injects no _cluster map failed with "index of untyped
nil" before the documented cozy.local fallback could apply. Default the
parent map so the fallback is actually reachable, and extend the test to
cover an absent map as well as an empty domain — the previous test only
exercised a domain set to the empty string, which never reached the
failing path.

Signed-off-by: Aleksei Sviridkin <[email protected]>
Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
…host

The external SAN dereferenced .Values._namespace.host directly, so a
render without that injected value aborted on a nil pointer with no
indication of what was missing. TLS now defaults on whenever external is
true, which puts that lookup on the default path for every externally
reachable Redis.

Wrap it in required with an explicit message, matching how the nats chart
guards the same lookup in its own certificate template. Failing is the
right behaviour rather than dropping the SAN: a certificate issued
without the external hostname is accepted by nothing outside the cluster
and the resulting handshake error points nowhere near the cause.

Signed-off-by: Aleksei Sviridkin <[email protected]>
Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
…ections

TLS replaces plaintext in this operator rather than running beside it —
with TLS on, Redis and Sentinel are configured with port 0 and serve only
on the TLS port. Since tls.enabled falls back to external, an already
deployed instance with external: true switches to TLS-only on its first
reconcile after the upgrade and every plaintext client breaks. That was
documented as a feature and nowhere as a change of behaviour for existing
instances. State it as a breaking change and name tls.enabled: false as
the opt-out.

Also spell out which hostname the certificate actually covers. The SAN
list carries the four service names and the external tenant hostname, and
its only IP entries are the loopback addresses used by the in-pod probes
and the metrics sidecar, so connecting to the external LoadBalancer by IP
fails for clients that verify the hostname.

The TLS section marker sat above the version parameter, which put version
in the TLS table in the generated README instead of Common parameters.
Move the marker below it. Regenerating also reorders version ahead of tls
in keysOrder, matching how postgres orders the same two fields.

Pin the PackageSource CRD replacement policy the way the mongodb-operator
one is pinned. The RedisFailover CRD gained spec.tls with the fork, and
Helm never upgrades CRDs shipped in crds/, so losing CreateReplace would
leave the live CRD behind and the apiserver would strip spec.tls from
every RedisFailover the chart renders — TLS silently off, no error.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
…oller lands

The trust anchor was selected only by the internal.cozystack.io/tenant-ca
label, which nothing stamps until the CA-extraction controller merges. A
label-only entry selects nothing until then, so a TLS-enabled Redis gave
the tenant no way to obtain the certificate needed to verify the server —
the dashboard Role grants the Secret by name, but it never became a
tenant secret and so never appeared on the product surface.

Name the operator's CA-only Secret directly alongside the auth secret.
It holds ca.crt and no private key, so this widens tenant visibility by
exactly the object the tenant needs and nothing else. The label selector
stays for the converged path; once the controller publishes its
projection the anchor is reachable under both names.

Cover the two assertions the feature depended on and did not have:
caCertSecretName, which has to agree with both the dashboard Role and the
declared CA source, and the loopback IP SANs, added by their own fix
commit and until now asserted nowhere.

Correct the tri-state helper's comment. It claimed the kindIs "invalid"
guard covers an explicitly null tls.enabled; the schema types that field
as boolean and rejects null before any template runs. The guard covers
the reachable shape, a tls map with no enabled key, now tested.

Signed-off-by: Aleksei Sviridkin <[email protected]>
Assisted-By: Claude <[email protected]>
Redis and Sentinel read tls-cert-file and tls-key-file once at startup,
so a leaf renewed by cert-manager 30 days before its one-year expiry is
served only after the pods restart. The operator does that restart: on
every reconcile it reads the TLS Secret and stamps a hash of tls.crt and
ca.crt into the pod template of the Redis StatefulSet and the Sentinel
Deployment, so a renewal rolls Redis through the operator's own roller
(the StatefulSet is OnDelete; the roller replaces one pod per reconcile,
replicas before the master) and Sentinel through its Deployment.

The TLS section presented the setup as turnkey and said nothing about
renewal. Say what happens, and name the fallback: when the roll cannot
run, because the operator is down or its reconcile of this RedisFailover
keeps failing, the deadline is the old certificate's expiry, since past
it the operator's own TLS connections to Sentinel and Redis fail along
with the clients.

The CA has a clock of its own: five years, reissued on the same key
before the end, so a ca.crt copied once verifies new leaves only until
the original CA certificate expires. Say that too, and point at
re-reading the projection instead of baking the copy in.

A stakater reloader annotation is not an alternative here and is not
used: the operator regenerates the pod template on every reconcile, so
nothing written into it from outside survives to the next one.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The certificate covers <release>.<tenant-host> and the TLS section told
users to connect there, but the external Service is a plain LoadBalancer
with no external-dns annotation — unlike the opensearch external Service,
which annotates its hostname explicitly. Nothing points that name at the
LoadBalancer address, so the one documented external path does not work
as written, and the IP fallback fails hostname verification because the
certificate's only IP SANs are loopback.

Describe what is actually true and name the manual step needed to reach
the endpoint from outside. Publishing the name from the chart is the
better fix, but that is a platform-integration decision about which
hostname redis should claim, not a documentation change.

Also disambiguate the CA source comment. Source and destination are
different Secrets — the declared source is read, the projection is
written — and the previous wording put both names close enough together
to read as a contradiction.

The hostname-verification note names which clients it applies to:
redis-cli, the one the README shows, verifies the chain and not the
name, so it connects to a bare IP and hides the mismatch.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
Requiring _namespace.host turned a nil dereference into a hard render
failure for a shape that legitimately has no host. packages/apps/tenant
computes an empty host whenever neither the tenant nor its parent sets
one, and Helm's required rejects the empty string as readily as nil, so
an external Redis running in such a tenant would have stopped templating
altogether on upgrade — no object left to inspect and no error pointing
at the cause.

Emit the external SAN when a host exists and skip it when none does. A
tenant without a host has no external name to certify, so there is
nothing being silently dropped, and the in-cluster names still cover
every client that verifies inside the cluster. Reading through a
defaulted dict keeps an absent _namespace from dereferencing nil.

Replace the test that pinned the failure with two that pin rendering,
for an empty host and for an absent namespace map.

Add the coupling the CA-only secret name needs. Its name has to match
across the operator spec that publishes it and the tenant grant that
exposes it; renaming one alone leaves the tenant without a trust anchor
and produces no error anywhere.

Signed-off-by: Aleksei Sviridkin <[email protected]>
Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
TLS was enabled automatically whenever external was true. In this
operator TLS is not additive — the rendered config sets port 0 and moves
the listener to tls-port — so that inference turned every already-running
externally-reachable Redis into a TLS-only server on its first reconcile
after the upgrade and cut every plaintext client with it. A platform
upgrade must not sever working connections.

Resolve tls.enabled to false when unset, whatever external says. Enabling
TLS becomes an explicit act at creation time, and existing releases keep
the behaviour they have today.

Creation time, not any time: a running instance cannot change mode. A
replica restarted onto the TLS listener cannot replicate from a master
still on plaintext, and Sentinels already on TLS would fail over to a
stale replica, so the operator's RedisFailover schema rejects a change of
tls.enabled and the HelmRelease upgrade fails while the instance keeps
serving what it served before. The README says so and points at creating
a new instance and migrating.

Pin the decision directly: external: true on its own now has to leave the
cert-manager chain unrendered, the operator without spec.tls, and the
tenant without TLS secrets. Every case that previously reached TLS through
external says enabled: true instead, and the null and empty-map forms
resolve to off alongside the unset one.

With nothing inferred there is no upgrade-time behaviour change left, so
the breaking-change marker and footer are gone from this branch.

Say how to get out of it as well. The cozystack Redis API accepts the
change, so the rejection lands on the HelmRelease, and every later edit
to that instance queues behind the failure until tls.enabled goes back
to what it was.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
When external access is on the chart creates <release>-external-lb
itself, but the leaf certificate covered only the four Services the
operator creates. That name resolves in-cluster like any other Service,
so a client dialing it under TLS failed hostname verification against a
Service this chart owns. Add its four DNS forms next to the operator's,
gated on the same flag that creates it.

Also record at the image pin how the digest moves: it is refreshed by release preparation, not by this change, so an edit to the Dockerfile under images/ ships only once the next repin lands.

The README list of in-cluster names the certificate covers names the
external LoadBalancer Service as well.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The comment read as a justification and it was wrong. It described the
backstop as guarding against any Secret in the namespace acquiring the
tenant-ca label. The lineage webhook is not namespace-wide — it returns
early for any object whose ownership does not resolve to this release's
app CR — and the two Secrets listed do not even reach it today, because
the platform runs cert-manager with enableCertificateOwnerRef false and
leaves them without an ownerReference. The entries are defence in depth
against a future where that changes, not a gate that fires now. State
the scope rather than implying a broader guarantee.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
cozyvalues-gen quotes a default only when the declared type is the literal
`string` or `quantity`; a named alias, including one declared `@enum
{string}`, falls through to a YAML re-parse of the bare value where the 1.1
boolean words resolve. A chart spelling out a `"no"` default under such a
type emits `"default": false` into a string-typed, enum-constrained property
and `+kubebuilder:default:=false` onto the matching Go field. Templates read
values.yaml rather than the schema, so the chart suites stay green while the
shipped schema and CRD carry a default the field rejects.

Vendored schemas under packages/*/*/charts/ are left out of the scan:
cozyvalues-gen never writes them, they change only through `make update`,
and a violation there would have no fix inside this tree.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The generated README renders the tls.authClients default as `{}` because the
block carries no keys. Materialising the default to fix that column corrupts
the schema and the CRD, so note the trap at the line someone would edit.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The TLS section described the per-release cert-manager chain without
saying that cert-manager has to be present to issue it. It is present on
every system variant: the platform installs it in the variant-independent
body of the system bundle, so isp-hosted carries it as well. What
isp-hosted lacks is reloader, and the renewal section already covers
that gap.

The cert-manager sentence separates the two ways a cluster can lack it:
without the controller but with its CRDs the resources render and nothing
issues them, and without the CRDs the release fails on an unknown kind.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The tenant needs the release's CA certificate to verify the server, and
the platform's contract for handing one over is the key-free
<release>.tenant-ca projection: the CA-extraction controller lifts a
single named key out of a source Secret, refuses to publish anything
that parses as PEM private key material, and stamps
internal.cozystack.io/tenant-ca on what it writes. postgres, mongodb,
nats and qdrant all reach the tenant this way; redis is the fifth.

Source the chart's own cert-manager CA Secret, <release>-ca-tls, rather
than the CA-only Secret the operator publishes at
spec.tls.caCertSecretName. Both carry the same certificate, but
<release>-ca-tls exists as soon as cert-manager issues the CA, so the
tenant read path does not also depend on the operator having reconciled
the RedisFailover and written its copy. That the source holds the CA
private key beside ca.crt is the intended shape rather than a
compromise — projecting one key out of a key-bearing Secret is what the
controller is for, and it refuses private key material outright.

Gated on tls.enabled, because the source Secret only exists when TLS is
on. The test pins that the gate is tls.enabled and not external access:
an externally exposed plaintext release must render no sentinel, or the
controller waits forever on a Secret nothing creates.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The tenant read path was two grants on Secret names: the operator's
CA-only <release>-ca-cert in the dashboard Role, and the same name in the
ApplicationDefinition's include list. Replace both with the projection
the previous commit declares.

A name grant conveys whatever occupies the name. The operator holds
create and update on Secrets cluster-wide, so what sits at
<release>-ca-cert is the operator's choice rather than the platform's,
and RBAC has no key-level filter to catch it if that ever stops being
just ca.crt. <release>.tenant-ca is written only by the CA-extraction
controller, which copies one named key and refuses PEM private key
material outright. That is the difference between a name that happens to
hold a certificate today and provenance the platform vouches for, and it
is the reason postgres, mongodb, nats and qdrant all reach the tenant
this way — mongodb's operator publishes a CA Secret of its own too, and
that one is excluded rather than granted.

Also drop spec.caCert from the ApplicationDefinition. The
ApplicationDefinition CRD does not define that field and Flux applies
these server-side, so the apiserver rejects the whole object with "field
not declared in schema" and the redis-rd release never goes ready. It was
carried deliberately as a merge-order gate for a controller that reads
it; that controller landed as the TenantProjection reconciler, which
reads a sentinel in the app chart instead, so the gate has nothing left
to wait for.

hack/check-redis-rd-secrets.bats pins the whole rendered Secret surface
rather than probing for known-bad names, alongside the existing guards
for nats, qdrant and mongodb: an include with neither resourceNames nor
matchLabels restricts nothing, and a second matchLabels entry selecting
a label cert-manager also stamps on the key-bearing Secrets would grant
the CA private key while every name in the file stayed correct.

The connection example passes the generated password as well. Password
generation is on by default, so the command as printed connected and then
answered NOAUTH on the first real command.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
freshworks-oss/redis-operator v3.3.5 has no TLS support. The operator
work that packages/apps/redis renders against was ported upstream as
freshworks-oss/redis-operator#85 and is not in a release yet, so carry it
here over the v3.3.5 tarball until one lands.

Three pieces, and TLS is silently off unless all three are present:

- images/redis-operator/patches/tls.diff is the source half. The
  Dockerfile already applies patches/*.diff, and the shell expands that
  to labels.diff then tls.diff; they touch disjoint files, so the order
  does not matter, but it is alphabetically stable either way.
- patches/crd-tls.diff teaches the vendored CRD about spec.tls. Without
  it the apiserver prunes spec.tls from every RedisFailover the redis
  chart renders and the patched binary waits for a field it never sees.
- patches/rbac-tls.diff grants Create, Get and Update on Secrets, so the
  operator can publish the CA-only Secret named by
  spec.tls.caCertSecretName. Those are the only Secret calls in the
  patched source: upstream v3.3.5 only ever reads one, and the patch
  adds Create and Update on a Secret it addresses by name, so nothing
  lists, watches, patches or deletes.

tls.diff is cut from that PR's commit 2ad2ed8d1 rebased onto v3.3.5, and
its header pins that SHA so the next refresh can be diffed against it.
Beyond the TLS listener it carries the renewal roll: on every reconcile the operator stamps a hash of the TLS
Secret's tls.crt and ca.crt into the pod template of the Redis
StatefulSet and the Sentinel Deployment, so a renewed certificate rolls
Redis through the operator's own roller and Sentinel through its
Deployment. The effective spec.tls.authClients value is folded into
the same hash, so changing it rolls the pods the same way instead of
landing at the next unrelated restart, and the operator presents its
client certificate whenever the Secret carries one, independent of that
field, so neither direction of the change locks it out of the instance
it reconciles. It also mounts the TLS volume 0440 when the pod declares an
fsGroup, rejects a caCertSecretName that names the TLS Secret itself,
and waits for the TLS material before it writes the ConfigMaps and the
workloads, so a first TLS install starts its pods once.

tls.enabled is fixed at creation. Redis replication and Sentinel cannot
straddle plaintext and TLS: a replica restarted onto the other protocol
cannot sync from the master, and Sentinels restarted onto it fail over
to such a replica while the master still takes writes. The rule that
refuses the change is a CEL transition rule on the RedisFailover spec;
it lives in the CRD, so it ships here through patches/crd-tls.diff
together with the schema block it references. The cut carries source
only, and the Dockerfile strips the tarball's manifests/, charts/,
example/ and .github/ before applying the patches, so the image builds
from source alone and the two upstream tests that read the CRD YAML skip
instead of failing against v3.3.5's stale copy; the CRD half is
patches/crd-tls.diff.

cert-manager Certificates get no grant. The patched operator manages a
Certificate only under spec.tls.certManager, and packages/apps/redis
pins the other mode: it always supplies spec.tls.certificateSecret, so
that code path is unreachable as the platform ships it. A comment at
the rule site records the decision and names the verbs the rule would
need, so a re-vendor review reads the absence as deliberate.

The chart patches go through the mechanism the packages that customise a
vendored chart already use — reloader, grafana-operator, ingress-nginx,
seaweedfs and kamaji all re-apply a patches/*.diff at the end of update:
— because `rm -rf charts` discards a hand-edit and the next `make update`
would take TLS back out without failing.

tls.diff also carries one change that is not TLS and takes effect with
TLS off: MakeSlaveOfWithPort connects to the replica on its own port
rather than on the master's. It changes a dial, not a pod template, so
nothing rolls when this image ships; the probes dial localhost only
under TLS, and the plaintext pod templates stay byte-identical to
v3.3.5. The patch header names it, so a reader of patches/ does not
take the file for TLS-only.

The grant is narrower than the ported operator's own chart asks for:
that one also grants patch on Secrets, and create, delete, get, list,
patch, update and watch on Certificates. None of that is reachable from
the source as shipped here.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
…alue

The no / optional / yes constraint on tls.authClients lives in
values.schema.json, and nothing the chart renders shows whether it is
enforced: a bogus value either fails validation up front or reaches the
template as a string. Assert the failure, so the enum cannot be dropped
from the schema without a red test.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The operator side of the carried patch computes the leaf's SAN list in
redisCertificateSANs. The comment in certmanager.yaml still named it
redisCertificateDNSNames, a name the patched source does not contain.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
Everything the TLS unit tests can only render now runs on a live
cluster. A second fixture with tls.enabled asserts that cert-manager
issues the chain, that the patched operator brings two Redis replicas
and the Sentinel Deployment to Ready through its --tls probes, that the
operator publishes its CA-only Secret under the widened ClusterRole, and
that the CA-extraction controller hands the tenant exactly ca.crt
through core.cozystack.io/tenantsecrets while the key-bearing Secrets
and the operator's copy stay off that path. The last check completes a
handshake through the rfr- service name with the same CA the tenant
receives.

A third case applies tls.enabled to a running plaintext release and
asserts the refusal end to end: the HelmRelease upgrade fails with the
schema's own message, the StatefulSet keeps its revision and both
replicas Ready, plaintext still answers and no TLS listener appears.

The TLS case also asserts the topology the operator built rather than
stopping at readyReplicas: exactly one master, a replica whose link is
up, and sentinels that moved off the bootstrap address. Ready alone
witnesses the bootstrap, since a pod still replicating from the initial
127.0.0.1 never gets there, but the readiness script exits 0 for
anything reporting role:master, so it stops witnessing once the
bootstrap is over.

Both PING calls note where the pinger credentials come from: the ACL user
the operator writes into every redis.conf it generates, and the one its
own liveness probe uses. Nothing in this tree defines it, so a reader
grepping for it here finds nothing and reads the calls as broken.

Every redis-cli call runs under the container's own timeout, and both
script operations carry an explicit one. The inner cap is the part
that matters. The flip case dials TLS at a plaintext listener that
will never send a ServerHello, and redis-cli has no handshake
deadline of its own. Chainsaw does kill the shell when its exec
budget runs out, but it captures the script's stdout and stderr
through pipes, and Go's exec then waits for every holder of the write
end to close it; the orphaned kubectl is one, so the suite would go
silent until the job ceiling instead of failing. Expiry reads as the
refusal for the negative check and as a named failure for the PINGs.
Both engine images provide /usr/bin/timeout: busybox on the sentinel,
coreutils on the data pods.

The negative half of the tenant-secret pair requires the error to say
NotFound. A 403, a timeout or an unserved CRD would otherwise be read
as proof that the Secret is not served, which is the direction that
fails silently.

The dashboard Role is asserted on the deployed object, not only in the
chart unit tests: it must reach the auth Secret and nothing else, which
is the name-granting half of the pair the label-driven projection
completes. Every kubectl in both scripts runs under a wall-clock bound,
because the per-command cap inside a container cannot fire when the exec
stream itself never establishes, and a kubectl left running holds the
script's stdout open. The plaintext StatefulSet in the flip case waits
5m rather than 2m: it measured 98s of that budget on a real run, being
the third failover the group brings up in one namespace.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>

@IvanHunters IvanHunters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict

NOT LGTM

Round two, against 25a06181 (the branch was rebased since my last round, so the commits are rewritten; I compared trees, not shas). Five of my eight findings are closed and I checked each closure by reverting it and watching a suite go red rather than by reading the change. Three are still open, all of them at two rounds now. One new defect turned up, and it is the same class as the one that closed.

I was wrong about one thing last round and want to say so plainly: my suggested redis-cli -t 5 would not have bounded the stall. -t caps the TCP connect, which succeeds against a plaintext port, so the wait is the TLS handshake after it. The inner timeout 15 in the fix is the right instrument.

Closed since my last round

  • The suite stall. hack/e2e-chainsaw/redis/chainsaw-test.yaml:551 now carries timeout: 2m on the op, every redis-cli runs under timeout 15 and every kubectl under timeout 30. /usr/bin/timeout is present in both engine images, which I checked by running them rather than trusting the comment that says so. CI corroborates: on the run at d5efbd228 the flip step took 15.6s (CMD RUN 09:23:32.455, SCRIPT DONE 09:23:48.058) against 2h58m56s of silence at the same step before, and all three redis cases completed in 3m50s with the suite carrying on into qdrant and postgres.
  • tls.authClients no longer changes nothing. The directive is folded length-prefixed into tlsSecretContentHash, and checker.go deletes every pod whose revision hash differs, so the roller delivers it. The reverse direction is fixed at the right layer too: check.go:44 now presents the operator's client certificate whenever the Secret carries a pair, independent of the spec, so the flip back cannot lock the operator out of its own instance. Replacing the hash lines with _ = tlsAuthClients(rf) turns TestTLSSecretHashTracksAuthClients red on all three subtests; restoring the authClients == yes gate turns TestTLSConfigForPresentsClientCertificate red on three more.
  • The Secret-name collision, and by a better construction than I proposed. The separator is now a dot, so the suffix set {.ca-tls, .tls, .ca-cert, .tenant-ca} is free of mutual suffixes and R1 + ".tls" == R2 + ".ca-tls" cannot hold. That does not depend on app-name validation, and it holds even for a release name containing a dot. Rendering the adversarial pair confirms it: redis-foo-ca-tls is no longer any release's Secret name. Reverting the separator in certmanager.yaml alone fails 3 of the 48 unittests.
  • The tenantsecret negative assertion. The loop now requires NotFound in the message. Driven against five stubbed outcomes it separates them correctly, and reverting the guard reproduces the old fail-open exactly: forbidden, unserved and timeout all go back to exit 0.
  • The authClients description. The constraint reached values.schema.json, the README table and the Go type with identical text, and the README goes further than I asked by stating the CA key is deliberately out of reach. I verified the underlying claim instead of accepting it: tenant ClusterRoles grant core.cozystack.io/tenantsecrets and nothing on cert-manager.io or raw secrets, so neither the CA Secret nor the in-namespace Issuer is reachable.

Still open from my earlier rounds

[MAJOR] hack/e2e-chainsaw/redis/chainsaw-test.yaml:86, still nothing exercises an existing cluster (round 2)

All four Chainsaw cases still create their instances from scratch under the new operator and the new CRD:

$ grep -rn '^  name: ' hack/e2e-chainsaw/{redis,valkey}/chainsaw-test.yaml
redis/chainsaw-test.yaml:5:    name: redis
redis/chainsaw-test.yaml:86:   name: redis-tls
redis/chainsaw-test.yaml:463:  name: redis-tls-flip-rejected
valkey/chainsaw-test.yaml:5:   name: valkey
$ grep -rn 'file: redis\|file: valkey' hack/e2e-chainsaw/{redis,valkey}/chainsaw-test.yaml
redis:20 redis.yaml   redis:115 redis-tls.yaml   redis:486 redis-flip.yaml
redis:522 redis-flip-tls.yaml   valkey:20 valkey.yaml

Every fixture is a fresh apps.cozystack.io/v1alpha1 object. redis-tls-flip-rejected does edit a running plaintext instance, but it created that instance itself under this branch, so the transition an existing customer actually takes is still untested.

Two things did materially de-risk it, and both arrived as hermetic pins rather than as arguments. The spec-level CEL rule was exercised against the real apiextensions validator with the shipped cozystack CRD staged into the fixture paths, and an object with no tls block stays updatable because the rule evaluates false == false on both sides. And the rebuilt binary renders a byte-identical plaintext StatefulSet, Sentinel Deployment and both ConfigMaps compared against a pristine v3.3.5 build, for Engine: Valkey as well; TestPlaintextProbesKeepUpstreamCommands guards that and is not inert, since making redisCLITLSFlags return --tls for a plaintext failover turns all four of its subtests red.

What is still unpinned is what needs an apiserver: SSA field ownership on the already-running rfr-* StatefulSet and rfs-* Deployment, and the CreateReplace of the CRD against stored objects. A Chainsaw case that pins the operator image to the pre-patch digest, creates the instance, then swaps to the rebuilt digest would close this permanently.

[MINOR] packages/system/redis-operator/values.yaml:4, the committed digest is still the pre-patch build (round 2)

The file is untouched by this PR, and the digest it pins was stamped by abb38ab72, whose tree carries only labels.diff under images/redis-operator/patches/. So the committed tree renders spec.tls against a binary built from a patch set that does not contain tls.diff.

The signal did improve, and it came from main through the rebase rather than from this branch: redis-operator is now in the root Makefile build: list and hack/build-matrix.sh emits it, so PR CI builds the package, make image stamps the fresh digest, and the per-package job uploads it for reassembly. docs/agents/image-refs.md is edited here to drop redis-operator from the live-violations list, consistent with that. What stays reachable is any path that builds the platform bundle without rebuilding this package first: make -C packages/core/platform image on its own, or the dev-loop package-deploy flow.

The blast radius grew since my last round, through a consumer this PR did not add. Valkey landed on main and creates RedisFailover on the same operator, CRD and widened ClusterRole. The pinned digest therefore also lacks valkey-cli-auth.diff, which switches valkeyEngine.CLIAuthEnvName() from VALKEYCLI_AUTH to REDISCLI_AUTH, and that value is exported by the generated shutdown.sh and ready.sh, so it drives the readiness probe. Checked against the pinned engine images rather than reasoned about:

$ docker run --rm --entrypoint sh valkey/valkey:8.1.8 -c '...'
-- REDISCLI_AUTH:   PONG
-- VALKEYCLI_AUTH:  NOAUTH Authentication required.
$ docker run --rm --entrypoint sh valkey/valkey:7.2.13 -c '...'
v7 REDISCLI_AUTH:   PONG
v7 VALKEYCLI_AUTH:  NOAUTH Authentication required.

On the committed digest with the default authEnabled: true, a Valkey readiness script gets NOAUTH and the pods never go Ready. Through that path the stale pin fails loudly instead of silently, and the Valkey Chainsaw case asserting readyReplicas: 2 would catch it, but only because CI rebuilds the image. Pinning the rebuilt digest here removes the whole question.

[MINOR] packages/system/redis-operator/patches/rbac-tls.diff:44, still no ownership check on the Secret write (round 2)

The only change since last round is a no-op skip when the stored content already matches. CreateOrUpdateSecret still issues a full Update on whatever Secret occupies the name, and there is no ownerReference, label or adoption check anywhere on the path. The validation is still only the inequality with the TLS Secret's own name, at api/redisfailover/v1/validate.go:134.

One correction to my own description: it is slightly worse than replacing Data. generateRedisCACertSecret sets OwnerReferences, and the write is a full Update, so overwriting a foreign Secret also reparents it to the RedisFailover and it gets garbage-collected when that object is deleted.

Still not reachable from the tenant surface, for the reasons I gave last round, and still the right destination is upstream. I am leaving it open rather than closing it because nothing in this tree changed about it.

New in this round

[MAJOR] hack/e2e-chainsaw/redis/chainsaw-test.yaml:578, the negative dial probes are fail-open, the same defect that just closed one file over

if k -n "$NAMESPACE" exec rfr-redis-flip-0 -c redis -- \
  timeout 15 redis-cli --tls --insecure -h "rfr-redis-flip.$NAMESPACE.svc" -p 6379 \
  --user pinger --pass pingpass --no-auth-warning ping 2>/dev/null | grep -qx PONG; then
  echo "a TLS listener appeared on an instance whose flip was rejected" >&2
  exit 1
fi

Stderr is discarded and the verdict is "did the word PONG appear". A missing pod, a 403 on exec, a wrong container name, or the outer timeout 30 firing all produce no PONG, so the if is false and the assertion reports success. It can only fail when a TLS listener genuinely answers; it cannot tell "there is no listener" from "I could not ask". set -eu does not help, because the if consumes the pipeline's status.

Driven against stubbed outcomes, four distinct failures to ask all read as the refusal being asserted:

refused    exit=0   read as: no TLS listener
notimeout  exit=0   read as: no TLS listener
nopod      exit=0   read as: no TLS listener
rbac       exit=0   read as: no TLS listener
tlsup      exit=1   a TLS listener appeared on an instance whose flip was rejected

This is the security-relevant half of the flip case: it is the assertion that a rejected flip really did leave TLS off. The notimeout row is a mode the timeout fix introduced by putting a binary in the exec path; it is not live today because both images carry it, but nopod and rbac are live now.

The shape you already use twenty lines above is the fix, and you apply it correctly to the positive dial in the same script:

pong=$(k ... timeout 15 redis-cli --tls --insecure ... ping 2>&1) \
  || pong='(could not ask: exec failed or hit the cap)'
case "$pong" in
  *PONG*) echo "a TLS listener appeared on an instance whose flip was rejected" >&2; exit 1 ;;
  *"Connection reset"*|*"connection refused"*|*"I/O error"*|*"no reply"*) : ;;
  *) echo "cannot determine whether a TLS listener is up: $pong" >&2; exit 1 ;;
esac

hack/e2e-chainsaw/redis/chainsaw-test.yaml:399 has the identical shape for the mirror assertion, that the plaintext listener does not answer once TLS is on, and needs the same treatment.

Caveats

  • The E2E job on this exact head has not finished: E2E (in-tree) in run 33645509783 is queued. My timing evidence for the closed stall comes from run 33608673237 on d5efbd228, a superseded rewrite of the same logical tip whose chainsaw file I compared against this head. A green run here would be the direct proof.
  • The two .diff-anchored items and the values.yaml one are not inline comments: those files are either untouched by this PR or came from main with the rebase, so GitHub has no changed line to hang them on.
  • Valkey is not this PR's work and I am not asking for anything about it here beyond the digest pin. For the record and for whoever owns that package: an externally exposed Valkey is plaintext-only with no TLS option in its schema, which is an asymmetry with Redis, and there is no check-valkey-rd-secrets.bats counterpart to the redis Secret-surface guard.

Checked and correct

  • The vendored chart is not drifted from its patches, which is the failure mode where make update silently reverts a hand-edit. Pulling upstream 3.3.5 fresh and applying crd-tls.diff and rbac-tls.diff reproduces the committed charts/ tree with no differences.
  • The carried patch builds and passes its own suite: upstream v3.3.5 tarball with manifests charts example .github removed exactly as the Dockerfile does, git apply --check clean for all three diffs, go build ./... exit 0, go test ./... all ok.
  • TestGeneratedCRDRejectsTurningTLSOnOrOff skips in the image build because the Dockerfile deletes manifests/; staging the shipped cozystack CRD into the fixture paths runs it for real, and it passes including the accepted-transition cases.

# Expiry of that cap IS the refusal being asserted: a plaintext
# listener never sends a ServerHello, so redis-cli waits it out and
# prints no PONG for grep to find.
if k -n "$NAMESPACE" exec rfr-redis-flip-0 -c redis -- \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] the negative dial probe is fail-open, the same defect that just closed one file over

Stderr is discarded and the verdict is "did the word PONG appear". A missing pod, a 403 on exec, a wrong container name, or the outer timeout 30 firing all produce no PONG, so the if is false and the assertion reports success. It can only fail when a TLS listener genuinely answers; it cannot tell "there is no listener" from "I could not ask". set -eu does not help, because the if consumes the pipeline's status.

Driven against stubbed outcomes, four distinct failures to ask all read as the refusal being asserted:

refused    exit=0   read as: no TLS listener
notimeout  exit=0   read as: no TLS listener
nopod      exit=0   read as: no TLS listener
rbac       exit=0   read as: no TLS listener
tlsup      exit=1   a TLS listener appeared on an instance whose flip was rejected

This is the security-relevant half of the flip case: it is the assertion that a rejected flip really did leave TLS off. The notimeout row is a mode the timeout fix introduced by putting a binary in the exec path; it is not live today because both images carry it, but nopod and rbac are live now.

The shape you already use twenty lines above is the fix, and you apply it correctly to the positive dial in the same script:

pong=$(k ... timeout 15 redis-cli --tls --insecure ... ping 2>&1) \
  || pong='(could not ask: exec failed or hit the cap)'
case "$pong" in
  *PONG*) echo "a TLS listener appeared on an instance whose flip was rejected" >&2; exit 1 ;;
  *"Connection reset"*|*"connection refused"*|*"I/O error"*|*"no reply"*) : ;;
  *) echo "cannot determine whether a TLS listener is up: $pong" >&2; exit 1 ;;
esac

Line 399 has the identical shape for the mirror assertion, that the plaintext listener does not answer once TLS is on, and needs the same treatment.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both probes rewritten on 3cf9431 in the capture-and-classify form, with one change to your sketch: pong=$(...) || pong='(could not ask)' would overwrite the captured text on every non-zero exit, and redis-cli exits 1 on the real refusal too, so the refusal would have read as "could not ask" and failed. The exit code is kept in rc and the text stays in pong.

The accepted refusal is the one each listener actually gives, verified by running redis-cli in the shipped images rather than from the old comment. Here (flip case): a plaintext listener holds a ClientHello in silence, because processInlineBuffer scans for a newline with strchr and the handshake header carries a NUL before any newline, so redis-cli sits until the inner timeout 15 kills it, silent, exit 124. What the script accepts is kubectl's command terminated with exit code 124 trailer, which only appears when the exec reached the container and the inner cap fired; the outer cap kills kubectl without it. On line 399 (mirror) the TLS listener resets the plaintext client at AUTH and redis-cli prints I/O error, so that line is the refusal there, and an inner-cap expiry on that probe is a state the check did not read and fails.

Stubbed the k() wrapper for both: refusal signature exits 0; empty output with 124 (outer cap), pod not found, forbidden, missing container, missing timeout, unresolvable name, connection refused and an SSL negotiation failure all exit 1 with the dial's output in the message; PONG exits 1 with the security message. The old probes on the same stubs exit 0 on everything except PONG.

# `port 0` beside the TLS port, and every check above would stay
# green if it stopped doing so. The mirror of the flip case: here the
# plaintext dial is the one that must not answer.
if k -n "$NAMESPACE" exec rfr-redis-secure-0 -c redis -- \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] mirror of the fail-open probe: the plaintext-must-not-answer assertion cannot fail on an unaskable pod

Same shape as line 578 and the same consequence, for the mirror claim. 2>/dev/null | grep -qx PONG inside an if means a missing pod, a 403 on exec, or the outer timeout 30 firing all read as "the plaintext listener is gone", which is exactly the thing this step exists to prove.

This one carries a little more weight than its twin, because it is the only assertion that port 0 actually took effect. The comment above it says as much: every other check in the case would stay green if the operator stopped writing port 0. So if this probe cannot distinguish "no plaintext listener" from "could not reach the pod", nothing in the suite witnesses that TLS replaced plaintext rather than joining it.

Fix is the capture-and-classify form from the finding on line 578.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same fix on 3cf9431; the accepted refusal here is the I/O error redis-cli prints when the TLS listener resets the plaintext client at AUTH. Detail in the thread on line 578.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: redis-tls

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] still open from my earlier round: nothing exercises an existing cluster (round 2)

All four Chainsaw cases still create their instances from scratch under the new operator and the new CRD:

$ grep -rn '^  name: ' hack/e2e-chainsaw/{redis,valkey}/chainsaw-test.yaml
redis/chainsaw-test.yaml:5:    name: redis
redis/chainsaw-test.yaml:86:   name: redis-tls
redis/chainsaw-test.yaml:463:  name: redis-tls-flip-rejected
valkey/chainsaw-test.yaml:5:   name: valkey
$ grep -rn 'file: redis\|file: valkey' hack/e2e-chainsaw/{redis,valkey}/chainsaw-test.yaml
redis:20 redis.yaml   redis:115 redis-tls.yaml   redis:486 redis-flip.yaml
redis:522 redis-flip-tls.yaml   valkey:20 valkey.yaml

Every fixture is a fresh apps.cozystack.io/v1alpha1 object. redis-tls-flip-rejected does edit a running plaintext instance, but it created that instance itself under this branch, so the transition an existing customer actually takes is still untested.

Two things did materially de-risk it since my last round, and both arrived as hermetic pins rather than as arguments. The spec-level CEL rule was exercised against the real apiextensions validator with the shipped cozystack CRD staged into the fixture paths, and an object with no tls block stays updatable because the rule evaluates false == false on both sides. And the rebuilt binary renders a byte-identical plaintext StatefulSet, Sentinel Deployment and both ConfigMaps compared against a pristine v3.3.5 build, for Engine: Valkey as well; TestPlaintextProbesKeepUpstreamCommands guards that and is not inert, since making redisCLITLSFlags return --tls for a plaintext failover turns all four of its subtests red.

What is still unpinned is what needs an apiserver: SSA field ownership on the already-running rfr-* StatefulSet and rfs-* Deployment, and the CreateReplace of the CRD against stored objects. A Chainsaw case that pins the operator image to the pre-patch digest, creates the instance, then swaps to the rebuilt digest would close this permanently.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replayed on a live cluster today, against 25a061818. Starting point: a plaintext Redis with two replicas created under the pre-patch operator (v3.3.5, digest 551f3934), running for seven hours. The upgrade path mirrored what helm-controller does for this package: the CRD replaced first (upgradeCRDs: CreateReplace is set on the PackageSource), then the operator chart upgraded to this branch with the rebuilt image.

Results:

  • rfr-redis-upgrade-base: currentRevision == updateRevision (6d57f8f7f9) before and after, 2/2 Ready, zero restarts, pod start times unchanged. The StatefulSet and Sentinel Deployment specs hash identical before and after, and managedFields shows no write from the new operator at all: its last Update on both objects is still the creation.
  • The stored RedisFailover survived the CRD replace with the same resourceVersion, took a metadata edit, and a server dry-run customConfig append went through (generation 2).
  • The replaced schema accepts spec.tls on a new object (server dry-run create), refuses the flip on the live one with the CEL message, and prunes an unknown tls.bogus key.
  • Plaintext listener: PING returns PONG, a SET/GET round trip works, the role is still master.

On the Chainsaw case that pins the pre-patch digest and swaps: I agree it is the permanent form of this, and I would rather land it as its own PR. It needs the old digest reachable from the sandbox registry and an ordered CRD swap in the harness, which is e2e plumbing rather than this change. If you want it in here, say so.

- - "get"
+ - create
+ - get
+ - update

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] still open from my earlier round: no ownership check on the Secret write (round 2)

The only change since last round is a no-op skip when the stored content already matches. CreateOrUpdateSecret still issues a full Update on whatever Secret occupies the name, and there is no ownerReference, label or adoption check anywhere on the path. The validation is still only the inequality with the TLS Secret's own name, at api/redisfailover/v1/validate.go:134.

One correction to my own description from last round: it is slightly worse than replacing Data. generateRedisCACertSecret sets OwnerReferences, and the write is a full Update, so overwriting a foreign Secret also reparents it to the RedisFailover and it gets garbage-collected when that object is deleted.

Still not reachable from the tenant surface, for the reasons I gave last round, and still the right destination is upstream. I am leaving it open rather than closing it because nothing in this tree changed about it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reparenting point is taken: a full Update carrying OwnerReferences adopts a foreign Secret, not just its data, so the upstream guard has to refuse the write when the stored object is not owned by the failover. It is on the freshworks-oss/redis-operator#85 follow-up list next to the CA-publish opt-out; nothing changes here by design, and this thread stays open until that lands.

@lexfrei

Copy link
Copy Markdown
Contributor

On the pinned digest in values.yaml: the first-party digests in that file are stamped only by the release-prep commit, PRs do not touch them, and PR CI rebuilds the package and overlays the fresh digest before the sandbox install. That is why the redis and valkey Chainsaw cases run against the patched binary. The stale pin ships as-is until the next release prep restamps it, the same rule holds for every first-party image in the tree, and the Valkey readiness consequence you traced is the kind of gap that restamp closes. I will not repin it in this PR. What stays reachable, as you say, is a bundle built without make image for this package first. That path is the dev loop, not a release.

The two negative dials in the chainsaw suite, the plaintext dial at a
TLS instance and the TLS dial at the instance whose flip was rejected,
decided by whether PONG appeared on stdout with stderr discarded. A
missing pod, a 403 on exec, a wrong container name or the outer cap
firing print no PONG either, so each of them passed as the refusal
being asserted, and the flip case is the check that a rejected flip
left TLS off.

Keep what the dial printed and pass only on the refusal each listener
actually gives, verified in the shipped images: a TLS listener drops a
plaintext client at AUTH and redis-cli reports it as an I/O error; a
plaintext listener holds a ClientHello in silence, because the inline
parser's newline scan stops at a NUL in the handshake header, so the
inner cap expires and kubectl's exit-code trailer carries the 124.
That trailer is what separates the cap firing inside the pod from the
wrapper's cap firing outside it. A listener that answers PONG fails
the check, and anything else fails as a state the check did not read,
with the dial's output in the message.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
@lexfrei Aleksei Sviridkin (lexfrei) removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued /hold label Sep 2, 2026

@IvanHunters IvanHunters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict

LGTM with non-blocking notes

Round three, against 3cf9431f. The delta since my last round is one file, hack/e2e-chainsaw/redis/chainsaw-test.yaml, +54/-13, and it is the fix to both findings I raised. I checked the two rewrites by driving them rather than by reading them, and the E2E run on this head settles the one I raised two rounds ago.

Three items stay open and none of them blocks: one is a coverage gap whose replay has been run and reported, one is a packaging inconsistency whose published artifact is already correct, and one is an upstream hardening item we agree on. They are below with what would retire each.

One correction of mine to record: you were right to change the sketch I posted. pong=$(...) || pong='(could not ask)' overwrites the captured text on every non-zero exit, and redis-cli exits 1 on the genuine refusal too, so the refusal would have been read as "could not ask" and the case would have failed on the very state it asserts. Keeping the text in pong and the status in rc is correct.

Closed since my last round

Both negative dial probes. The fail-open is gone, and I checked it by extracting each block and driving it against stubbed kubectl outcomes rather than by reading the diff:

                    flip probe                 plaintext probe
PONG                fails (listener found)     fails (listener answers)
own refusal         accepted                   accepted
other's refusal     rejected                   rejected
missing pod         rejected: cannot determine rejected: cannot determine
403 on exec         rejected: cannot determine rejected: cannot determine
outer k() cap       rejected: cannot determine rejected: cannot determine
wrong container     rejected: cannot determine rejected: cannot determine
connection refused  rejected: cannot determine rejected: cannot determine

Every "I could not ask" state now fails closed, each probe accepts only the refusal its own listener produces, and each rejects the other's. That last column is what makes this a real classification rather than a broader pattern match.

The suite stall. Settled by CI on this head rather than by argument. From the E2E job:

--- PASS: chainsaw/redis-tls-flip-rejected (75.17s)
--- PASS: chainsaw/redis-tls (60.60s)
--- PASS: chainsaw/redis (56.39s)
--- PASS: chainsaw/valkey (63.89s)

Against 2h58m56s of silence at that step and fourteen suites that never started, two rounds ago. The TLS fixture this PR's description walks through has now actually executed.

Still open, none blocking

hack/e2e-chainsaw/redis/chainsaw-test.yaml:86, no suite starts from an existing cluster

All three redis cases still create their instances from scratch under the new operator and the new CRD, so the transition an existing customer takes is not encoded anywhere that runs on every push.

I am not blocking on it, and I want to be exact about why, because the evidence is not mine. In my last round I wrote that a clean replay or an in-suite image swap would move this to a non-blocking note. You ran the replay and reported it clean with specifics: a plaintext instance on the pre-patch operator, CRD replaced first, currentRevision == updateRevision unchanged, 2/2 Ready, zero restarts, same resourceVersion, managedFields showing no writes. A hermetic review cannot reach a cluster, so that is your measurement and not my verification, and I am taking it at the value I said I would. What I did verify myself, in an earlier round and again here, is the part that does not need a cluster: the plaintext render is byte-identical across the change, the patched operator emits identical plaintext StatefulSet, Sentinel Deployment and ConfigMaps against a pristine v3.3.5 build, and the transition rule evaluates false == false on a stored object with no tls block, so it cannot fire on an upgrade that changes nothing.

What retires it: the Chainsaw case you propose as a separate PR, pinning the operator image to the pre-patch digest, creating the instance, then swapping to the rebuilt digest. Until that lands, this path is covered by a measurement nobody can re-run from the repository.

packages/system/redis-operator/patches/rbac-tls.diff:44, no ownership check on the Secret write

Unchanged this round, and by design. CreateOrUpdateSecret still issues a full Update on whatever Secret occupies the name, with no ownerReference, label or adoption check on the path, and the only validation remains the inequality with the TLS Secret's own name. Agreed that the guard belongs upstream in freshworks-oss/redis-operator#85 alongside the CA-publish opt-out, and agreed on the reparenting point: the write carries OwnerReferences, so overwriting a foreign Secret adopts it and it is garbage-collected with the failover. Not reachable from the tenant surface as the platform ships. Leaving the thread open until the upstream change lands is the right shape.

packages/system/redis-operator/values.yaml:4, the committed digest is still the pre-patch build

Third round, still untouched, still v3.3.5@sha256:551f3934..., a digest stamped by a commit whose tree carries only labels.diff. The published artifacts are fine: the package is in the root build: list, make image stamps the fresh digest, and the release and main lanes run it. What stays reachable is a tree read without a build, which is make -C packages/core/platform image on its own or the dev-loop deploy: there the chart admits and renders spec.tls against a binary that ignores it and serves plaintext, with no signal anywhere. Through Valkey the same pin also predates valkey-cli-auth.diff, so a Valkey instance built from the committed tree gets NOAUTH on its readiness script. I accept that a hand-pinned digest goes stale the moment CI stamps a new one, so I am not asking for it again; recording it because it has now survived three rounds and the next reader of this file will hit the same question.

Caveats

  • The E2E Tests check on this head is green but reads in-tree e2e soft: node-join deadline missed, suite failed, merge not blocked. The suite did fail. The failures are kubernetes-latest and kubernetes-previous, both on the node-join deadline, and the job log says so explicitly, so nothing here is redis-adjacent. Worth knowing that a green tick on this lane is not the same as a green suite.
  • The refusal strings the two probes now key on, I/O error for a TLS listener dropping a plaintext client and kubectl's command terminated with exit code 124 trailer for a plaintext listener holding a ClientHello, are the load-bearing constants of both assertions. I confirmed the classification logic behaves correctly given those strings; whether the shipped redis-cli and kubectl emit exactly them is what the passing E2E run on this head demonstrates, not something I checked independently. If either string ever drifts, both probes fail closed and say "cannot determine", which is the right direction to fail in.
  • Not re-reviewed this round: everything outside the one changed file. The carried operator patch, the chart, the CRD and the RBAC diff were reviewed in rounds one and two and are byte-identical to what I looked at then.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: redis-tls

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] still open, not blocking: no suite starts from an existing cluster (round 3)

All three redis cases still create their instances from scratch under the new operator and the new CRD, so the transition an existing customer takes is not encoded anywhere that runs on every push.

Not blocking, and the evidence for that is yours rather than mine. Last round I wrote that a clean replay or an in-suite image swap would move this to a non-blocking note; you ran the replay and reported it clean with specifics (pre-patch operator, CRD replaced first, currentRevision == updateRevision unchanged, 2/2 Ready, zero restarts, same resourceVersion, managedFields showing no writes). A hermetic review cannot reach a cluster, so I am taking that at the value I said I would rather than re-deriving it.

What I did verify without a cluster, in an earlier round and again here: the plaintext render is byte-identical across the change, the patched operator emits identical plaintext StatefulSet, Sentinel Deployment and ConfigMaps against a pristine v3.3.5 build, and the transition rule evaluates false == false on a stored object with no tls block.

What retires it: the Chainsaw case you propose as a separate PR, pinning the operator image to the pre-patch digest, creating the instance, then swapping to the rebuilt digest. Until that lands, this path rests on a measurement nobody can re-run from the repository.

- - "get"
+ - create
+ - get
+ - update

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] still open, not blocking: no ownership check on the Secret write (round 3)

Unchanged this round, and by design. CreateOrUpdateSecret still issues a full Update on whatever Secret occupies the name, with no ownerReference, label or adoption check on the path, and the only validation remains the inequality with the TLS Secret's own name.

Agreed the guard belongs upstream in freshworks-oss/redis-operator#85 alongside the CA-publish opt-out, and agreed on the reparenting point: the write carries OwnerReferences, so overwriting a foreign Secret adopts it and it is garbage-collected with the failover. Not reachable from the tenant surface as the platform ships.

Leaving the thread open until the upstream change lands is the right shape; nothing is being asked of this PR.

@kvaps Andrei Kvapil (kvaps) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. A 6500-line patch against a vendored operator is the shape I open most suspiciously, so here is what changed my mind.

It is a carry, not a fork. The patch header pins the upstream commit it was cut from (freshworks-oss/redis-operator#85, 2ad2ed8d1) and states its own retirement condition — retire when a release carrying #85 lands. The chart patches re-apply at the end of update: the same way reloader, grafana-operator, ingress-nginx, seaweedfs and kamaji carry theirs, which matters because rm -rf charts would otherwise take TLS back out on the next make update and nothing would fail.

The non-TLS change is better than the header claims. The header names MakeSlaveOfWithPort as taking effect with TLS off and argues it is safe because it changes a dial rather than a pod template. Reading the hunk, it is a latent bug fix:

- Addr: net.JoinHostPort(ip, masterPort), // this is IP and Port for the RedisFailover redis
+ Addr: net.JoinHostPort(ip, localPort),

The old code dialled the replica at ip on the master's port, with a comment that read as if that were intended. It is invisible while both run the same port, which is the same-cluster failover path, and it breaks external-master mode where masterPort belongs to a Redis outside the cluster. MakeSlaveOf now passes redisPort for both and preserves the old behaviour on that path. So this is neutral for every current deployment and a fix for one that was broken.

Opt-in is typed, not just documented. Enabled *bool with no default, and the field doc says outright that TLS is not inferred from external — which is the wrong inference someone would otherwise reach for, given both concern reachability. AuthClients carries its own consequence in the doc: the directive is read once at startup, so changing it restarts Redis and Sentinel.

The RBAC delta is the part I would hold up as an example. Create, Get and Update on Secrets, which the header states are every Secret call in the patched source; no list and no watch because the Secret is addressed by name; nothing patches or deletes. Certificates are deliberately not granted, and the rationale sits at the rule site rather than only in the commit, so a future re-vendor reads the absence as a decision instead of an omission. That is narrower than the ported operator's own chart asks for.

CI is green including E2E on 3cf9431f1.

The thing to keep an eye on is the retirement condition, since it depends on an upstream PR merging and releasing — worth tracking somewhere that outlives this PR, so the carry does not quietly become permanent.

@lexfrei
Aleksei Sviridkin (lexfrei) merged commit 3a369fe into main Sep 4, 2026
49 of 51 checks passed
@lexfrei
Aleksei Sviridkin (lexfrei) deleted the feat/tls-redis branch September 4, 2026 15:32
Andrei Kvapil (kvaps) pushed a commit that referenced this pull request Sep 7, 2026
…ss fork (#3406)

## What this PR does

`spotahome/redis-operator` was archived on 2026-06-11 (read-only; last
release v1.2.4, 2022). This migrates the operator to the
actively-maintained **freshworks-oss/redis-operator** fork (v3.3.5,
Apache-2.0).

### Why freshworks-oss

It keeps the same `RedisFailover` CRD and API group
(`databases.spotahome.com`) — so existing managed Redis is unaffected —
and adds a native `spec.engine: Redis|Valkey` switch, letting managed
Valkey drop its Redis-compatibility shim image and run the official
`valkey/valkey` image directly.

### Changes

- Re-vendor the chart from
`oci://ghcr.io/freshworks-oss/charts/redis-operator` (v3.3.5), which
carries the `engine`-aware CRD.
- Build the operator image from freshworks-oss v3.3.5 source.
- Re-target the label/annotation-preservation patch onto the freshworks
Go module (`github.com/freshworks/redis-operator`).

### Verification

- freshworks-oss source + our `labels.diff` patch compiles natively with
Go 1.26.3 (`service` and `cmd` packages).
- `git apply --check` of the re-targeted patch against v3.3.5 succeeds.
- The operator chart renders with the cozystack-registry image override.
- The image is built (freshworks-oss v3.3.5 source + our patch,
multi-arch amd64/arm64), pushed to
`ghcr.io/cozystack/cozystack/redis-operator`, and pinned by digest in
`values.yaml`.
- `hack/promote-retag_test.bats` and `hack/nightly-mirror_test.bats`
pass with the pinned digest.

### Note for maintainers

`packages/system/redis-operator` is not part of the automated `make
build`; its image is built out-of-band and the digest is pinned in
`values.yaml`. That image is already built and pinned in this PR; re-run
`make -C packages/system/redis-operator image` only if you bump the
operator version.

### Follow-up

The cozystack/redis-operator fork carries cert-manager TLS work
(#2729) that freshworks-oss does not have yet; the
plan is to upstream it to freshworks-oss so it lands in the shared
maintained fork rather than a cozystack-only one.

### Release note

```release-note
feat(redis-operator): migrate from the archived spotahome/redis-operator to the actively-maintained freshworks-oss/redis-operator fork (same RedisFailover CRD, adds native spec.engine: Redis|Valkey)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for injecting additional environment variables into the
Redis Operator (including a default operator group id).
  * Added default configurable labels for the Redis Operator.
* **Bug Fixes**
* Preserves and merges existing Service labels and annotations during
Redis Operator updates.
* **Chores**
* Updated the Redis Operator and Helm chart to the Freshworks-maintained
3.3.5 release, aligned chart/image sources, and simplified Helm chart
retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/database Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse) kind/feature Categorizes issue or PR as related to a new feature size/XXL This PR changes 1000+ lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants