Skip to content

fix(cozystack-basics): validate legacy Ingress hostnames against the tenant apex - #3200

Merged
Aleksei Sviridkin (lexfrei) merged 1 commit into
mainfrom
fix/ingress-hostname-policy
Jul 4, 2026
Merged

Aleksei Sviridkin (lexfrei) merged 1 commit into
mainfrom
fix/ingress-hostname-policy

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Jul 4, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

The Gateway (cozystack-gateway-hostname-policy) and Route (cozystack-route-hostname-policy) admission policies constrain tenant hostnames to the tenant apex on the Gateway API path. Gateway API is opt-in and off by default, so in the default configuration tenant applications publish through a legacy networking.k8s.io/Ingress on the shared ingress-nginx, which had no equivalent policy.

This adds cozystack-ingress-hostname-policy, a fail-closed ValidatingAdmissionPolicy + binding for Ingress in tenant-* namespaces — the legacy-Ingress counterpart of the existing Gateway/Route hostname policies, and a new Layer 8 in the gateway security model. A hostname on spec.rules[].host or spec.tls[].hosts[] is allowed when it is within the namespace's own apex (namespace.cozystack.io/host), or when it is a concrete (non-wildcard) name entirely outside the platform root apex (_cluster.root-host). The second case keeps the kubernetes app's Proxied exposeMethod working, since it routes a user-supplied external domain to a nested cluster. A hostname under the platform root apex but outside the namespace's own apex, a wildcard taken through the outside-root path (which could otherwise match the platform apex itself), a rule with no host (absent or empty — an unbounded catch-all), and spec.defaultBackend are rejected.

The policy is fail-closed: a tenant-* namespace missing its own apex label is denied, and the policy renders only when _cluster.root-host is set, so it is never shipped in a state that would admit every hostname. cozy-* namespaces are cluster-admin-managed and not gated. This is defense-in-depth in the same family as the Gateway/Route policies — by design tenants hold only read access to Ingress.

The change ships helm-unittest coverage (render, plus the root-host, empty-host, and wildcard guards) and behavioral admission tests against the live apiserver in hack/e2e-apps/gateway.bats (in-apex allow, external-domain allow, under-root-outside-apex deny, wildcard-matching-apex deny, hostless deny, empty-string-host deny, defaultBackend deny), and documents the new layer in packages/extra/gateway/README.md.

Screenshots

N/A — no UI change.

Release note

fix(cozystack-basics): constrain legacy Ingress hostnames in tenant namespaces to the tenant apex

Summary by CodeRabbit

  • New Features
    • Added fail-closed defense-in-depth Layer 8 (cozystack-ingress-hostname-policy) to enforce allowed legacy Ingress rule and TLS hostnames for tenant namespaces, while blocking unsafe catch-all configurations (including missing/empty hosts and spec.defaultBackend) when the platform root host is configured.
  • Tests
    • Added end-to-end and render/deny validation for permitted vs denied hostname patterns, wildcard behavior, required namespace host labeling, and correct deny/fail-closed rendering.
  • Documentation
    • Updated the gateway security model to document Layer 8 behavior.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request strengthens the security posture of the Cozystack platform by implementing a new ValidatingAdmissionPolicy for legacy Kubernetes Ingress resources. By enforcing hostname constraints similar to those already present for the Gateway API, it prevents tenant namespaces from claiming hostnames outside their designated apex, while still allowing legitimate external domain routing. This change ensures a consistent security boundary across both legacy and modern ingress paths.

Highlights

  • New Ingress Admission Policy: Introduced cozystack-ingress-hostname-policy, a fail-closed ValidatingAdmissionPolicy that constrains legacy Ingress hostnames in tenant namespaces to the tenant's own apex or valid external domains.
  • Defense-in-Depth: Added Layer 8 to the gateway security model, ensuring that even when the opt-in Gateway API is disabled, legacy Ingress resources are properly validated against hostname spoofing.
  • Testing and Coverage: Implemented comprehensive helm-unittest coverage and live behavioral admission tests in hack/e2e-apps/gateway.bats to verify allow/deny scenarios for various Ingress configurations.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@github-actions github-actions Bot added area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review kind/bug Categorizes issue or PR as related to a bug labels Jul 4, 2026
@dosubot dosubot Bot added area/networking Issues or PRs related to networking (ingress, gateway, vpn, metallb, kube-ovn) kind/feature Categorizes issue or PR as related to a new feature labels Jul 4, 2026
@coderabbitai

coderabbitai Bot commented Jul 4, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds a legacy Ingress hostname admission policy, Helm render tests, e2e coverage, and gateway security-model documentation.

Changes

Ingress hostname policy enforcement

Layer / File(s) Summary
Policy template and CEL validation logic
packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml
Defines the ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding, gated on _cluster.root-host, with CEL checks for tenant namespaces, host/TLS host scope, wildcard handling, hostless/defaultBackend rejection, and fail-closed admission settings.
Helm unit tests for policy rendering and CEL behavior
packages/system/cozystack-basics/tests/ingress-hostname-policy_test.yaml
Verifies rendered document shape, namespace targeting, allowed and denied hostname branches, wildcard and catch-all rejection, fail-closed settings, missing-label behavior, and zero-document rendering when root-host is unset.
End-to-end bats tests for hostname enforcement
hack/e2e-apps/gateway.bats
Adds Ingress admission tests in tenant-test that confirm the policy and binding exist, then exercise denial and allowance cases for rule hosts, TLS hosts, wildcard hosts, hostless rules, and defaultBackend.
Security model documentation update
packages/extra/gateway/README.md
Adds a Layer 8 entry describing the legacy Ingress hostname policy and its fail-closed admission behavior.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • cozystack/cozystack#2470: Adds related hostname enforcement tests for a different ValidatingAdmissionPolicy in the gateway area.

Suggested labels: size/XXL, area/platform

Suggested reviewers: kvaps, lllamnyp, androndo, IvanHunters, sircthulhu, myasnikovdaniil

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding legacy Ingress hostname validation against the tenant apex.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ingress-hostname-policy

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the cozystack-ingress-hostname-policy ValidatingAdmissionPolicy (VAP) and its binding to enforce tenant-scoped hostname constraints on legacy Ingress resources, which serve as the default publishing path when the Gateway API is disabled. It includes comprehensive unit tests, E2E test cases, and updated documentation. The review feedback highlights a critical security vulnerability where the case-sensitive string comparisons in the CEL expression allow policy bypasses via mixed-case hostnames. To resolve this, the reviewer recommends using .lowerAscii() for case-insensitive validation, updating the unit and E2E tests to verify mixed-case handling, and making a minor wording improvement in the documentation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +53 to +56
{{- $rootHost := index .Values._cluster "root-host" }}
{{- if $rootHost }}
{{- $A := `namespaceObject.metadata.labels["namespace.cozystack.io/host"]` }}
{{- $celValidator := printf `(namespaceObject == null || !has(namespaceObject.metadata.labels) || !("namespace.cozystack.io/host" in namespaceObject.metadata.labels)) ? false : (!has(object.spec.defaultBackend) && (!has(object.spec.rules) || object.spec.rules.all(r, has(r.host) && r.host != "" && (r.host == %s || r.host.endsWith("." + %s) || (!r.host.startsWith("*.") && !(r.host == %q || r.host.endsWith(%q)))))) && (!has(object.spec.tls) || object.spec.tls.all(t, !has(t.hosts) || t.hosts.all(h, h != "" && (h == %s || h.endsWith("." + %s) || (!h.startsWith("*.") && !(h == %q || h.endsWith(%q))))))))` $A $A $rootHost (printf ".%s" $rootHost) $A $A $rootHost (printf ".%s" $rootHost) -}}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-high high

Case-Sensitivity Bypass Vulnerability

Hostnames in DNS and Ingress controllers (like ingress-nginx) are treated case-insensitively. However, the CEL expression uses case-sensitive string operators (==, endsWith, startsWith).

This allows a tenant to completely bypass the hostname policy and hijack domains under the platform root apex (such as the dashboard or other tenants' domains) simply by using uppercase or mixed-case letters in the host or tls.hosts fields of their Ingress resource (e.g., DASHBOARD.EXAMPLE.ORG instead of dashboard.example.org).

To prevent this, we should convert all hostnames to lowercase using .lowerAscii() before performing any comparisons, and also ensure the platform root host is lowercased.

{{- $rootHost := index .Values._cluster "root-host" | lower }}
{{- if $rootHost }}
{{- $A := "namespaceObject.metadata.labels[\"namespace.cozystack.io/host\"]" }}
{{- $celValidator := printf "(namespaceObject == null || !has(namespaceObject.metadata.labels) || !(\"namespace.cozystack.io/host\" in namespaceObject.metadata.labels)) ? false : (!has(object.spec.defaultBackend) && (!has(object.spec.rules) || object.spec.rules.all(r, has(r.host) && r.host != \"\" && (r.host.lowerAscii() == %s.lowerAscii() || r.host.lowerAscii().endsWith(\".\" + %s.lowerAscii()) || (!r.host.lowerAscii().startsWith(\"*.\") && !(r.host.lowerAscii() == %q || r.host.lowerAscii().endsWith(%q)))))) && (!has(object.spec.tls) || object.spec.tls.all(t, !has(t.hosts) || t.hosts.all(h, h != \"\" && (h.lowerAscii() == %s.lowerAscii() || h.lowerAscii().endsWith(\".\" + %s.lowerAscii()) || (!h.lowerAscii().startsWith(\"*.\") && !(h.lowerAscii() == %q || h.lowerAscii().endsWith(%q))))))))" $A $A $rootHost (printf ".%s" $rootHost) $A $A $rootHost (printf ".%s" $rootHost) -}}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this in this PR. A mixed-case hostname cannot bypass the policy: the apiserver validates Ingress host / tls.hosts as DNS-1123 (IsDNS1123Subdomain, uppercase rejected — kubernetes/kubernetes#71140), so this policy only ever sees already-lowercased hosts — the same reason the existing cozystack-gateway-hostname-policy and cozystack-route-hostname-policy use a case-sensitive comparison. The .lowerAscii() defense-in-depth hardening is tracked for all three policies together in #3205.

Comment on lines +96 to +104
asserts:
- documentIndex: 0
matchRegex:
path: spec.validations[0].expression
pattern: '!\(r\.host == "example\.org" \|\| r\.host\.endsWith\("\.example\.org"\)\)'
- documentIndex: 0
matchRegex:
path: spec.validations[0].expression
pattern: '!\(h == "example\.org" \|\| h\.endsWith\("\.example\.org"\)\)'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

Update the test assertions to match the new case-insensitive CEL expression using .lowerAscii().

    asserts:
      - documentIndex: 0
        matchRegex:
          path: spec.validations[0].expression
          pattern: '!\(r\.host\.lowerAscii\(\) == "example\.org" \|\| r\.host\.lowerAscii\(\)\.endsWith\("\.example\.org"\)\)'
      - documentIndex: 0
        matchRegex:
          path: spec.validations[0].expression
          pattern: '!\(h\.lowerAscii\(\) == "example\.org" \|\| h\.lowerAscii\(\)\.endsWith\("\.example\.org"\)\)'

Comment on lines +112 to +120
asserts:
- documentIndex: 0
matchRegex:
path: spec.validations[0].expression
pattern: '!r\.host\.startsWith\("\*\."\) && !\(r\.host == "example\.org"'
- documentIndex: 0
matchRegex:
path: spec.validations[0].expression
pattern: '!h\.startsWith\("\*\."\) && !\(h == "example\.org"'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

Update the wildcard test assertions to match the new case-insensitive CEL expression using .lowerAscii().

    asserts:
      - documentIndex: 0
        matchRegex:
          path: spec.validations[0].expression
          pattern: '!r\.host\.lowerAscii\(\)\.startsWith\("\*\."\) && !\(r\.host\.lowerAscii\(\) == "example\.org"'
      - documentIndex: 0
        matchRegex:
          path: spec.validations[0].expression
          pattern: '!h\.lowerAscii\(\)\.startsWith\("\*\."\) && !\(h\.lowerAscii\(\) == "example\.org"'

namespace: tenant-test
spec:
rules:
- host: "dashboard.example.org"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

Use a mixed-case hostname (e.g., "Dashboard.Example.Org") to verify that the policy is case-insensitive and correctly rejects foreign-apex hostnames regardless of case.

  - host: "Dashboard.Example.Org"

Comment on lines +612 to +617
tls:
- hosts:
- "harbor.test.example.org"
secretName: noop-tls
rules:
- host: "harbor.test.example.org"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

Use mixed-case hostnames (e.g., "Harbor.Test.Example.Org") to verify that the policy correctly allows in-apex hostnames regardless of case.

  tls:
  - hosts:
    - "Harbor.Test.Example.Org"
    secretName: noop-tls
  rules:
  - host: "Harbor.Test.Example.Org"

5. **`cozystack-namespace-host-label-policy`** — VAP on core `v1 Namespace` CREATE/UPDATE. Rejects any set or change of the `namespace.cozystack.io/host` label, except by the same trusted-caller whitelist as layer 4. This closes both first-time label writes on CREATE and first-time adds on UPDATE — only cozystack/Flux service accounts (which apply the tenant chart) can stamp the label.
6. **Render-time `fail` in cozystack-basics.** The cozystack-basics chart fails the helm render if `_cluster.gateway-attached-namespaces` contains any `tenant-*` entry. Triggers on the helm-install path before the cluster ever sees the values — complements layer 3 which triggers at `kubectl apply` time.
7. **`cozystack-route-hostname-policy`** — VAP on `gateway.networking.k8s.io/v1 HTTPRoute` and `v1alpha2 TLSRoute` CREATE/UPDATE. Scoped to `tenant-*` namespaces (cozy-* are cluster-admin-managed and trusted to publish under any apex). Rejects any `spec.hostnames` entry that is not equal to the namespace's `namespace.cozystack.io/host` label or a subdomain of it. Defense-in-depth against an app chart bug or supply-chain compromise that emits Gateway API resources outside the tenant's apex — tenants in Cozystack do not hold `gateway.networking.k8s.io/*` RBAC by design, so this is not a tenant-user defense. The within-apex cross-namespace case (a tenant chart claiming a hostname that is published by a `cozy-*` app) is handled by the controller at reconciliation time: when two routes from different namespaces claim the same hostname, the `cozy-*` namespace wins and the loser receives a `HostnameConflict` condition under the controller's name in `Status.Parents`.
8. **`cozystack-ingress-hostname-policy`** — VAP on core `networking.k8s.io/v1 Ingress` CREATE/UPDATE. Gateway API is opt-in and **off by default**, so in the default configuration tenant applications publish through a legacy Ingress on the shared ingress-nginx; layers 1-7 constrain tenant hostnames on the opt-in Gateway path, and this VAP adds a hostname constraint on the default Ingress path. Scoped to `tenant-*` namespaces (cozy-* are cluster-admin-managed and trusted). A hostname on `spec.rules[].host` or `spec.tls[].hosts[]` is allowed when it is within the namespace's own `namespace.cozystack.io/host` apex, OR when it lies entirely outside the platform root apex (`_cluster.root-host`) — the second case lets a tenant route its own external custom domain (for example the `kubernetes` app's Proxied `addons.ingressNginx.hosts`, which routes a user-supplied domain to a nested cluster). A hostname that falls under the platform root apex but outside the namespace's own apex is rejected; a rule with no host (an unbounded catch-all) and `spec.defaultBackend` (a catch-all for unmatched traffic) are also rejected. Fail-closed: a `tenant-*` namespace missing its host label is denied, and the policy renders only when `_cluster.root-host` is set. This bounds which apexes a tenant Ingress may claim under the platform domain; it does not attempt to resolve every possible hostname collision.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The word "bounds" is used as a verb here, but "limits" or "constrains" is more idiomatic and clear in this context.

Suggested change
8. **`cozystack-ingress-hostname-policy`** — VAP on core `networking.k8s.io/v1 Ingress` CREATE/UPDATE. Gateway API is opt-in and **off by default**, so in the default configuration tenant applications publish through a legacy Ingress on the shared ingress-nginx; layers 1-7 constrain tenant hostnames on the opt-in Gateway path, and this VAP adds a hostname constraint on the default Ingress path. Scoped to `tenant-*` namespaces (cozy-* are cluster-admin-managed and trusted). A hostname on `spec.rules[].host` or `spec.tls[].hosts[]` is allowed when it is within the namespace's own `namespace.cozystack.io/host` apex, OR when it lies entirely outside the platform root apex (`_cluster.root-host`) — the second case lets a tenant route its own external custom domain (for example the `kubernetes` app's Proxied `addons.ingressNginx.hosts`, which routes a user-supplied domain to a nested cluster). A hostname that falls under the platform root apex but outside the namespace's own apex is rejected; a rule with no host (an unbounded catch-all) and `spec.defaultBackend` (a catch-all for unmatched traffic) are also rejected. Fail-closed: a `tenant-*` namespace missing its host label is denied, and the policy renders only when `_cluster.root-host` is set. This bounds which apexes a tenant Ingress may claim under the platform domain; it does not attempt to resolve every possible hostname collision.
8. **`cozystack-ingress-hostname-policy`** — VAP on core `networking.k8s.io/v1 Ingress` CREATE/UPDATE. Gateway API is opt-in and **off by default**, so in the default configuration tenant applications publish through a legacy Ingress on the shared ingress-nginx; layers 1-7 constrain tenant hostnames on the opt-in Gateway path, and this VAP adds a hostname constraint on the default Ingress path. Scoped to `tenant-*` namespaces (cozy-* are cluster-admin-managed and trusted). A hostname on `spec.rules[].host` or `spec.tls[].hosts[]` is allowed when it is within the namespace's own `namespace.cozystack.io/host` apex, OR when it lies entirely outside the platform root apex (`_cluster.root-host`) — the second case lets a tenant route its own external custom domain (for example the `kubernetes` app's Proxied `addons.ingressNginx.hosts`, which routes a user-supplied domain to a nested cluster). A hostname that falls under the platform root apex but outside the namespace's own apex is rejected; a rule with no host (an unbounded catch-all) and `spec.defaultBackend` (a catch-all for unmatched traffic) are also rejected. Fail-closed: a `tenant-*` namespace missing its host label is denied, and the policy renders only when `_cluster.root-host` is set. This limits which apexes a tenant Ingress may claim under the platform domain; it does not attempt to resolve every possible hostname collision.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (4)
packages/extra/gateway/README.md (1)

94-94: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Layer 8 description omits the wildcard-vs-outside-root rejection edge case.

The bullet documents the own-apex and outside-root-apex allow paths and the under-root-outside-apex / hostless / defaultBackend deny paths, but doesn't mention that a wildcard hostname taken through the outside-root path is also denied (the CEL gates the outside-root branch on !r.host.startsWith("*."), and this is specifically exercised by the new e2e test for *.org in hack/e2e-apps/gateway.bats). Worth a short clause noting wildcards are rejected on the outside-root path (while a wildcard under the tenant's own apex is still allowed).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/extra/gateway/README.md` at line 94, Update the Layer 8 bullet in
the gateway README to mention the wildcard edge case: in the hostname policy
description, add that the “outside the platform root apex” allow path does not
permit wildcard hostnames, while wildcards remain allowed only when they stay
within the namespace’s own apex. Refer to the
`cozystack-ingress-hostname-policy` description and keep the wording aligned
with the existing own-apex/outside-root allow and deny clauses.
hack/e2e-apps/gateway.bats (3)

404-699: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

No e2e coverage for a tenant-* namespace missing its own host label.

The README (Layer 8 description) states the policy fails closed when a tenant-* namespace lacks its namespace.cozystack.io/host label, but none of the new tests exercise that path end-to-end (only the helm-unittest layer, per the stack outline, presumably covers it). Consider adding one e2e probe against a tenant-* namespace without the label to pin the fail-closed contract at the live-apiserver layer too, matching the existing diagnostic test pattern at lines 62-78.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-apps/gateway.bats` around lines 404 - 699, Add an e2e test that
exercises the fail-closed path for a tenant-* namespace missing its
namespace.cozystack.io/host label, since the current gateway.bats coverage only
probes labeled namespaces. Use the existing diagnostic/probe style from
cozystack-ingress-hostname-policy and create a namespace-scoped Ingress in a
tenant-* namespace without the label, then assert the ValidatingAdmissionPolicy
rejects it with the expected admission failure. Keep the new test alongside the
other ingress hostname policy cases so the live-apiserver contract is pinned in
the same suite.

501-533: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Weaker assertions for hostless/empty-host/defaultBackend rejections.

These three tests only assert grep -qi "ValidatingAdmissionPolicy" on rejection, unlike the apex-violation tests which also assert the specific "must be within" substring from the policy's shared messageExpression. Since the policy uses one message for every violation branch, the same substring check would also apply here and slightly strengthen these tests against false positives from an unrelated VAP/webhook rejection.

Also applies to: 535-570, 572-598

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-apps/gateway.bats` around lines 501 - 533, The rejection checks in
the hostless, empty-host, and defaultBackend ingress tests are too weak because
they only match ValidatingAdmissionPolicy generically. Update the affected test
cases in gateway.bats, including the cozystack-ingress-hostname-policy VAP test
and the related hostless/empty-host/defaultBackend assertions, to also verify
the shared policy message substring "must be within" alongside the existing
ValidatingAdmissionPolicy check. This should mirror the stronger apex-violation
tests and reduce false positives from unrelated admission failures.

428-457: 🩺 Stability & Availability | 🔵 Trivial | ⚖️ Poor tradeoff

Inline cleanup inside test bodies, per existing file convention.

Each new test performs its own pre-clean and/or post-apply kubectl delete cleanup inline rather than relying on framework-level teardown. This mirrors the file's pre-existing pattern (used throughout the rest of gateway.bats), so it's consistent with current style — but it repeats the same issue previously flagged for harbor.bats.

Based on learnings, in E2E bats tests like hack/e2e-apps/harbor.bats, inline cleanup should be replaced by framework-level teardown/cleanup, applying "to all bats tests under hack/e2e-apps (and similar E2E suites) to ensure consistent and reliable cleanup without embedding it in individual tests."

Given the scope of an already-established pattern across dozens of existing tests in this file, a full migration here would be high effort; flagging for awareness rather than blocking this PR.

Also applies to: 466-499, 506-533, 542-570, 576-598, 604-630, 639-661, 670-698

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-apps/gateway.bats` around lines 428 - 457, The new gateway.bats
tests are doing inline pre/post test cleanup with repeated kubectl delete calls
inside the test bodies, which should be moved to the suite’s shared
teardown/cleanup pattern instead. Update the affected test blocks in
gateway.bats (including the ingress-out-of-apex-host-probe flow and the other
newly added cases) to rely on framework-level cleanup helpers rather than
embedding deletion logic in each test. Keep the test bodies focused on setup,
apply, and assertions, and use the existing bats teardown/cleanup conventions
already used elsewhere in the suite.

Source: Learnings

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@hack/e2e-apps/gateway.bats`:
- Around line 404-699: Add an e2e test that exercises the fail-closed path for a
tenant-* namespace missing its namespace.cozystack.io/host label, since the
current gateway.bats coverage only probes labeled namespaces. Use the existing
diagnostic/probe style from cozystack-ingress-hostname-policy and create a
namespace-scoped Ingress in a tenant-* namespace without the label, then assert
the ValidatingAdmissionPolicy rejects it with the expected admission failure.
Keep the new test alongside the other ingress hostname policy cases so the
live-apiserver contract is pinned in the same suite.
- Around line 501-533: The rejection checks in the hostless, empty-host, and
defaultBackend ingress tests are too weak because they only match
ValidatingAdmissionPolicy generically. Update the affected test cases in
gateway.bats, including the cozystack-ingress-hostname-policy VAP test and the
related hostless/empty-host/defaultBackend assertions, to also verify the shared
policy message substring "must be within" alongside the existing
ValidatingAdmissionPolicy check. This should mirror the stronger apex-violation
tests and reduce false positives from unrelated admission failures.
- Around line 428-457: The new gateway.bats tests are doing inline pre/post test
cleanup with repeated kubectl delete calls inside the test bodies, which should
be moved to the suite’s shared teardown/cleanup pattern instead. Update the
affected test blocks in gateway.bats (including the
ingress-out-of-apex-host-probe flow and the other newly added cases) to rely on
framework-level cleanup helpers rather than embedding deletion logic in each
test. Keep the test bodies focused on setup, apply, and assertions, and use the
existing bats teardown/cleanup conventions already used elsewhere in the suite.

In `@packages/extra/gateway/README.md`:
- Line 94: Update the Layer 8 bullet in the gateway README to mention the
wildcard edge case: in the hostname policy description, add that the “outside
the platform root apex” allow path does not permit wildcard hostnames, while
wildcards remain allowed only when they stay within the namespace’s own apex.
Refer to the `cozystack-ingress-hostname-policy` description and keep the
wording aligned with the existing own-apex/outside-root allow and deny clauses.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 12960f9b-cec0-49ef-9605-ca467d33fbd0

📥 Commits

Reviewing files that changed from the base of the PR and between 10f8e2d and 5a408f7.

📒 Files selected for processing (4)
  • hack/e2e-apps/gateway.bats
  • packages/extra/gateway/README.md
  • packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml
  • packages/system/cozystack-basics/tests/ingress-hostname-policy_test.yaml

@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/ingress-hostname-policy branch 2 times, most recently from 1c136c3 to d2be06b Compare July 4, 2026 16:48

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml (1)

62-86: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoff

Optional: hoist the repeated apex/root checks into spec.variables.

The validator repeats the namespace-apex accessor and root-apex comparison several times across the rules and TLS branches. Defining them once would make this security-critical CEL easier to audit and reduce the chance of future edits diverging between branches. ValidatingAdmissionPolicy supports spec.variables in admissionregistration.k8s.io/v1.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml`
around lines 62 - 86, The CEL in the ValidatingAdmissionPolicy repeats the same
namespace-apex and root-apex checks in both the rules and TLS branches, making
it harder to audit and maintain. Hoist the shared expressions into
spec.variables on cozystack-ingress-hostname-policy, then reference those
variables from the existing $celValidator/messageExpression logic so the
namespaceObject.metadata.labels and rootHost comparisons are defined once and
used consistently throughout the policy.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml`:
- Around line 62-86: The CEL in the ValidatingAdmissionPolicy repeats the same
namespace-apex and root-apex checks in both the rules and TLS branches, making
it harder to audit and maintain. Hoist the shared expressions into
spec.variables on cozystack-ingress-hostname-policy, then reference those
variables from the existing $celValidator/messageExpression logic so the
namespaceObject.metadata.labels and rootHost comparisons are defined once and
used consistently throughout the policy.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 12d4742d-de72-49b1-8c39-dfacf79c4ca6

📥 Commits

Reviewing files that changed from the base of the PR and between 1c136c3 and d2be06b.

📒 Files selected for processing (4)
  • hack/e2e-apps/gateway.bats
  • packages/extra/gateway/README.md
  • packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml
  • packages/system/cozystack-basics/tests/ingress-hostname-policy_test.yaml
✅ Files skipped from review due to trivial changes (1)
  • packages/extra/gateway/README.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • hack/e2e-apps/gateway.bats
  • packages/system/cozystack-basics/tests/ingress-hostname-policy_test.yaml

…tenant apex

The Gateway (cozystack-gateway-hostname-policy) and Route
(cozystack-route-hostname-policy) admission policies constrain tenant
hostnames on the Gateway API path. Gateway API is opt-in and off by
default, so in the default configuration tenant applications publish
through a legacy networking.k8s.io/Ingress on the shared ingress-nginx,
which had no equivalent policy.

Add a fail-closed ValidatingAdmissionPolicy + Binding for Ingress in
tenant-* namespaces. A hostname on spec.rules[].host or spec.tls[].hosts[]
is allowed when it is within the namespace's own apex
(namespace.cozystack.io/host), or when it is a concrete (non-wildcard)
name entirely outside the platform root apex (_cluster.root-host); a
hostname under the platform root apex but outside the namespace's own
apex, and a wildcard taken through the outside-root path (which could
match the platform apex itself), are rejected. The outside-root branch
keeps the kubernetes app's Proxied exposeMethod working, since it routes
a user-supplied external domain to a nested cluster. A rule with no host
(absent or empty, an unbounded catch-all) and spec.defaultBackend are
also rejected.

The policy templates the platform root apex into its CEL and renders
only when _cluster.root-host is set, so it is never shipped in a state
that would admit every hostname. A tenant-* namespace missing its own
apex label is denied.

Cover the policy with helm-unittest (render, including the root-host,
empty-host, and wildcard guards) and behavioral admission tests against
the live apiserver (in-apex allow, external-domain allow,
own-apex-wildcard allow, external-tls allow, under-root-outside-apex
deny, prefix-adjacent-under-root deny, wildcard-matching-apex deny,
hostless deny, empty-string-host deny, defaultBackend deny), and document
it as an additional layer in the gateway security model.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml`:
- Around line 52-62: The fail-closed precondition in the ingress hostname policy
only checks for the presence of namespace.cozystack.io/host, so an empty string
can still bypass the intended rejection path. Update the CEL guard in the
$celValidator template to treat a blank namespace.cozystack.io/host value the
same as missing, while keeping the existing namespaceObject and labels checks
intact. Locate the logic around $rootHost, $A, and the generated expression in
the ingress-hostname-policy.yaml template and add an explicit empty-value
condition before allowing the custom-domain branch.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6f1fb6f2-9da7-4c9f-b63d-bfec6d11258e

📥 Commits

Reviewing files that changed from the base of the PR and between d2be06b and d89ad70.

📒 Files selected for processing (4)
  • hack/e2e-apps/gateway.bats
  • packages/extra/gateway/README.md
  • packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml
  • packages/system/cozystack-basics/tests/ingress-hostname-policy_test.yaml
✅ Files skipped from review due to trivial changes (1)
  • packages/extra/gateway/README.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • packages/system/cozystack-basics/tests/ingress-hostname-policy_test.yaml
  • hack/e2e-apps/gateway.bats

@IvanHunters

Copy link
Copy Markdown
Collaborator

Verdict

LGTM with non-blocking notes

The CEL policy enforces exactly what the PR body and README claim; I verified the allow/deny logic against a full truth table and the sibling Route policy, and found no bypass of the platform apex or a sibling tenant. One upgrade-impact edge case on pre-existing non-conforming Ingresses is undocumented (MINOR).

Findings

[MINOR] packages/system/cozystack-basics/templates/ingress-hostname-policy.yaml:71 — existing non-conforming tenant Ingress blocks its own HelmRelease on next reconcile, undocumented

The VAP uses failurePolicy: Fail + validationActions: [Deny] on CREATE/UPDATE. It does not touch already-created objects, but the next Flux reconcile / operator re-apply of a pre-existing tenant Ingress issues an UPDATE, which admission then rejects — wedging that HelmRelease. The realistic trigger is the kubernetes app's Proxied path (packages/apps/kubernetes/templates/ingress.yaml:25, host from user-supplied addons.ingressNginx.hosts): a customer who set that host to a name under root-host but outside the tenant's own apex (e.g. k8s.example.org while the apex is foo.example.org) has a working Ingress today that this PR will start rejecting on the next reconcile. That configuration is a cross-scope collision the policy intends to forbid, so denying it is arguably correct, but the upgrade transition is silent: the operator sees a stuck HR, not a clear "your Ingress host is out of apex" signal at upgrade time. Neither the PR body's upgrade section nor packages/extra/gateway/README.md:94 mentions this existing-Ingress transition. Recommend a sentence in the README / release note noting that pre-existing tenant Ingresses with an out-of-apex host under the root domain will fail reconciliation after upgrade and must be re-hosted, so operators can audit before rolling out. In-tree charts are safe: harbor defaults its host to <release>.<tenant-apex> (packages/apps/harbor/templates/ingress.yaml:4, in-apex); tenant/seaweedfs/bootbox Ingresses are either in-apex or land in cozy-* namespaces the policy does not gate.

Claim mismatches

[OK] All PR-body claims verified. Specifically checked and confirmed:

  • Own-apex exact + subdomain allowed; sibling-under-root (bar.example.org) denied; bare platform apex (example.org) denied; under-root-outside-apex denied via the endsWith("." + apex) dot-boundary — no prefix-adjacency bypass (eviltest.example.org denied).
  • Concrete outside-root name allowed (Proxied external-domain path preserved).
  • Wildcard through outside-root path denied: the !h.startsWith("*.") gate plus the apiserver's own Ingress host validation (wildcards must be exactly *.<subdomain>, bare * / *foo / trailing-dot FQDNs rejected) close every admissible wildcard shape; *.org, *.example.org, *.customer.com all denied. Own-apex wildcard (*.test.example.org) allowed as intended.
  • Fail-closed on missing apex label (? false : branch) and policy omitted when root-host unset (verified via helm template with and without the value, and the unittest suite — 46/46 pass).
  • Empty/absent host and spec.defaultBackend denied; the r.host != "" guard correctly prevents an empty host from slipping through the outside-root branch.

Caveats

  • Threat model confirmed accurate: packages/system/cozystack-basics/templates/clusterroles.yaml:27-29 grants tenants only get,list,watch on ingresses, so this is genuine defense-in-depth against chart bugs / supply-chain / confused-deputy, not a tenant-user gate — bypasses here are correspondingly lower-severity.
  • Phase 5b fresh-install: no new PackageSource / bundle wiring / cert-manager dep / CRD / image / values.yaml-schema change. VAP/VAPBinding are apiserver-native (admissionregistration.k8s.io/v1, GA >=1.30). root-host is a mandatory platform value (set at install; migrations/7 patches it into the cozystack ConfigMap), so the policy always renders on real clusters — the "omit when unset" path is only for degenerate/test states. No cold-start dependency. No make generate artifacts required. Clean.
  • Phase 5b upgrade: no migration required; the only upgrade risk is the pre-existing-Ingress transition captured in Findings.
  • CEL evaluated by hand-simulating the exact string operations (no CEL evaluator available in-env); cross-checked against the e2e admission tests in hack/e2e-apps/gateway.bats which exercise the same vectors against a live apiserver. e2e tests use the correct @test shape (no silently-dead setup_file/teardown hooks) and poll durable outcomes (apply accept/reject), not hook resources.

@lexfrei
Aleksei Sviridkin (lexfrei) merged commit a44d11d into main Jul 4, 2026
20 of 21 checks passed
@lexfrei
Aleksei Sviridkin (lexfrei) deleted the fix/ingress-hostname-policy branch July 4, 2026 21:18
Aleksei Sviridkin (lexfrei) added a commit that referenced this pull request Jul 6, 2026
## What this PR does

Adds `docs/security/threat-model.md`, a design-level threat model for
Cozystack. It documents the actors, the actions that cross trust
boundaries, and the security goals and non-goals, organized by trust
boundary (T0–T4) following the CNCF TAG-Security Actors/Actions model.

It records the usage and multitenancy model (managed-services backend vs
direct dashboard/API access; combined hard multitenancy for tenant VMs
and Kubernetes clusters, soft multitenancy for managed data services),
the primary trust-boundary invariant (tenant RBAC grants no HelmRelease
or arbitrary core writes; the aggregated API server fixes the chart
reference server-side), and the two admission enforcement points (core
kube-apiserver vs `cozystack-api`). Hostname integrity covers both the
Gateway API path and the default legacy-Ingress path
(`cozystack-ingress-hostname-policy`, #3200). Every architectural claim
cites a source file so the model can be re-verified as the code evolves.

This is the in-repo companion to the project's CNCF TAG-Security
self-assessment, which references it.

### Screenshots

N/A — documentation only.

### Release note

```release-note
docs(security): add a design-level threat model (docs/security/threat-model.md)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Added a security threat model for the platform.
* Clarified system boundaries, trust tiers, and the main actors
involved.
* Documented key security goals, non-goals, and important enforcement
points.
* Described expected behavior for tenant actions, hostname handling, and
cross-boundary access.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/networking Issues or PRs related to networking (ingress, gateway, vpn, metallb, kube-ovn) area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review kind/bug Categorizes issue or PR as related to a bug kind/feature Categorizes issue or PR as related to a new feature size/XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants