Skip to content

fix(tests): reject unreadable RD preset schemas - #4191

Open
myasnikovdaniil wants to merge 1 commit into
mainfrom
fix/rd-presets-read-failure
Open

myasnikovdaniil wants to merge 1 commit into
mainfrom
fix/rd-presets-read-failure

Conversation

@myasnikovdaniil

@myasnikovdaniil myasnikovdaniil commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

rd-presets-check discards jq errors, so malformed schema can pass the check or appear as missing presets. It now fails with the schema path and jq diagnostic. An absent schema or a readable schema without resourcesPreset is still skipped.

Regression tests use real jq for malformed input and partial output. They also cover the canonical preset set, exact matching and the long-list case fixed by #4204.

Testing

  • make unit-tests -j4 --output-sync=target passed.
  • Nine cases in hack/check-rd-presets_test.bats, run with bats and dash hack/cozytest.sh.
  • Both read-failure cases fail against main. Eleven deliberate code mutations also fail the expected test.
  • shellcheck hack/check-rd-presets.sh hack/check-rd-presets_test.bats.

Screenshots

Not applicable.

Downstream repositories

  • No downstream repository is affected by this change

No files or make targets were moved or renamed. The change is limited to the RD checker and its tests.

Release note

fix(tests): Fail the RD preset check when a schema cannot be read, with the schema path and jq diagnostic.

@coderabbitai

coderabbitai Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: cozystack/cozystack/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 774c2782-3aa7-49ac-9634-fb6faa2534a0

📥 Commits

Reviewing files that changed from the base of the PR and between 39f4c34 and 47fe046.

📒 Files selected for processing (2)
  • hack/check-rd-presets.sh
  • hack/check-rd-presets_test.bats

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.


📝 Walkthrough

Walkthrough

The preset checker now captures and reports jq errors when it cannot read a schema, marks that schema check as failed, and continues to the next schema. Its Bats tests cover schema failures, absent schemas, canonical preset checks, and large output.

Changes

Preset check reliability

Layer / File(s) Summary
Handle jq schema read failures
hack/check-rd-presets.sh
The script captures jq error output, reports failed schema reads, and skips preset checks for affected files. The final failure message directs users to run make generate in the affected chart directory.
Test schema handling and preset validation
hack/check-rd-presets_test.bats
Tests cover malformed schemas, partial jq output, absent schemas, canonical and missing preset names, large output, and the exact 47-name expected list.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 47fe0

The checker retains safe preset matching and reports schema-read failures; the added tests cover these behaviors. No concrete merge-blocking regression is established.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the change that rejects unreadable RD preset schemas. This change is present in the script and test updates.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added area/testing Issues or PRs related to testing (e2e, bats, unit tests) kind/bug Categorizes issue or PR as related to a bug size/M This PR changes 30-99 lines, ignoring generated files labels Sep 10, 2026
@lexfrei

Copy link
Copy Markdown
Contributor

The Broken pipe in those logs is not the jq pipe. It's line 39, the preset loop:

if ! printf '%s\n' "$enums" | grep -Fqx -- "$want"; then

grep -q exits on the first match and closes the pipe, printf takes EPIPE, pipefail promotes it into the pipeline status, and ! reads that as "not found". The value has to be present for the check to call it missing.

bash names the last line of a multi-line command in that message, so the jq substitution would have reported line 32:

$ cat -n t.sh
     5  out=$(printf '%s\n' "$big" \
     6    | head -n 1 \
     7    || true)
     9  printf '%s\n' "$big" | grep -Fqx -- "1"
$ bash t.sh
t.sh: line 7: printf: write error: Broken pipe
t.sh: line 9: printf: write error: Broken pipe

The one-preset FAIL says the same. Truncated jq output is a prefix, so everything past the cut goes missing at once, and rabbitmq.yaml holds a single copy of the enum with s1.xlarge near the middle, so that run would have listed about half the set. It listed one. Four reds, one preset and one Broken pipe in each: rabbitmq/s1.xlarge and postgres/m1.small on 09-09, harbor/c1.medium and http-cache/t1.2xlarge in run 34454715175 attempts 1 and 2 today. The loop runs 21 files x 47 values per run and one of those 987 pipelines loses the race.

Line 39 needs the same here-string you gave jq:

if ! grep -Fqx -- "$want" <<<"$enums"; then

Without that the check keeps going red after this merges, with a different file and preset each time.

@github-actions github-actions Bot added size/L This PR changes 100-499 lines, ignoring generated files and removed size/M This PR changes 30-99 lines, ignoring generated files labels Sep 10, 2026
@myasnikovdaniil myasnikovdaniil changed the title fix(tests): stop the RD preset check turning a failed read into drift fix(tests): stop the RD preset check reporting presets that are present Sep 10, 2026
@myasnikovdaniil
myasnikovdaniil force-pushed the fix/rd-presets-read-failure branch from 654c6dd to 39f4c34 Compare September 10, 2026 12:01
@lexfrei

Copy link
Copy Markdown
Contributor

This is now stopping E2E from starting on other PRs, not only reddening the unit lane. On #3802 two runs died the same way with a different victim each time, and E2E (in-tree) was skipped both times because the unit lane runs first:

run 34560955738   opensearch-rd     missing: c1.4xlarge
run 34565433782   tcp-balancer-rd   missing: m1.large

Both presets are present in those files and hack/check-rd-presets.sh passes on the same tree locally, so that PR has no way to reach a green run while this sits on main.

This PR and #4194 rewrite the same membership check, so whichever lands second conflicts on that line. Which of the two should go in?

@lexfrei

Copy link
Copy Markdown
Contributor

The here-string part landed in #4204, so after a rebase that line resolves to what main has. The jq status guard and the bats suite are still new. Main has no test for this script, so both are worth keeping.

Discarding jq's exit status lets malformed schemas pass the preset check
or turns a partial read into a misleading missing-preset report.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <[email protected]>
@myasnikovdaniil
myasnikovdaniil force-pushed the fix/rd-presets-read-failure branch from 39f4c34 to 47fe046 Compare September 28, 2026 09:32
@myasnikovdaniil myasnikovdaniil changed the title fix(tests): stop the RD preset check reporting presets that are present fix(tests): reject unreadable RD preset schemas Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/testing Issues or PRs related to testing (e2e, bats, unit tests) kind/bug Categorizes issue or PR as related to a bug size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants