Skip to content

test(ci): pin the label-writing workflows with contract tests - #3664

Open
Shaurya Kesarwani (shaurya703) wants to merge 2 commits into
cozystack:mainfrom
shaurya703:label-workflows-contract
Open

Shaurya Kesarwani (shaurya703) wants to merge 2 commits into
cozystack:mainfrom
shaurya703:label-workflows-contract

Conversation

@shaurya703

@shaurya703 Shaurya Kesarwani (shaurya703) commented Aug 8, 2026 •

Copy link
Copy Markdown

Closes #3589

Adds hack/label-workflows-contract.bats, following the pattern of hack/promote-gate-contract.bats / hack/release-freeze-contract.bats (same helpers: comment-stripped pins over job/input blocks, one filter per language since pr-labeler.yaml mixes YAML # and JavaScript // comments).

What it pins

Cross-file label resolution (the headline check from the issue):

  • every value in typeToKind and scopeToArea, plus the three literal toAdd.add(...) labels (area/release, kind/breaking-change, area/uncategorized), resolves against a name: in .github/labels.yml
  • stale.yaml's four label inputs and all 16 exempt entries (12 issue + 4 PR) resolve the same way; failures name the missing label

Structure:

  • trigger list stays [opened, reopened, synchronize] — edited cannot creep back in as an executable line
  • exactly the five documented type mappings; a mapping demoted to a // comment stops counting
  • additive-only: addLabels once, no removeLabel/setLabels, !existing.has(l) filter present, area/uncategorized fallback present
  • stale policy: label choices, exempt-list sizes, days-before-stale: 60, days-before-close: 14, operations-per-run: 100, remove-stale-when-updated: true, and the specific renewal-path exemptions (lifecycle/frozen, triage/accepted, do-not-merge/hold)

Verification

  • Passes under both bats and hack/cozytest.sh (POSIX-only constructs — no process substitution; the first draft used one and cozytest.sh caught it, which is a nice validation of running both)
  • Mutation-checked per the issue's bar — each of these went red and was restored: typo in a scopeToArea value, edited added to the trigger list, fallback line demoted to //, days-before-stale changed, typo in an exempt label, feat mapping demoted to //, typo in the literal kind/breaking-change
  • Picked up automatically by BATS_UNIT_FILES (wildcard) — no Makefile change
  • The two documented drift examples (the edited event claim and the revert type in docs/agents/contributing.md) are left as-is: they are doc statements, and which side is authoritative is a maintainer call. The tests pin the workflow's current executable behavior.

Summary by CodeRabbit

  • Tests
    • Added automated validation for pull request labeling and stale-item workflows.
    • Verifies label definitions, event triggers, category mappings, fallback behavior, exemptions, timing settings, and workflow limits.
    • Helps ensure labels are applied consistently and stale-item handling follows the configured rules.

@github-actions github-actions Bot added the size/L This PR changes 100-499 lines, ignoring generated files label Aug 8, 2026
@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Added a Bats contract test suite for pr-labeler.yaml and stale.yaml. The suite validates workflow mappings, triggers, label definitions, fallback behavior, exemptions, timing values, operation limits, and stale-label handling.

Changes

Workflow contract tests

Layer / File(s) Summary
Workflow parsing and label validation
hack/label-workflows-contract.bats
Adds parsers for workflow content, embedded JavaScript, label definitions, stale inputs, and exemption lists. Validates that referenced labels exist.
Pull-request labeling contracts
hack/label-workflows-contract.bats
Pins the five pull-request mappings and required triggers. Verifies duplicate filtering, additive labeling, and the uncategorized fallback.
Stale policy contracts
hack/label-workflows-contract.bats
Pins stale and close labels, exemptions, timing values, operation limits, stale-label removal, daily scheduling, and manual dispatch.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested labels: area/testing

Suggested reviewers: lexfrei

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The contract tests cover most requirements, but the documented edited event and revert type discrepancies remain unfixed as required by issue #3589. Update the documentation or workflow so the edited trigger and revert type agree, then extend the contract assertions if needed.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the contract tests added to pin the label-writing workflows.
Out of Scope Changes check ✅ Passed The changes are limited to the requested contract test suite under hack/ and contain no unrelated code changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the area/ci Issues or PRs related to CI workflows, GitHub Actions, automation label Aug 8, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (8)
hack/label-workflows-contract.bats (8)

87-89: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

The grep -v '^typeToKind\|^scopeToArea' filter is dead code.

Line 88 selects only toAdd.add('<literal>') occurrences, because the pattern requires single quotes. Calls of the form toAdd.add(typeToKind[type]) never match. After sed, every remaining value is a label string, so no line can start with typeToKind or scopeToArea. Remove the filter, or keep it and add a comment that states it is a guard against a future literal-string regression.

♻️ Proposed change
   literal_labels="$(code_lines < "$PR_LABELER" | script_lines \
-    | grep -o "toAdd.add('[^']*')" | sed "s/toAdd.add('//; s/')//" \
-    | grep -v '^typeToKind\|^scopeToArea' || true)"
+    | grep -o "toAdd.add('[^']*')" | sed "s/toAdd.add('//; s/')//" || true)"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 87 - 89, Remove the
redundant grep -v filter from the literal_labels pipeline after extracting
toAdd.add(...) values, leaving the existing literal-label matching and sed
transformation unchanged.

62-69: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

The local keyword contradicts the "POSIX-only" comment.

Line 63 states the helpers stay POSIX because hack/cozytest.sh runs the file in plain sh. Line 64 then uses local, which POSIX does not define. dash and busybox ash accept it, so this is not a runtime failure today. Either drop the POSIX claim, or remove local. The same pattern appears at lines 22 and 28.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 62 - 69, Remove the
non-POSIX local declarations from the helper functions containing the
all/missing logic and the matching declarations near lines 22 and 28. Preserve
the existing variable assignments and behavior while keeping the POSIX-only
claim accurate for hack/cozytest.sh.

112-114: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

The edited guard misses the YAML block-sequence form.

Line 114 inspects only lines that match ^[[:space:]]*types:. If someone rewrites the trigger as a block sequence, edited appears on its own line and the guard passes:

    types:
      - opened
      - reopened
      - edited

Line 110 would also fail in that case, so the test still fails overall. However the negative assertion itself gives no protection. Scan the whole on: section for edited instead.

♻️ Proposed change
-  ! code_lines < "$PR_LABELER" | grep -E '^[[:space:]]*types:' | grep -q 'edited'
+  # Covers both the flow-sequence and block-sequence spellings of `types:`.
+  ! code_lines < "$PR_LABELER" | grep -qw 'edited'
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 112 - 114, Update the
negative assertion in the label-workflow contract test to scan the entire on:
section for edited rather than only types: lines, so both inline and YAML
block-sequence forms are detected. Preserve the existing intent that edited must
not appear as an executable trigger value.

48-59: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Hard-coded indentation makes these helpers brittle.

stale_input and stale_exempt_list require exactly 10 spaces before the key and exactly 12 spaces for folded continuation lines. A pure reformat of stale.yaml then produces an empty value, and the failure message does not explain the cause. This is acceptable for a pinning contract test. Consider matching a looser indentation prefix, or add a guard that fails with an explicit message when the key is not found.

♻️ Optional: loosen the indentation requirement
 stale_input() {
-  code_lines < "$STALE" | awk -v key="          $1: " '
-    index($0, key) == 1 { sub(key, ""); print; exit }'
+  code_lines < "$STALE" | awk -v key="$1" '
+    $1 == key ":" { sub(/^[[:space:]]*[^:]+:[[:space:]]*/, ""); print; exit }'
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 48 - 59, Update stale_input
and stale_exempt_list to avoid relying on fixed indentation when parsing STALE,
while preserving their current extraction behavior; alternatively, add explicit
guards that fail with a clear message when the expected key or folded list is
not found.

35-40: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Strip // comments before block detection, not after.

js_object_block finds the opening line with index($0, name) on the raw file. A commented line such as // const typeToKind = { also opens the block. The script_lines filter runs after awk, so it cannot prevent that. This weakens the stated contract that commented-out code must never satisfy an assertion.

♻️ Proposed change
 js_object_block() {
-  awk -v name="const $1 = {" '
+  script_lines < "$2" | awk -v name="const $1 = {" '
     index($0, name) { inside = 1; next }
     inside && /^[[:space:]]*};[[:space:]]*$/ { exit }
-    inside' "$2" | script_lines
+    inside'
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 35 - 40, Update
js_object_block so each input line has // comments removed before evaluating the
opening-pattern match and block boundaries. Ensure commented-out declarations
such as // const typeToKind = { never set inside, while preserving extraction of
active object blocks for subsequent script_lines processing.

142-164: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider pinning the stale.yaml trigger list.

The stated objective is to pin the trigger list for both workflows. The suite pins the pr-labeler.yaml triggers at line 110. It does not pin the stale.yaml triggers. stale.yaml runs on schedule with cron 37 4 * * * and on workflow_dispatch. A change to the cron, or the addition of a push trigger, would go undetected. Add an assertion for the cron value and for the absence of event triggers that could cause repeated writes.

♻️ Proposed additional test
`@test` "stale runs only on a daily schedule and manual dispatch" {
  triggers="$(code_lines < "$STALE" | awk '/^on:/ { inside = 1; next }
    inside && /^[^[:space:]]/ { exit }
    inside')"
  printf '%s\n' "$triggers" | grep -qE "^    - cron: '37 4 \* \* \*'"
  printf '%s\n' "$triggers" | grep -qE '^  workflow_dispatch:'
  ! printf '%s\n' "$triggers" | grep -qE '^  (push|pull_request|pull_request_target|issues):'
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 142 - 164, Extend the
contract tests around STALE workflow trigger validation, alongside the existing
pr-labeler trigger assertions, with a test that extracts the stale.yaml on block
and asserts cron remains “37 4 * * *” and workflow_dispatch is present. Also
assert push, pull_request, pull_request_target, and issues triggers are absent
so only the daily schedule and manual dispatch can run the workflow.

160-163: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

stale_input returns trailing inline comments as part of the value.

code_lines removes only whole-line # comments. stale_input prints the rest of the line after the key. If a maintainer documents a value inline, for example days-before-stale: 60 # two months, the value becomes 60 # two months and line 160 fails. The rest of stale.yaml uses inline comments already, such as the cron line, so this edit is likely.

♻️ Proposed change: strip trailing comments and whitespace
 stale_input() {
   code_lines < "$STALE" | awk -v key="          $1: " '
-    index($0, key) == 1 { sub(key, ""); print; exit }'
+    index($0, key) == 1 {
+      sub(key, "")
+      sub(/[[:space:]]+#.*$/, "")
+      sub(/[[:space:]]+$/, "")
+      print
+      exit
+    }'
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 160 - 163, Update
stale_input to remove trailing inline comments and surrounding whitespace from
the extracted value, not just whole-line comments handled by code_lines. Ensure
entries such as days-before-stale remain parsed as the bare value when followed
by an inline # comment, while preserving existing behavior for uncommented
values.

43-45: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Use a YAML-safe label parser.

The static $3 extraction works now, but it still rejects quoted labels and labels containing spaces. This affects labels already in .github/labels.yml such as quality-of-life, help wanted, and good first issue, so the contract test can miss future label changes. Update defined_labels() to parse the name: value instead of splitting by spaces.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 43 - 45, Update
defined_labels() to use a YAML-safe parser that extracts each name: value rather
than relying on awk’s whitespace-delimited $3 field. Ensure quoted labels and
labels containing spaces, including existing entries such as “help wanted” and
“good first issue,” are returned intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@hack/label-workflows-contract.bats`:
- Around line 87-89: Remove the redundant grep -v filter from the literal_labels
pipeline after extracting toAdd.add(...) values, leaving the existing
literal-label matching and sed transformation unchanged.
- Around line 62-69: Remove the non-POSIX local declarations from the helper
functions containing the all/missing logic and the matching declarations near
lines 22 and 28. Preserve the existing variable assignments and behavior while
keeping the POSIX-only claim accurate for hack/cozytest.sh.
- Around line 112-114: Update the negative assertion in the label-workflow
contract test to scan the entire on: section for edited rather than only types:
lines, so both inline and YAML block-sequence forms are detected. Preserve the
existing intent that edited must not appear as an executable trigger value.
- Around line 48-59: Update stale_input and stale_exempt_list to avoid relying
on fixed indentation when parsing STALE, while preserving their current
extraction behavior; alternatively, add explicit guards that fail with a clear
message when the expected key or folded list is not found.
- Around line 35-40: Update js_object_block so each input line has // comments
removed before evaluating the opening-pattern match and block boundaries. Ensure
commented-out declarations such as // const typeToKind = { never set inside,
while preserving extraction of active object blocks for subsequent script_lines
processing.
- Around line 142-164: Extend the contract tests around STALE workflow trigger
validation, alongside the existing pr-labeler trigger assertions, with a test
that extracts the stale.yaml on block and asserts cron remains “37 4 * * *” and
workflow_dispatch is present. Also assert push, pull_request,
pull_request_target, and issues triggers are absent so only the daily schedule
and manual dispatch can run the workflow.
- Around line 160-163: Update stale_input to remove trailing inline comments and
surrounding whitespace from the extracted value, not just whole-line comments
handled by code_lines. Ensure entries such as days-before-stale remain parsed as
the bare value when followed by an inline # comment, while preserving existing
behavior for uncommented values.
- Around line 43-45: Update defined_labels() to use a YAML-safe parser that
extracts each name: value rather than relying on awk’s whitespace-delimited $3
field. Ensure quoted labels and labels containing spaces, including existing
entries such as “help wanted” and “good first issue,” are returned intact.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a34c2aec-4ab9-46e5-b181-bd09a618b821

📥 Commits

Reviewing files that changed from the base of the PR and between d956536 and 035967a.

📒 Files selected for processing (1)
  • hack/label-workflows-contract.bats

@shaurya703

Copy link
Copy Markdown
Author

Addressed the review in the follow-up commit — all 8 points were valid or partially valid; 7 applied, 1 applied in part:

  • Comment-before-block-detection and YAML-safe label parsing: both were real holes in the file's own stated contract — fixed exactly as suggested, plus mutation checks for each (commented-out const typeToKind = { and a spaced label now go red).
  • edited block-sequence spelling: fixed by scanning the whole trigger section before jobs: rather than grep -qw over the full file — the word edited also appears in the (stripped) comment above the trigger today, but scoping to the trigger section keeps the guard robust if prose mentions it elsewhere later.
  • Inline-comment stripping in stale_input, dead grep -v filter, stale trigger pinning test: applied as suggested.
  • local vs POSIX comment: reworded the comment — the actual constraint is no process substitution under hack/cozytest.sh's sh runner; local is supported there and already used by the other contract files, so I kept it for consistency.
  • Indentation-sensitive stale helpers (flagged optional): kept the exact-indent contract for the folded exempt lists — a reformat there should fail loudly in a pinning test — but stale_input values now survive trailing inline comments, which was the likely real-world edit.

Both runners pass: bats and hack/cozytest.sh (7/7).

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/label-workflows-contract.bats`:
- Around line 184-193: The stale workflow trigger test should validate the
complete allowlist rather than reject only selected events. Update the test
around “stale runs only on a daily schedule and manual dispatch” to parse
top-level on keys, assert that only schedule and workflow_dispatch are present,
and require exactly one schedule entry with cron “37 4 * * *”.
- Around line 130-132: The edited assertion in the PR labeler workflow contract
test currently scans all content before jobs:, so restrict it to executable
lines belonging to the on: block only. Update the pipeline around PR_LABELER and
code_lines to identify the on: section and stop at its next top-level section,
while preserving detection for edited in both flow-sequence and block-sequence
trigger syntax.
- Around line 43-54: Update defined_labels() to strip matching single or double
quotes from parsed label names, while preserving existing handling of spaces and
inline comments. Ensure assert_labels_defined receives unquoted values for both
YAML quote styles.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b014703e-63bd-4c2f-89a0-5f0e61255c7d

📥 Commits

Reviewing files that changed from the base of the PR and between 035967a and 999fe48.

📒 Files selected for processing (1)
  • hack/label-workflows-contract.bats

Comment on lines +43 to +54
# Every label defined in .github/labels.yml, one per line. Parses the value
# after `- name: ` verbatim (labels may contain spaces, e.g. "help wanted"),
# stripping optional quotes and trailing inline comments.
defined_labels() {
code_lines < "$LABELS" | awk '/^- name: / {
sub(/^- name: /, "")
sub(/[[:space:]]+#.*$/, "")
sub(/[[:space:]]+$/, "")
gsub(/^"|"$/, "")
print
}'
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
git ls-files | grep -F 'hack/label-workflows-contract.bats' || true

echo "== relevant file excerpt =="
if [ -f hack/label-workflows-contract.bats ]; then
  nl -ba hack/label-workflows-contract.bats | sed -n '1,90p'
fi

echo "== find labels file and assert usages =="
rg -n "defined_labels|assert_labels_defined|labels.yml|assert_labels" hack .github 2>/dev/null || true

echo "== labels file sample =="
if [ -f .github/labels.yml ]; then
  nl -ba .github/labels.yml | sed -n '1,120p'
fi

echo "== behavioral probe: awk function =="
awk '
function code_lines() {
  return getline line < "."
}
defined_labels() {
  code_lines < "hack/label-workflows-contract.bats"
}
BEGIN {
  names[1] = "- name: help wanted"
  names[2] = "- name: \"help wanted\""
  names[3] = "- name: '\''help wanted'\''"
  for (i=1; i<=3; i++) {
    line = names[i]
    sub(/^- name: /, "", line)
    sub(/[[:space:]]+#.*$/, "", line)
    sub(/[[:space:]]+$/, "", line)
    gsub(/^"/, "", line)
    gsub(/"$/, "", line)
    print line
  }
}
'

Repository: cozystack/cozystack

Length of output: 278


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== relevant file excerpt =="
if [ -f hack/label-workflows-contract.bats ]; then
  awk '{printf "%6d\t%s\n", NR, $0}' hack/label-workflows-contract.bats | sed -n '1,110p'
fi

echo "== find labels file and assert usages =="
rg -n "defined_labels|assert_labels_defined|labels.yml|assert_labels" hack .github 2>/dev/null || true

echo "== labels file sample =="
if [ -f .github/labels.yml ]; then
  awk '{printf "%6d\t%s\n", NR, $0}' .github/labels.yml | sed -n '1,140p'
fi

echo "== behavioral probe: awk function =="
tmp="$(mktemp)"
printf '%s\n' \
  "- name: help wanted" \
  '- name: "help wanted"' \
  "- name: 'help wanted'" \
  "- name: 'help wanted'#" \
  "- name: \"help wanted\" #" \
  "-//-" > "$tmp"
awk '
function code_lines() {
  return getline line < "'"$tmp"'"
}
defined_labels() {
  code_lines < "hack/label-workflows-contract.bats"
}
BEGIN {
  while ((getline line < "'"$tmp"'") > 0) {
    if (line !~ /^- name: /) continue
    sub(/^- name: /, "", line)
    sub(/[[:space:]]+#.*$/, "", line)
    sub(/[[:space:]]+$/, "", line)
    gsub(/^"/, "", line)
    gsub(/"$/, "", line)
    printf "%s\n", line
  }
}
' /dev/null

rm -f "$tmp"

Repository: cozystack/cozystack

Length of output: 12821


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== labels.yml remaining excerpts =="
awk '{printf "%6d\t%s\n", NR, $0}' .github/labels.yml | sed -n '140,420p'

echo "== behavioral probe: stripped lines with current awk logic =="
tmp="$(mktemp)"
printf '%s\n' \
  "- name: help wanted" \
  '- name: "help wanted"' \
  "- name: 'help wanted'" \
  "- name: 'help wanted'#" \
  "- name: \"help wanted\" #" \
  "//" > "$tmp"
awk '/^- name: / {
  sub(/^- name: /, "")
  sub(/[[:space:]]+#.*$/, "")
  sub(/[[:space:]]+$/, "")
  gsub(/^"/, "")
  gsub(/"$/, "")
  print
}' "$tmp"

rm -f "$tmp"

Repository: cozystack/cozystack

Length of output: 9294


Strip both YAML quote styles.

defined_labels() only removes double quotes. A single-quoted label like - name: 'help wanted' keeps the quotes through defined_labels() and can make assert_labels_defined report a missing label. Strip matching single or double quotes, or use a YAML parser.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 43 - 54, Update
defined_labels() to strip matching single or double quotes from parsed label
names, while preserving existing handling of spaces and inline comments. Ensure
assert_labels_defined receives unquoted values for both YAML quote styles.

Comment on lines +130 to +132
# must not creep back in as an executable line — in either the flow-sequence
# or block-sequence spelling. Scan the whole trigger section (before jobs:).
! code_lines < "$PR_LABELER" | awk '/^jobs:/ { exit } { print }' | grep -qw 'edited'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== file list =="
git ls-files | rg '(^|/)hack/label-workflows-contract\.bats$|PR_LABELER|label-workflows' || true

echo "== target file outline/size =="
wc -l hack/label-workflows-contract.bats
sed -n '1,180p' hack/label-workflows-contract.bats

echo "== repository workflow examples around PR_LABELER symbol =="
rg -n "PR_LABELER|name: PR Auto-Label|edited|on:|pull_request|jobs:" . --glob '!vendor/**' --glob '!node_modules/**' | head -200

Repository: cozystack/cozystack

Length of output: 30396


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== pr-labeler workflow =="
wc -l .github/workflows/pr-labeler.yaml
sed -n '1,120p' .github/workflows/pr-labeler.yaml

echo "== exact grep behavior probe =="
tmp="$(mktemp)"
tee "$tmp" >/dev/null <<'EOF'
name: PR Auto-Label (edited)
on:
  pull_request:
    types: [open, reopened, synchronize]
jobs:
  label:
    runs-on: ubuntu-latest
EOF
export PR_LABELER="$tmp"

bash -lc '
REPO_ROOT="$(pwd)"
. hack/label-workflows-contract.bats
printf "name-edit scan result: %s\n" "$( code_lines < "$PR_LABELER" | awk "/^jobs:/ { exit } { print }" | grep -qw "edited" && echo FAIL || echo PASS )"

on_only_scan() {
  code_lines < "$PR_LABELER" | awk '
    /^on:/ { in_on=1 }
    /^[^[:space:]]/ && in_on { in_on=0 }
    in_on { print }
  ' | grep -qw "edited" && echo FAIL || echo PASS
}
printf "on-block scan result: %s\n" "$( on_only_scan )"
'

Repository: cozystack/cozystack

Length of output: 5204


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
text = ".github/workflows/pr-labeler.yaml"
path = Path(".github/workflows/pr-labeler.yaml")
print(f"has word edited in pr-labeler={ 'edited' in path.read_text() }")
print(f"top-level name contains edited={ any('edited' in line for line in path.read_text().splitlines()[:4]) }")
PY

tmp="$(mktemp)"
bash -c 'cat > "$1" <<EOF
name: PR Auto-Label (edited)
on:
  pull_request_target:
    types: [opened, reopened, synchronize]
jobs:
  label:
    runs-on: ubuntu-latest
EOF' sh "$tmp"

bash -c '
set -u
PR_LABELER="$1"
awk "
/^[^[:space:]]/ {
  if (line) print line
  line = (\$0)
  next
}
/^[[:space:]]/ {
  line = line \$0
  next
}
" "$PR_LABELER" < /dev/null
' sh "$tmp" | sed 's/.*edited.*/FOUND_EDITED/'

Repository: cozystack/cozystack

Length of output: 310


Scope the edited check to the on: block.

This scan covers every executable line before jobs:, not only trigger types. A top-level value such as name: PR Auto-Label (edited) can fail the test even when the pull-request trigger does not include edited. Limit the edited search to lines under the on: block.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 130 - 132, The edited
assertion in the PR labeler workflow contract test currently scans all content
before jobs:, so restrict it to executable lines belonging to the on: block
only. Update the pipeline around PR_LABELER and code_lines to identify the on:
section and stop at its next top-level section, while preserving detection for
edited in both flow-sequence and block-sequence trigger syntax.

Comment on lines +184 to +193
@test "stale runs only on a daily schedule and manual dispatch" {
triggers="$(code_lines < "$STALE" | awk '/^on:/ { inside = 1; next }
inside && /^[^[:space:]]/ { exit }
inside')"
[ -n "$triggers" ]
printf '%s\n' "$triggers" | grep -qF " - cron: '37 4 * * *'"
printf '%s\n' "$triggers" | grep -qE '^ workflow_dispatch:'
# No event trigger may run the write policy on user activity.
! printf '%s\n' "$triggers" | grep -qE '^ (push|pull_request|pull_request_target|issues|issue_comment):'
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "List candidate files:"
git ls-files | rg '(^|/)label-workflows-contract\.bats$|label-workflows|contract|stale' || true

echo
echo "Target excerpt:"
if [ -f hack/label-workflows-contract.bats ]; then
  nl -ba hack/label-workflows-contract.bats | sed -n '150,210p'
else
  echo "hack/label-workflows-contract.bats not found"
fi

echo
echo "Search stale workflow files:"
git ls-files | rg '(\.ya?ml|\.yml)$|stale' | sed -n '1,120p'

echo
echo "Search stale triggers in workflow files:"
if [ -f hack/label-workflows-contract.bats ]; then
  rg -n "stale|schedule|workflow_dispatch|pull_request|pull_request_target|issues|issue_comment|push:" .prettierrc.yaml .github hack -g '*.yml' -g '*.yaml' -g '*.bats' || true
fi

Repository: cozystack/cozystack

Length of output: 570


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Target excerpt:"
awk '{printf "%6d\t%s\n", NR, $0}' hack/label-workflows-contract.bats | sed -n '160,205p'

echo
echo "Stale workflow excerpt:"
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/stale.yaml | sed -n '1,140p'

echo
echo "Trigger-related searches:"
rg -n "stale|workflow_dispatch|push:|pull_request|pull_request_target|issues|issue_comment|schedule:|cron:" .github/workflows hack/label-workflows-contract.bats || true

echo
echo "Read-only parser probe across installed Python attrs:"
python3 - <<'PY'
from pathlib import Path
import re, sys
try:
    import yaml
    has_yaml = True
except Exception as e:
    print(f"PyYAML unavailable: {type(e).__name__}: {e}")
    has_yaml = False
stale = Path(".github/workflows/stale.yaml")
bats = Path("hack/label-workflows-contract.bats")
print(f"stale exists={stale.exists()} bats exists={bats.exists()} tools:")
print(f"grep={sys.executable}")

if has_yaml:
    data=yaml.safe_load(stale.read_text())
    on=data.get("on")
    print("yaml parsed triggers:", on)
    print("keys:", sorted(on.keys()) if isinstance(on, dict) else type(on))
    print("schedule entries:", on.get("schedule") if isinstance(on, dict) and isinstance(on.get("schedule"), list) else type(on.get("schedule")) if isinstance(on, dict) else None)
    print("workflow_dispatch present:", on.get("workflow_dispatch") not in (None, {}))

test_text=bats.read_text()
print("bats contains exactly keys check:", "on:" in test_text)
# model the current awk slice: starts after first line containing ^on:, stops at first subsequent line without leading whitespace.
slices=[]
inside=False
for i,line in enumerate(test_text.splitlines(),1):
    if re.search(r'^on:', line):
        inside=True
        continue
    if inside:
        if not re.search(r'^[^\s]', line):
            slices.append((i,line))
        else:
            break
print("awk-trigger-like text:", "\n".join(slices))
print("grep current allowed cron in awk text:", all(re.search(r"cron: '37 4 \* \* \*'", t) for _,t in slices))
print("grep workflow_dispatch in awk text:", any(re.search(r'^\s*workflow_dispatch:', t) for _,t in slices))
PY

Repository: cozystack/cozystack

Length of output: 24117


Assert the complete stale trigger allowlist.

Lines 184-193 require one known cron and workflow_dispatch, but the negative check only rejects five event names and can miss extra schedules. Parse the top-level on: keys, allow only schedule and workflow_dispatch, and assert exactly one expected cron entry.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/label-workflows-contract.bats` around lines 184 - 193, The stale
workflow trigger test should validate the complete allowlist rather than reject
only selected events. Update the test around “stale runs only on a daily
schedule and manual dispatch” to parse top-level on keys, assert that only
schedule and workflow_dispatch are present, and require exactly one schedule
entry with cron “37 4 * * *”.

pr-labeler.yaml and stale.yaml decide which labels land on issues and
PRs, and neither had a test behind it. A mistyped label name is not an
error, it is a rule that silently stops matching.

hack/label-workflows-contract.bats pins:

- every label either workflow can write or match resolves against a
  name: in .github/labels.yml (typeToKind, scopeToArea, the three
  literal adds, stale/close labels, both exempt lists)
- the trigger list stays [opened, reopened, synchronize] and 'edited'
  does not creep back in
- exactly the five documented type mappings; commented-out types do not
  count (JS // and YAML # comments are both stripped)
- the labeler stays additive-only (addLabels only, no removeLabel or
  setLabels) with the area/uncategorized fallback
- stale policy values: label choices, 12 issue and 4 PR exemptions,
  days-before-stale 60, days-before-close 14, operations-per-run 100

Every assertion was mutation-checked: break the pinned line, watch the
test go red, restore. POSIX-only constructs so hack/cozytest.sh can run
the file in plain sh; picked up automatically by BATS_UNIT_FILES.

Closes cozystack#3589

Signed-off-by: shaurya703 <[email protected]>
…gers

- js_object_block strips // comments before block detection, so a
  commented-out 'const typeToKind = {' can never open the block
- defined_labels parses the name: value verbatim (labels with spaces,
  quotes, inline comments) instead of awk $3
- stale_input strips trailing inline comments from values
- the 'edited' guard scans the whole trigger section, catching the
  block-sequence spelling too
- removed the dead grep -v filter on literal labels
- reworded the sh-compat comment: the real constraint is no process
  substitution; local is supported by the runner and used by the other
  contract files
- new test pins stale.yaml triggers: daily cron + workflow_dispatch
  only, no event triggers

All previous mutation checks still go red, plus three new ones for the
holes closed here (block-sequence 'edited', cron change, commented-out
typeToKind).

Signed-off-by: shaurya703 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci Issues or PRs related to CI workflows, GitHub Actions, automation size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

testing: pin the label-writing workflows with contract tests

1 participant