Skip to content

fix(hack): declare the shared package targets and the root test target phony - #3353

Merged
Aleksei Sviridkin (lexfrei) merged 3 commits into
mainfrom
fix/package-mk-phony-declaration
Aug 14, 2026
Merged

Aleksei Sviridkin (lexfrei) merged 3 commits into
mainfrom
fix/package-mk-phony-declaration

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Jul 19, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

hack/package.mk set .PHONY with = instead of ::

.PHONY=help show diff apply delete update image

That assigns a variable which happens to be named .PHONY; it declares nothing. make -p prints it back among the variables (.PHONY = help show diff apply delete update image) rather than as a special target, so none of the targets package.mk defines was phony in any package that includes it.

Nothing in the tree collides with those names today, so this is preventive rather than a fix for present breakage. The failure it forecloses is quiet: with a directory named image beside a package Makefile, make image prints make: 'image' is up to date. and exits 0 without building. A file of that name is enough on its own where the rule has no prerequisites; where it has one, the recipe keeps running until that prerequisite is shadowed too.

The list changed as well. suspend, resume, check and clean are defined in this file and were missing, and check is the one that matters most, since it is the prerequisite that keeps show, diff, apply and delete rebuilding. update and image are gone, because package.mk does not define them; the including package Makefile does. Naming an undefined target in .PHONY creates it as an empty target, turning No rule to make target (exit 2) into Nothing to be done (exit 0) wherever the rule is missing. That reaches CI too: hack/build-matrix.sh parses the root Makefile build: target and runs make -C <pkg> image per unit, so a package arriving in that list without an image: rule would go from a red job to a green one that builds nothing.

.DEFAULT_GOAL on the line above keeps its =. That one is a real variable, so the assignment form is right there. A comment now says which of the two lines is which, since they read alike and one of them just changed shape.

hack/package-mk-phony.bats covers the declaration and is picked up automatically by the hack/*.bats glob in bats-unit-tests. Five of its six tests cover package.mk: that the declared names are really phony with a file of each name sitting beside the Makefile; a control that mutates a copy back to the assignment form and requires the probe to report up-to-date, so the first cannot pass vacuously; that the declared list and the defined targets are the same set, checked both directions; and two pinning the update/image exclusion, one requiring the loud failure to survive and one extending the list in a copy and requiring it to disappear. The loud-failure test pins both the exit status 2 and the diagnostic. A wrapper that kept make's wording while returning something else would pass a check on the message alone.

One point decides where the guarantee actually lives. The first test reads its names from the .PHONY: line, so it cannot see a name dropped from that line at all. The set-equality test is what guards every name's presence.

One known gap is left in place. The first test decides by matching make's is up to date, which make prints only for a target that has a recipe, and a recipe-less alias never draws it: with no prerequisite both forms print Nothing to be done, and with a prerequisite that is itself out of date, which every target in this file is for being phony, the working form prints that prerequisite's recipe while the broken form prints Nothing to be done. The probe separates neither pair, so such a name would pass the first test either way. Measured on both shapes, not assumed. Every target package.mk defines has a recipe, so nothing is uncovered today; the control test does not close the gap either, since an alias is a defined target and the control requires is up to date from it, which turns it red against the probe rather than against the alias. Worth closing if an alias is ever added.

One more for whoever edits the suite next: hack/cozytest.sh rewrites every bare } at column 0 into return 0 followed by }. That is aimed at @test blocks, but it does not distinguish them from a plain shell function, so a helper factored out of these tests would silently become unable to return a non-zero status. Tests 1 and 2 repeat a loop rather than share one for reasons of their own, but this is the trap waiting for anyone who merges them.

Two properties of the suite are worth stating, because both are easy to get wrong in the other direction. The set-equality test extracts the defined targets with a pattern that accepts a rule naming several targets at once, so suspend resume: check keeps both names in the set; anchoring the colon to a single name drops every name on such a line, and for a target nothing else depends on that goes unnoticed. And the control test probes the targets package.mk defines rather than the names on its .PHONY: line, because is up to date is only a meaningful answer for a name that has a rule: a declared-but-undefined name prints Nothing to be done in both the working and the broken form, so probing it in the control fails against the probe, while the set-equality test holds the message that names the defect.

Last one, about how the tests find the repo. All six read it as repo=$(pwd), which assumes the runner starts at the repo root. That holds for make bats-unit-tests and for the invocation the header documents, so nothing is broken, and the assumption is the ordinary shape in hack/ rather than an outlier: about half the bats files there resolve repo-relative paths against the cwd the same way, several stating it in their own headers, and this is not even the only one spelling it with $(pwd). What is worth naming is the failure mode: started elsewhere, these tests go red with no '.PHONY:' declaration found in hack/package.mk, which reads like a real finding rather than a wrong directory.

The root Makefile had the same defect, and there it had already fired. test was not on its .PHONY: line and a test/ directory sits beside it, so make test printed 'test' is up to date and exited 0 without running anything, while docs/agents/overview.md advertises that command as the way to run the full suite. CI never used it: the workflow calls test-controllers and drives e2e out of packages/core/testing, so the cost fell on anyone following the docs by hand. One name on the declaration closes it, and the suite gained a sixth test that goes red without it. Every other undeclared target in that file is unshadowed today, so none of them is reachable the way test was, and they are left alone here. #3709 covers the same class in the four packages/core/ Makefiles, which include only hack/common-envs.mk and so are out of reach of this change entirely.

Downstream repositories

scripts/package.mk in cozystack/external-apps-example was a byte-for-byte copy of hack/package.mk (identical sha256 before this change), carrying the same inert .PHONY= line. This PR splits the two files, and the copy was fixed separately. It landed the same nine names, and update and image are absent there for the reason they are absent here: scripts/package.mk does not define them either, and neither does any Makefile that includes it. What separates the two files now is the two-line comment added here, and nothing else.

Release note

NONE

Summary by CodeRabbit

  • Bug Fixes
    • Fixed task declaration handling so supported commands are correctly recognized and executed.
    • Updated available commands by replacing update and image with suspend, resume, check, and clean.
  • Tests
    • Added coverage to verify command declarations, prevent regressions, and ensure invalid or missing commands report errors correctly.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a syntax error in the shared package Makefile. By correctly declaring the phony targets, it prevents potential build issues where the presence of local files or directories with the same names as the targets could cause the build process to skip execution.

Highlights

  • Corrected .PHONY declaration: Changed the assignment of .PHONY in hack/package.mk from a variable assignment to a proper special target declaration to ensure targets are correctly identified as phony.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@github-actions github-actions Bot added area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review kind/bug Categorizes issue or PR as related to a bug labels Jul 19, 2026
@dosubot dosubot Bot added the kind/cleanup Categorizes issue or PR as related to cleanup of code, process, or technical debt label Jul 19, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes the syntax of the .PHONY target declaration in hack/package.mk. The reviewer suggests also adding other targets defined in the file (suspend, resume, check, clean) to the .PHONY list to prevent potential conflicts with local files or directories of the same name.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread hack/package.mk Outdated
@@ -1,5 +1,5 @@
.DEFAULT_GOAL=help
.PHONY=help show diff apply delete update image
.PHONY: help show diff apply delete update image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While fixing the .PHONY declaration, consider also adding the other targets defined in this file (suspend, resume, check, clean) to the .PHONY list to prevent potential conflicts with local files or directories of the same name.

.PHONY: help show diff apply delete update image suspend resume check clean

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding suspend resume check clean is right and it's in the final list. I dropped update and image, though — the file defines neither (only the %-update pattern rule), so declaring them phony makes make image/make update report success without running anything in packages that lack those recipes. Final line: .PHONY: help show diff apply delete suspend resume check clean.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

hack/package.mk now uses standard .PHONY: syntax, updates the declared phony targets, and adds Bats coverage for declaration and behavior.

Changes

Package Makefile

Layer / File(s) Summary
Update phony target declaration
hack/package.mk
The declaration changes from .PHONY=... to .PHONY: ...; update and image are replaced by suspend, resume, check, and clean.
Validate phony target behavior
hack/package-mk-phony.bats
Tests verify phony behavior, exact target coverage, rejection of assignment syntax, and deliberate exclusion of update and image.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: area/testing

Suggested reviewers: myasnikovdaniil

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: declaring the shared package targets as phony.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/package-mk-phony-declaration

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@myasnikovdaniil myasnikovdaniil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOT LGTM — the = → : diagnosis is exactly right, but the target list it declares is wrong at both ends: it adds two targets package.mk does not define, and omits the two that were actually vulnerable. One line fixes both.

Business context: hack/package.mk:2 reads .PHONY=help show diff apply delete update image — a variable assignment rather than a declaration — so no target in any package is phony today.

To set expectations on severity up front: this is developer tooling, not a production path. Nothing here breaks a cluster or a release. But since the line is being corrected anyway, it is worth landing the correct list rather than trading one slightly-wrong list for a differently-slightly-wrong one — which is what the current diff does.

Problem 1: update and image are not defined by package.mk, so declaring them phony makes them silently succeed

hack/package.mk defines help show apply diff suspend resume delete check clean %-update. It does not define update or image. Naming a target in .PHONY: creates it as an explicit empty target, so make reports success for work that never ran.

Evidence — measured on packages/apps/qdrant, which defines neither:

main:        make -n image  -> "No rule to make target 'image'.  Stop."   exit 2
             make -n update -> "No rule to make target 'update'. Stop."   exit 2

this PR:     make -n image  -> "Nothing to be done for 'image'."          exit 0
             make -n update -> "Nothing to be done for 'update'."         exit 0
             make image (real invocation, not -n)                          exit 0

Scope, counted across the 155 Makefiles that include hack/package.mk:

count
lack an image target → make image flips 2 → 0 125
lack an update target → make update flips 2 → 0 76
lack both 60

Mostly the *-rd resource-definition packages plus pure-chart apps (apps/bucket, apps/kafka, apps/qdrant, apps/tenant, extra/etcd, extra/seaweedfs, system/keycloak, library/cozy-lib).

Worth noting this is the same failure shape the PR description cites as its own motivation — a target reporting up-to-date without an error — just arrived at from the other direction.

On CI: no live exposure today, and I checked rather than assumed. The only variable-package make image is .github/workflows/pull-requests.yaml:188, whose matrix comes from hack/build-matrix.sh parsing the root Makefile's build: target; all 27 matrix units plus core/talos and core/installer define image (make -n image exit 0 for all 29). make update-all at tags.yaml:615 runs in the website checkout. hack/helm-unit-tests.sh:22 gates on test, which is not in this list.

One thing that is worth flagging explicitly, though: this PR's green CI is not evidence of safety. hack/package.mk is in build-matrix.sh:25's full_rebuild_pattern, so this PR ran the full matrix and every Build job passed — but that only exercised the 29 packages that do define image. The latent trap is that if a package is later added to root build: without an image target, CI currently fails loudly; after this change the Build job goes green, uploads an empty patch fragment, and finalize bundles an unpatched digest.

Problem 2: the list omits the only targets that were actually vulnerable

I probed each target by planting a decoy file of the same name in a package directory and checking whether make no-op'd:

help   -> VULNERABLE (no-op'd)      show    -> immune (recipe ran)
check  -> VULNERABLE (no-op'd)      diff    -> immune (recipe ran)
clean  -> VULNERABLE (no-op'd)      apply   -> immune (recipe ran)
                                    delete  -> immune (recipe ran)
                                    suspend -> immune (recipe ran)
                                    resume  -> immune (recipe ran)

show, diff, apply, delete, suspend and resume are already immune — they all depend on check (hack/package.mk:7,10,13,16,19,22), which never exists as a file and therefore always reruns, forcing its dependents.

So of the eight targets in the current list: six are no-ops, one (help) is real protection, two (update, image) cause problem 1 — while check and clean, both genuinely vulnerable, are absent.

For completeness on how live the original bug is: I checked all 155 package directories for a file or directory named help|show|diff|apply|delete|update|image and found zero collisions. The convention is images/ — plural, a directory. So the bug being fixed is real in principle but not currently reachable, which is another reason to prefer getting the list right over landing it quickly.

Suggested fix

.PHONY: help show diff apply delete suspend resume check clean

Verified: image and update keep exiting 2; help, bare make, and test are unaffected; packages that do define image still build (mariadb, dashboard dry-run exit 0); and with a decoy show file the recipe still runs, so phony-ness is genuinely in effect.

Alternatives I considered and rejected:

  • Leave update/image to per-package declaration. Regression-free, but leaves check/clean unprotected, and only 3 of 30 image-defining and 5 of 79 update-defining packages currently declare their own — so 27 and 74 would stay undeclared.
  • A conditional such as $(if $(filter image,$(MAKECMDGOALS)),…). Makes phony-ness depend on goal ordering; too clever for a hygiene fix.
  • A shared fallback like image update: ; @exit 1. Not viable — packages defining their own image would then emit exactly the "overriding recipe" warnings that #3352 is removing.

On the existing suggestion in the thread

The inline suggestion of .PHONY: help show diff apply delete update image suspend resume check clean is half right: adding suspend resume check clean is correct, but keeping update image preserves problem 1. Worth not applying as written.

Unrelated nit

Line 1's .DEFAULT_GOAL=help is correctly = — it is a genuine variable. A one-line comment saying so would stop someone "fixing" it to : later by analogy with this change.

Comment thread hack/package.mk Outdated
@@ -1,5 +1,5 @@
.DEFAULT_GOAL=help
.PHONY=help show diff apply delete update image
.PHONY: help show diff apply delete update image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both problems land on this one line, so here is the combined fix as a suggestion.

Drops update and image (not defined by package.mk, so declaring them phony makes make image exit 0 in 125 packages and make update in 76), and adds suspend resume check clean — of which check and clean were the genuinely vulnerable ones, while show/diff/apply/delete/suspend/resume were already immune via their check prerequisite.

Suggested change
.PHONY: help show diff apply delete update image
.PHONY: help show diff apply delete suspend resume check clean

Verified after this change: image/update keep exiting 2; help, bare make and test unaffected; packages that do define image still build; and a decoy show file no longer suppresses the recipe.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied as suggested — this is the line I pushed. Verified image/update exit 2 again, help/bare make/test unaffected, and a same-named decoy file no longer suppresses the recipe.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The branch has carried your line since 21 July: .PHONY: help show diff apply delete suspend resume check clean, and the comment above .DEFAULT_GOAL now says which of the two lines is a variable, so nobody converts it by analogy later. Re-measured on the current head: 159 package Makefiles include the file, none of their directories holds a path named after any of the nine targets, and make update / make image exit 2 again wherever the rule is missing. hack/package-mk-phony.bats pins that last one on both the exit status and make's wording, with a control that adds the two names back to a copy and requires the failure to disappear, so the exclusion cannot be undone quietly.

@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch 2 times, most recently from 6774574 to fc1b1ea Compare July 21, 2026 16:00
@lexfrei

Copy link
Copy Markdown
Contributor Author

myasnikovdaniil Agreed on both counts — pushed the corrected list:

.PHONY: help show diff apply delete suspend resume check clean

That's exactly the concrete targets the file defines. update and image are gone (neither is defined here — only the %-update pattern rule), and check/clean are added. After the change: make image and make update are back to exiting 2 in packages without those recipes, help/show/bare make are unaffected, and a package that defines its own image still builds.

Left line 1's .DEFAULT_GOAL=help as-is since = is correct there — happy to add the clarifying comment in this PR if you'd like it.

@lexfrei

Copy link
Copy Markdown
Contributor Author

myasnikovdaniil The fixed list landed on the branch in fc1b1ea, nothing has changed since. I compared the two sets mechanically rather than by eye: the file defines nine targets, .PHONY lists the same nine, and there is no include that could add more.

%-update can't be added at all. .PHONY matches names literally, so .PHONY: %-update declares a file called that and leaves the pattern rule non-phony. Tried it with a decoy file, make still said up to date.

@lexfrei Aleksei Sviridkin (lexfrei) added area/build Issues or PRs related to image build infrastructure, multi-arch support and removed area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review labels Aug 7, 2026
@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch from fc1b1ea to 916731d Compare August 7, 2026 12:32
@github-actions github-actions Bot removed the size/XS This PR changes 0-9 lines, ignoring generated files label Aug 7, 2026
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@github-actions github-actions Bot added size/L This PR changes 100-499 lines, ignoring generated files area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review labels Aug 7, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/package-mk-phony.bats`:
- Around line 177-187: Update the `for t in update image` test loop to assert
that the captured `rc` from `make` equals 2 before evaluating the diagnostic
output. Keep the existing failure handling and output checks for undefined
targets unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: eba58d42-2494-470c-97ce-e165d9872fb6

📥 Commits

Reviewing files that changed from the base of the PR and between c6eb584 and 916731d.

📒 Files selected for processing (2)
  • hack/package-mk-phony.bats
  • hack/package.mk
🚧 Files skipped from review as they are similar to previous changes (1)
  • hack/package.mk

Comment thread hack/package-mk-phony.bats
@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch from 916731d to c31474a Compare August 9, 2026 18:14
@lexfrei

Copy link
Copy Markdown
Contributor Author

myasnikovdaniil The list is unchanged since July, still .PHONY: help show diff apply delete suspend resume check clean.

Added the .DEFAULT_GOAL comment you asked for in e97416a. The undeclared-target test pins exit 2 now instead of any nonzero status, that one is c31474a.

Re-ran your collision scan on the current tree: 159 package Makefiles include it now, still zero files or directories named after any of the nine targets.

@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch 2 times, most recently from b728a18 to 61fa93b Compare August 9, 2026 19:17
@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch 4 times, most recently from 737b822 to 02d52e9 Compare August 9, 2026 21:37
@lexfrei Aleksei Sviridkin (lexfrei) changed the title fix(hack): declare the shared package targets phony fix(hack): declare the shared package targets and the root test target phony Aug 9, 2026
@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch from 02d52e9 to 3c2b71e Compare August 11, 2026 15:32
package.mk set .PHONY with `=` rather than `:`, which assigns a
variable named .PHONY instead of declaring anything phony, so none
of the targets package.mk defines was phony in any package that
includes it.

Declare exactly the concrete targets package.mk defines: help, show,
diff, apply, delete, suspend, resume, check and clean. The earlier
list named update and image, which package.mk does not define —
naming an undefined target in .PHONY creates it as an empty target,
so `make image` and `make update` would report success without
running anything in the many packages that lack those recipes. It
also omitted check and clean, the targets a same-named file or
directory could actually shadow.

The neighbouring .DEFAULT_GOAL is a variable and correctly keeps its
equals sign. A comment now records which of the two adjacent lines is
which, so the corrected one does not pull the other into its shape by
analogy.

The trigger map in docs/agents/contributing.md called the two files a
byte-for-byte copy, which was exact rather than loose: they were the
same object. They stop being one here, so the entry now records that
they were identical and no longer are, and asks for a diff-and-port,
the wording the map already uses for the other copy that has
diverged. It deliberately does not enumerate which lines differ:
that set moves as the copy catches up, and an entry that names it
would be wrong again by the next port.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
The root Makefile declares nine of its targets phony and `test` is not
among them, while a `test/` directory sits beside it. Make therefore
takes that directory for the target's product: `make test` prints
"'test' is up to date", exits 0, and the recipe -- which installs the
platform and runs the e2e suite -- never executes.

docs/agents/overview.md advertises the command as the way to run the
full suite, so someone following it by hand gets a green exit and no
tests. CI is unaffected: the workflow calls test-controllers, and the
e2e path is driven out of packages/core/testing directly.

Every other undeclared target in this file is unshadowed today, so this
is the only one that misbehaves, and one name closes it.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch from 3c2b71e to 99438d6 Compare August 12, 2026 15:43
Aleksei Sviridkin (lexfrei) added a commit that referenced this pull request Aug 12, 2026
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does

`helm unittest` is fail-closed on a suite it cannot parse and on one
declaring `tests: []`, and fail-open on suites that are not there at
all: it prints `Test Suites: 0 passed, 0 total` and exits 0.
`hack/helm-unit-tests.sh` judges each package by that exit code alone,
so a chart whose suites were deleted, moved, or renamed past the
`tests/*_test.yaml` glob reports success having asserted nothing.

That is measurable in both directions rather than argued. Move
`packages/system/velero/tests/velero_test.yaml` aside and run the script
as it exists on main today:

```text
Running tests in packages/system/velero
helm unittest .

### Chart [ cozy-velero ] .


Charts:      1 passed, 1 total
Test Suites: 0 passed, 0 total
Tests:       0 passed, 0 total
```

The run ends with `All Helm unit tests passed.` and exit 0. The chart
passed, and the count of things it checked was zero. With this PR the
same tree exits 1 and names the package and the reason. Neither
`--strict` nor an explicit non-matching `-f` glob changes this on its
own; all three forms exit 0, checked against plugin 1.1.1.

So the script now reads the captured output and fails the package when
the zero-suite line appears.

Matching that line names a cause and a remedy where it applies, but the
set of ways to run nothing is not closed. A recipe that never invokes
`helm unittest`, and a `test:` rule that make finds nothing to do for,
print no marker of any kind and exit 0, so no additional match reaches
them; enumerating absences leaves a fresh hole every time a new shape
turns up. So the script also requires the line a real run always emits,
a `Test Suites:` count of at least one. That turns the question round:
every way of asserting nothing exits 0, and "what did this run assert"
is the question that has an answer. All 76 packages the script visits
emit that line today. The tradeoff is that a `test` target which
legitimately runs no `helm unittest` would need renaming or an opt-out,
which is the same bargain the zero-suite check already strikes.

The discovery gate has the same shape from the other side. `make -C dir
-n test` succeeds against a file-backed target, so a path named `test`
beside a package Makefile whose rule is not phony would make both the
gate and the run exit 0 without the recipe ever firing. The check keys
on what make reports rather than on the path existing, because a package
that declares the target phony runs its recipe whatever sits next to it,
and refusing that would be a false failure. The quoting around the
target name differs between make 3.x and 4.x, so it accepts either
opening quote; anchoring on the trailing quote alone would also match a
sub-make reporting some other target whose name ends in `test`.

That second check is guarding a live gap rather than a hypothetical one.
`hack/package.mk` line 2 reads `.PHONY=help show diff apply delete
update image`, which is a variable assignment and not a target
declaration, so it makes nothing phony. Of the packages the script
visits that define a `test` rule, 18 declare it phony in their own
Makefile; the rest are unprotected if a path of that name ever appears.
No such path exists today. Fixing the shared include is #3353 and is not
this PR.

One deliberate non-change, stated so it is not rediscovered as an
oversight. Running the suite now captures output instead of streaming
it, because a pipeline's exit status in POSIX `sh` reports the last
command rather than `make`. Streaming reads better and loses the status,
so it stays buffered.

The run is pinned to `LC_ALL=C`, since both checks match English wording
and a localized `make` would disarm one of them silently, which is the
failure mode the whole change exists to remove.

The guard is all-or-nothing, which is worth stating plainly. A chart
that loses four of its five suites still reports `Test Suites: 1 passed`
and passes; only total disappearance is caught. Tying the expected count
to what each chart actually has would be a per-chart number to maintain,
and a number maintained in one place while the suites move in another is
the failure this repo has been paying for elsewhere, so the cheap check
that catches the total loss is the one worth having.

One behaviour worth stating because the message does not: a suite in
which every test carries `skip:` reports `Test Suites: 0 passed, 1
skipped, 1 total` and exits 0, and the positive-evidence check refuses
it. That refusal is intended, since a wholly skipped suite asserted
nothing, but the message talks about suites expected under `tests/`,
which in that case are present and skipped on purpose. No package is in
that state today. Partial skips are unaffected: `2 passed, 1 skipped, 3
total` satisfies the check.

Three known gaps in what the change ships, none of them a wrong
statement. The positive-evidence message explains its cause but stops
short of naming a remedy, where the other two messages both end in an
action; the remedy for the legitimate case, renaming the target or
giving it an opt-out, currently lives only in the code comment. That
same message enumerates two ways a run reports nothing, and the
skipped-suite case above is a third, so it reads as a diagnosis where it
is really a list of the common causes. And no document in the tree
states the convention this tightens: `docs/agents/overview.md` still
describes the script accurately as running over every package that
defines a `test` target, but the contract is now that such a package
must also report at least one passing suite. All three are improvements
to make on the next touch of these files rather than reasons to hold the
change.

One adjacent gap stays open, and it is worth naming rather than leaving
someone to assume otherwise. This catches a chart whose suite *files*
vanished; it does not catch a package that loses its `test:` rule along
with them. Nothing ties the presence of `tests/*_test.yaml` to the
presence of a `test` target, so a change that removes both is skipped in
silence, and the script only complains when no package in the tree has a
`test` target at all. The test named `a package with no test target is
skipped, not failed` pins that permissiveness deliberately, because the
many packages that legitimately define no `test` rule would otherwise
turn every run red. Closing it needs an instrument keyed on the suite
files rather than on the Makefile target, which is a separate change.

`hack/helm-unit-tests.bats` adds eight tests, picked up automatically by
the `hack/*.bats` glob in `make unit-tests`. Three pin the new refusals,
and each matches its own message so none can stay green on a refusal
that came from another. Two pin what must not be refused: a phony target
with a colliding path, and a sub-make reporting an unrelated target up
to date. One pins that a chart which does run suites still passes, which
is what stops the refusals being satisfied by rejecting everything. Two
pin behaviour that was already there, namely that a failing package is
still reported by name and that a package with no `test` rule is skipped
rather than failed. That last one builds a tree carrying a second
package that does run a suite: with only the target-less package
present, the script takes an early exit above the failure summary and
the assertion would hold whatever the script had done. The fixtures stub
`make` output instead of invoking helm, so they need no plugin
installed.

Relates to #3453.

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(tests): `make helm-unit-tests` now fails a chart that runs no Helm unit test suites, instead of reporting success for a chart whose suite files were deleted, moved, or renamed out of the `tests/*_test.yaml` glob
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved Helm test validation to detect skipped test recipes, zero
discovered suites, and runs without evidence of a passing suite.
* Helm test failures now provide clearer diagnostics, including affected
directories and failed commands.
* Test output is consistently captured and replayed for easier
troubleshooting.

* **Tests**
* Added comprehensive coverage for successful, failed, skipped, empty,
and invalid Helm test scenarios.
* Expanded coverage across package test targets, including phony and
unrelated sub-make behavior.

* **Documentation**
  * Clarified requirements for package test targets and suite reporting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
`.PHONY=a b c` and `.PHONY: a b c` differ in the `=` against the
`:`, and only the second declares anything. The first defines an
ordinary variable named `.PHONY` that nothing reads, leaving every
listed target file-backed. A file or directory of that name beside a
package Makefile then stands in as the product of the rule: for a
target with no prerequisites make reports it up to date, skips the
recipe and exits 0, and a target whose prerequisite is itself
file-less keeps rebuilding until that prerequisite is shadowed too.
The neighbouring `.DEFAULT_GOAL=help` is a real variable and is
correct with `=`, so the two lines read alike while only one of them
works, which is what let the inert form sit there unnoticed.

Add hack/package-mk-phony.bats, six tests, picked up automatically
by the hack/*.bats glob in `bats-unit-tests`.

Three cover the declaration itself. The first reads the names off
the `.PHONY:` line rather than repeating them, plants a file named
after each beside a stub Makefile, and requires `make --dry-run` to
print the recipe instead of "is up to date". A negative control
mutates a copy back to the assignment form and requires the probe to
report up-to-date, over the targets package.mk defines rather than
the names it declares, because "is up to date" is only a meaningful
answer for a name that has a rule. The third keeps those two sets
equal: it requires the declared list and the defined targets to be
the same set. Defined but undeclared
is the direction the list rots in: check, clean, suspend and resume
sat undeclared for as long as the declaration itself was inert.
Declared but undefined is the empty-target trap in its general
form, so a future name cannot slip in the way update and image
once did.

Two more pin what the list deliberately leaves out. package.mk
defines neither `update` nor `image`; the including package Makefile
does, and most packages define neither. Naming them here would
create them as empty targets everywhere, turning "No rule to make
target" into "Nothing to be done". Since the CI build matrix is
parsed from the root Makefile `build:` target and runs
`make -C <pkg> image` per unit, a package reaching that list without
an `image:` rule would go from a red job to a green one that builds
no image. One test requires that loud failure to survive, pinning
both of the observables it is made of: make's exit status 2 and its
diagnostic. They are independent, and a wrapper that kept the
wording while returning something else would satisfy a check on the
message alone. Its control extends the declaration with the two
names in a copy and requires the failure to disappear, so the pin
tracks the list rather than some property of the harness.

Every decoy carries one fixed timestamp. Six of the nine targets take
check as a prerequisite, and apply and delete reach it through suspend
as well, so touching the decoys in list order can leave check newer
than its own dependents. Make then rebuilds them for that reason and
prints a recipe whatever .PHONY says, and the probe reports success
against a declaration that does nothing. Equal mtimes remove the
question in any list order. Measured on GNU Make 4.3: list order left
the per-target assertion inert for up to six of the nine, moving with
where the clock tick fell, while 3.81 fits the loop inside a single
timestamp and shows none of it. The control test plants its decoys
the same way for the same reason, and goes red if that is undone in
its own loop. The two loops are independent, so neither guards the
other, and the header is what keeps the arrangement from being
unpicked.

A sixth test leaves package.mk and covers the root Makefile, where this
class had already bitten: `test` is the one target in the tree with a
path of its own name beside it. It probes with --dry-run because the
recipe wants a live cluster, and fails loudly if that path ever goes
away rather than passing on a question with no subject.

`%-update` is not coverable: .PHONY matches target names literally
and does not expand patterns.

Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the fix/package-mk-phony-declaration branch from 99438d6 to d978a37 Compare August 13, 2026 23:01

@myasnikovdaniil myasnikovdaniil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three findings from my 2026-07-20 review are closed, and the suggested .PHONY: line landed verbatim at hack/package.mk:4.

I checked the two load-bearing claims on a scratch makefile rather than reasoning about them, because make's behaviour here is easy to describe wrong from memory. With a decoy file named hello beside it, .PHONY=hello prints 'hello' is up to date and the recipe never runs, .PHONY: hello runs it. And a name declared phony with no rule anywhere gives Nothing to be done for 'image' with exit 0, which is the silent success you were avoiding. image: is undefined in 130 of 159 package makefiles and update: in 79, so the exclusion is load-bearing and hack/package-mk-phony.bats:198-233 pins it from both directions with an explicit rc check rather than "nonzero".

Red phase strikes. Reverting to .PHONY=, dropping clean, dropping check, re-adding update image, and dropping test from the root .PHONY each turn exactly one test red with the right message.

Four things worth a look, none blocking.

The body says this file is the only one using the repo=$(pwd) spelling and that 14 of 58 files in hack/ derive the root from BATS_TEST_FILENAME. At head it is 24 of 61, and hack/overlay-main-images_test.bats:16 already uses root=$(pwd). More useful, the tree's idiom is ${BATS_TEST_FILENAME:-$0} and hack/bats-no-exit-trap.bats:89-94 documents why the fallback exists, so the set -u hazard that would justify avoiding it is already solved. Run the suite from anywhere but the repo root today and test 1 fails with no '.PHONY:' declaration found in hack/package.mk, which reads as a regression in package.mk rather than a wrong cwd.

The update/image exclusion is explained where nobody will read it. hack/package.mk:1-2 covers = versus : and says nothing about why line 4 stops where it does, so someone re-adding image hits test 2's probe failed to detect a file-backed update before test 3's accurate message. A clause at the site being edited would fix the ordering.

The extractor blind spot at hack/package-mk-phony.bats:159 is narrower than its comment implies. A name outside the character class is only shadowable if it also has no check prerequisite, since check carries the always-rebuild. _render: check plus a recipe leaves all 6 green and is not actually a hole. _render: with no prerequisites is.

And MAKEFLAGS=-B turns test 2 red on healthy code, because --always-make means nothing can report "is up to date". make -B unit-tests is not a normal invocation so I would not hold anything for it, MAKEFLAGS= on the probes would close it.

One process note: the file sorts at 69 of 116 in bats discovery and hack/ghcr-mirror_test.bats at 45 fails locally, so a full local make unit-tests never reaches this file. I ran it standalone. CI is the authority there and it is green.

@lexfrei
Aleksei Sviridkin (lexfrei) merged commit 340483f into main Aug 14, 2026
43 of 45 checks passed
@lexfrei
Aleksei Sviridkin (lexfrei) deleted the fix/package-mk-phony-declaration branch August 14, 2026 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build Issues or PRs related to image build infrastructure, multi-arch support area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review kind/bug Categorizes issue or PR as related to a bug kind/cleanup Categorizes issue or PR as related to cleanup of code, process, or technical debt size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants