fix: publish a device's $state after its children in refresh_tree - #32

Merging this, with four small fixups pushed to your branch rather than sending you back around for them. The diagnosis is right and the fix is the minimal correct one.

Crediting the part that made this reviewable in one pass: "74 publishes, 74 unique topics, identical topic sets" establishes up front that the message set is not in play and only sequence changed. I reproduced it independently, including with property values set, and it holds. The ordering direction also matches the convention's own add-children recipe, where a child is fully published before its parent announces. The pre-order walk inverted that on every reconnect, which is what made this the normal path rather than an edge case.

The strongest argument for your ordering is one your writeup does not make. After an ungraceful drop the LWT leaves the root retained as lost, and Homie 5 makes every child of a lost root lost too. Holding the root's $state until the recursion finishes therefore turns the whole refresh into one atomic commit, flipped by a single final publish. That is a commit barrier derived from the spec rather than asserted, and it is now the reason given in the code comment.

The part I want to push back on

You wrote in #31 that you defend against this consumer-side by "waiting for every declared descendant to describe itself before reporting ready," and characterised that as reasonable for a consumer but something the producer should not have made necessary. I would put it the other way around: that defence is mandatory and permanent, and as described it is still not sufficient, because it is a barrier rather than a loop. It runs once, passes, and stops reconciling. Commission a circuit a minute later and it is missed. The failure moves from startup to steady state, which makes it harder to find, not less real.

Three reasons no amount of producer-side ordering can retire that defence:

• Children are commissioned and decommissioned out of band, so there is no moment at which a producer knows it has published "all" its children. The strongest guarantee any producer can offer is per-transaction: within this cascade, my announcement follows the content it announces. That is a statement about a transaction, not about the world.
• Publish order does not survive retention. A consumer connecting afterwards receives the retained tree in broker-chosen order, and retained-set delivery order on SUBSCRIBE is unspecified. Your own broker-restart scenario is a late-subscriber case, so this change does not actually reach it.
• A declared child can be legitimately absent, having crashed or had its own LWT fire. "Declared" can never be made to mean "present."

There is also a case already in the SDK that breaks the same shape of parser and has nothing to do with child ordering: the content-hash suppression makes an unchanged $description a no-op, but it does not suppress the init to ready edge of an otherwise-empty transition. A consumer that waits for a $description after a ready edge can wait forever.

So $state=ready means "my own $description is current, you may act on it." It is not a rollup over the subtree, and it is not a barrier after which the tree stops changing. Our own Controller was never exposed to the bug you found precisely because it uses ready as a trigger to subscribe rather than a barrier to read, then lets each child's retained state cascade back through the same handler. Reconcile from current state on every update; never await message B because message A arrived.

That distinction had never been written down anywhere in this repo, which is the real root cause here: you read our observable behaviour, inferred a contract from it, and built against the inference. That is on us. I have written doc/consuming-a-homie-tree.md covering the producer-SHOULD / consumer-MUST asymmetry, what ready does and does not promise, and the three failure modes above. It lands right behind this.

The fixups

• ruff format collapse on child_states, the only CI blocker. Note our pin is 0.15.21, so ignore anything a local 0.16.x says about the Markdown files.
• Deleted Device._publish_self(). Your change left it with zero callers and a docstring still claiming it serves the reconnect cascade.
• One extra assertion in the pre-existing test_refresh_tree_three_levels. This one is worth explaining: your new test builds a flat two-level tree, and I confirmed that a fix which special-cased only the root, leaving intermediate devices on the old order, passes the entire 546-test suite as written. Two lines on the existing three-level fixture pin the recursive property, and they do fail on that mutant.
• Reworded the code comment and CHANGELOG entry. Your version said the new order makes ready "mean what it says at every level," and I did not want to ship that sentence, for the reasons above. Same change, narrower claim, plus your credit on the CHANGELOG line.

One thing I considered and deliberately did not do: publishing a root $state=init at the head of the cascade, to close the transient where children have a state and the root does not yet. It would propagate INIT down the effective-state table, so every controller would see the entire tree go non-ready on each broker reconnect, and Home Assistant availability would flap on the root's entities. The transient you introduce is a not-yet-arrived message, which every consumer must already tolerate; the one you removed was an active false claim. That is the right trade.

Ships as 0.18.1.