Skip to content

test: live-write coverage for feeds/retargeting/strategies + cassette refresh#536

Merged
axisrow merged 2 commits into
mainfrom
test/live-write-feeds-retargeting-strategies
Jun 3, 2026
Merged

test: live-write coverage for feeds/retargeting/strategies + cassette refresh#536
axisrow merged 2 commits into
mainfrom
test/live-write-feeds-retargeting-strategies

Conversation

@axisrow

@axisrow axisrow commented Jun 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add Phase 6 live-write lifecycle tests (tests/test_v5_live_write.py) for feeds, retargeting and strategies — mutating resources the sandbox cannot exercise (returns 8800/6000), so they were previously only covered via dry-run.
  • No real account ids anywhere. retargeting uses the synthetic goal placeholder ALL:12345:30 (same convention as the existing audiencetargets smoke tests); the live API returns 8800 and the test skips, recording the request shape rather than a passing lifecycle.
  • Refresh four cassettes that drifted out of sync with the CLI after the Аудит дублирования кода: системные дубли (~6000+ строк), незавершённая централизация хелперов #491 refactor (WSDL-parity gate made fields required): ads update now needs --type, feeds add needs --business-type, campaigns add --type SMART_CAMPAIGN needs --counter-id (synthetic 12345678). smartadtargets skips on 3500 as documented.
  • Document in tests/MANUAL_COVERAGE.md that the Yandex Direct API has no Strategies.delete operation (only add/update/get/archive/unarchive) — a created strategy can only be archived (StatusArchived=YES), not removed via the API.

Test plan

  • Live-write replay (no network): YANDEX_DIRECT_TOKEN=dummy YANDEX_DIRECT_LOGIN=dummy YANDEX_DIRECT_LIVE_WRITE=1 pytest -m integration_live_write → 12 passed, 9 skipped, 0 failed
  • Full suite: pytest -q2156 passed, 50 skipped, 0 failed
  • Secret/real-id scan across cassettes, test file and docs → clean (0 tokens, 0 logins, 0 real account ids)
  • flake8 → no new violations

🤖 Generated with Claude Code

…rifted cassettes

Add Phase 6 live-write lifecycle tests in test_v5_live_write.py covering
feeds (add/update/delete), retargeting (add/update/delete) and strategies
(add/update/archive/unarchive) — resources the sandbox cannot exercise
(8800/6000). retargeting uses the synthetic goal placeholder ALL:12345:30
(same convention as audiencetargets), so it records the request shape and
skips on 8800 — no real account ids are hardcoded.

Refresh four drifted cassettes that no longer matched the CLI after the
#491 refactor: ads update now requires --type, feeds add requires
--business-type, and SMART_CAMPAIGN add requires --counter-id. Re-record
under live API; smartadtargets skips on 3500 as documented.

Document in MANUAL_COVERAGE.md that the Yandex Direct API exposes no
Strategies.delete (only add/update/get/archive/unarchive), so a created
strategy can only be archived (StatusArchived=YES), not removed.

Verified: live-write replay 0 failed; full suite 2156 passed, 0 failed;
no secrets or real account ids in any cassette.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@axisrow

axisrow commented Jun 3, 2026

Copy link
Copy Markdown
Owner Author

@claude @codex review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production.

@claude

claude Bot commented Jun 3, 2026

Copy link
Copy Markdown

Claude finished @axisrow's task in 2m 31s —— View job


Review — PR #536 (live-write coverage + cassette refresh)

  • Read tests/test_v5_live_write.py
  • Inspect new cassettes (feeds, retargeting, strategies)
  • Review refreshed cassettes for drift correctness
  • Check tests/MANUAL_COVERAGE.md docs change
  • Scan for secrets / real account IDs
  • Verify error-handling & VCR replay semantics

Scoped to critical issues only (bugs, data-loss, logic, security), per the request. No blocking issues found. This is a clean, well-reasoned test-only PR. Details below.

✅ Verified correct

  • Retargeting skip path is sound. The cassette returns HTTP 200 with an embedded {"Code":8800} error. output.format_output()raise_for_api_result_errors() (direct_cli/output.py:42,74) raises DirectAPIResultError, which handle_api_errors converts to print_error + click.Abort() (exit code ≠ 0). So the guard r.exit_code != 0 and ("8800" in r.output or "not found" ...) (test_v5_live_write.py:1348) correctly triggers the skip rather than falling through to _extract_first_id (which would otherwise raise on the embedded Errors). Good — the exit_code != 0 check is meaningful here.
  • Strategies cassette replay with duplicate requests works. The lifecycle issues two byte-identical archive requests (the in-try archive and the finally archive). With match_on=[..., "body"] and replay-only record_mode (conftest.py:295), VCR resolves each call to the next unplayed matching interaction — and the cassette records exactly 5 interactions (add, update, archive, unarchive, archive) in call order, so both archives replay. Correct.
  • Refreshed cassettes match the new required-flag surface. ads update sends --type TEXT_AD (:572), feeds add sends --business-type RETAIL, campaigns add --type SMART_CAMPAIGN sends synthetic --counter-id 12345678. Consistent with the Аудит дублирования кода: системные дубли (~6000+ строк), незавершённая централизация хелперов #491 WSDL-parity gate.
  • No secrets / no real account IDs. All cassettes redact authorization/client-login/cookie/Set-Cookie. The baked IDs (feed 3436439, strategy 708915404) are resource IDs of test-created drafts — same convention as existing campaign-ID cassettes — not credentials or account/login identifiers.
  • retargeting rule shape is correctALL:12345:30 serializes to {"Operator":"ALL","Arguments":[{"ExternalId":12345,"MembershipLifeSpan":30}]}, and the MembershipLifeSpan-required note (else error 5000) is documented.

🟡 Non-blocking observations

  1. Strategies cleanup doesn't fail loudly — inconsistent with the file's own safety contract. The module header (:8) states "cleanup fails loudly with the created ID if Yandex Direct refuses deletion", and _safe_delete_campaign honors that (pytest.fail on non-zero exit). But the strategies finally (:1424-1425) fires the cleanup archive and ignores its result:

    finally:
        # No strategies delete method exists; leave the strategy archived.
        _invoke_live("strategies", "archive", "--id", str(sid))

    On a live run (account with a shared wallet → add succeeds), if that final archive fails, the test leaves a non-archived, potentially serving public strategy behind with no signal. Since strategies have no delete API, this is the one resource that can leak in an active state. Consider checking the exit code and pytest.fail-ing with the strategy ID (matching the campaign-cleanup pattern) so a failed cleanup is loud rather than silent. Replay/CI is unaffected (cassette archive succeeds); this only matters on live re-recording. Fix this →

  2. Documented, unavoidable side effect (FYI, not actionable). Because the API has no Strategies.delete, a live recording on a wallet-enabled account permanently creates a strategy that can only be archived — the recorded 708915404 is exactly this. This is correctly disclosed in MANUAL_COVERAGE.md and gated behind YANDEX_DIRECT_LIVE_WRITE=1. No spend risk since the strategy is never attached to a campaign. Flagging only so it's a conscious tradeoff.

Verdict

Solid. The skip guards, VCR matching, and cassette/CLI alignment all check out, and the docs accurately capture the API limitations. Only suggestion is tightening the strategies cleanup to fail loudly (#1) to match the module's stated contract — minor, and live-only.
· branch test/live-write-feeds-retargeting-strategies

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2288e21a62

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/test_v5_live_write.py Outdated
_assert_draft_or_success(r, "strategies unarchive")
finally:
# No strategies delete method exists; leave the strategy archived.
_invoke_live("strategies", "archive", "--id", str(sid))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fail when strategy cleanup archive is rejected

When this live-write test reaches an account where strategies add and unarchive succeed, the final archive is the only cleanup path because the service has no delete method. Discarding the _invoke_live result means a transient/API rejection of this cleanup command still lets the test pass while leaving the newly created shared strategy unarchived in the production account; mirror the feed/retargeting cleanup checks and fail with the strategy id if the archive command returns a nonzero result.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in aa60653. The strategies finally now captures the cleanup archive result and pytest.fails with the strategy id on a non-zero exit, mirroring _safe_delete_campaign and the feeds/retargeting cleanups. Replay/CI is unaffected (the cassette archive replays a 0-exit response); this only tightens the live re-record path so a rejected cleanup can't silently leave a non-archived public strategy.

The strategies live-write test's finally block fired the cleanup archive
but discarded its result, unlike _safe_delete_campaign and the feeds/
retargeting cleanups in this same PR. Since the Strategies service has no
delete method, archive is the only cleanup path: on a live re-record, a
rejected cleanup archive would silently leave a non-archived public shared
strategy in the production account — exactly what the module's "cleanup
fails loudly with the created ID" safety contract exists to prevent.

Check the exit code and pytest.fail with the strategy id, mirroring the
feeds/retargeting pattern. Replay/CI is unaffected (the cassette archive
replays a 0-exit response).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@axisrow

axisrow commented Jun 3, 2026

Copy link
Copy Markdown
Owner Author

@claude @codex review. Fixed the strategies cleanup in aa60653 — the finally block now checks the archive exit code and fails loudly with the strategy id, matching the feeds/retargeting cleanups and the module's safety contract. Please re-review. Focus on critical issues only: bugs, security, logical errors, data loss, performance. Do NOT nitpick style, naming, or formatting.

@claude

claude Bot commented Jun 3, 2026

Copy link
Copy Markdown

Claude finished @axisrow's task in 1m 21s —— View job


Re-review — PR #536 (strategies cleanup fix aa60653)

  • Review the aa60653 cleanup fix
  • Verify the finally block fails loudly with the strategy id
  • Confirm replay/CI is unaffected
  • Final verdict

The fix resolves the one non-blocking issue from the prior review. No remaining issues.

✅ Fix verified

The strategies finally now captures the cleanup-archive result and pytest.fails with the strategy id on a non-zero exit (test_v5_live_write.py:1423-1430):

finally:
    # No strategies delete method exists; archive is the only cleanup.
    r = _invoke_live("strategies", "archive", "--id", str(sid))
    if r.exit_code != 0:
        pytest.fail(
            f"Failed to archive strategy {sid}. "
            f"Manual cleanup required.\noutput: {r.output}"
        )

This is byte-for-byte the _safe_delete_campaign contract (:280-287) and matches the feeds (:1318-1323) and retargeting (:1369-1374) cleanups. The module's "cleanup fails loudly with the created ID" safety contract is now honored on every mutating resource — and since strategies is the one resource with no delete API, this is exactly where a silent leak would have been worst (a non-archived, potentially serving public shared strategy).

  • sid is always bound in finally — it's assigned at :1405 before the try, after _assert_success confirms the add, so the cleanup can never NameError.
  • Replay/CI unaffected. The fix adds an exit-code check on the already-existing cleanup archive call — no new request, so the cassette's interaction count is unchanged and still resolves both the in-try archive and the finally archive. CI confirms: Quality ✅ and API Coverage ✅ on this commit.
  • Live-only behavior change. The new branch only fires on a genuine non-zero exit during live re-recording; the cassette replays a 0-exit archive, so existing replay runs are identical.

Verdict

Looks good — ship it. The fix is correct, minimal, consistent with the existing cleanup helpers, and changes nothing on the replay/CI path. Everything else from the first review still stands (sound skip guards, correct VCR matching, no secrets/real account IDs, accurate MANUAL_COVERAGE.md notes).
· branch test/live-write-feeds-retargeting-strategies

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@axisrow
axisrow merged commit e6d41a6 into main Jun 3, 2026
8 checks passed
@axisrow
axisrow deleted the test/live-write-feeds-retargeting-strategies branch June 3, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant