Skip to content

fix: close #112 — imported XSD validation, runtime-deprecated registry, advideos discovery#119

Merged
axisrow merged 4 commits into
mainfrom
fix/issue-112-imported-xsd-and-runtime-deprecated
Apr 28, 2026
Merged

fix: close #112 — imported XSD validation, runtime-deprecated registry, advideos discovery#119
axisrow merged 4 commits into
mainfrom
fix/issue-112-imported-xsd-and-runtime-deprecated

Conversation

@axisrow

@axisrow axisrow commented Apr 28, 2026

Copy link
Copy Markdown
Owner

Closes #112.

Summary

Closes the two open bug classes from the issue #112 audit (the other three were already resolved on main):

What's in PR-A (runtime-deprecated registry)

  • RUNTIME_DEPRECATED_METHODS in direct_cli/wsdl_coverage.py (registry of WSDL-present methods that fail at runtime, e.g. agencyclients.add → error_code=3500).
  • assert_not_runtime_deprecated() helper in direct_cli/utils.py raising click.UsageError with replacement hint.
  • Wired into agencyclients.add: both real-call and --dry-run paths reject before request construction.
  • New 8th schema-gate bucket runtime_deprecated_unguarded in scripts/build_api_coverage_report.py — uses captured-client harness to verify each registry entry actually rejects with UsageError before the API call. Mirrored into report["schema"] and top-level report["runtime_deprecated_methods"].

What's in PR-B (imported XSD + smoke probes)

  • Cached imported XSD schemas in tests/wsdl_cache/imports/ (general.xsd, generalclients.xsd, adextensiontypes.xsd) with IMPORTED_XSD_REGISTRY (namespace → filename) and fetch_imported_xsd().
  • get_operation_request_schema() resolves <xsd:import> nested types and extension/@base inheritance — Notification.EmailSubscriptions, Clients[].Settings and similar nested fields are now visible to the validator.
  • New 9th schema-gate bucket nested_schema_violations — dry-runs every PAYLOAD_CASES entry, flags unknown nested keys.
  • New direct_cli/_smoke_probes.py module: advideo_probe_id() (env override YANDEX_DIRECT_TEST_ADVIDEO_ID, fallback through creatives.get, returns None on any failure including auth) + CLI entry python3 -m direct_cli._smoke_probes advideo.
  • scripts/test_safe_commands.sh uses run_advideos_probe_get with [INFO] skip-on-no-id semantics — never falsely passes on a dummy ID.
  • Extended refresh_all_caches() and scripts/check_wsdl_drift.py to also refresh/monitor imported XSDs (cron picks up XSD drift the same way as WSDL drift).

Tests (9 new)

PR-A:

  • test_runtime_deprecated_registry_shape
  • test_runtime_deprecated_methods_block_invocation
  • test_runtime_deprecated_methods_block_dry_run_too
  • test_schema_gate_flags_runtime_deprecated_without_cli_guard

PR-B:

  • test_imported_xsd_cache_matches_registry
  • test_get_operation_request_schema_resolves_imports
  • test_get_operation_request_schema_resolves_extension_base_fields
  • test_payload_cases_validate_nested_imported_fields
  • test_smoke_probes_advideo_returns_none_without_credentials (caught a real bug — advideo_probe_id was not catching create_client() failures; fixed in this PR)

Verification

  • pytest -q tests/ — 338 passed, 29 skipped (integration, no creds), 0 failed.
  • python3 scripts/build_api_coverage_report.py | jq '.summary.schema_parity_ok, .schema | keys'true, 9 buckets all empty.
  • Schema gate goes from 7 to 9 buckets.

Acceptance criteria mapping (issue #112)

AC Status
1 (coverage fails on unknown nested fields for imported XSD) nested_schema_violations bucket
2 (AgencyClients.Notification.EmailSubscriptions) ✅ already on main
3 (runtime-deprecated documented in coverage policy) RUNTIME_DEPRECATED_METHODS + runtime_deprecated_unguarded bucket
4 (AdVideos.get real probe ID) _smoke_probes.advideo_probe_id + bash wrapper
5 (no magic probe IDs) ✅ already on main

🤖 Generated with Claude Code

axisrow and others added 2 commits April 28, 2026 13:36
…stry, advideos discovery

Closes #112 by addressing the two open bug classes from the audit (the other
three were resolved on main).

PR-A — Runtime-deprecated method registry (closes AC #3):
- Add RUNTIME_DEPRECATED_METHODS to direct_cli/wsdl_coverage.py (registry of
  WSDL-present methods that Yandex rejects at runtime, e.g. agencyclients.add
  with error_code=3500).
- Add assert_not_runtime_deprecated() helper in direct_cli/utils.py raising
  click.UsageError with a stable replacement hint.
- Wire the assertion into agencyclients.add so both real-call and --dry-run
  paths fail with a clear message before request construction.
- Add 8th schema gate bucket runtime_deprecated_unguarded in
  scripts/build_api_coverage_report.py: walks RUNTIME_DEPRECATED_METHODS, uses
  the captured-client harness to verify each entry's CLI command actually
  rejects with UsageError before reaching the API. Bucket flips
  schema_parity_ok to False; mirrored into report["schema"] and a top-level
  report["runtime_deprecated_methods"] section for machine-readable parity.

PR-B — Imported XSD validation + AdVideos smoke discovery (closes AC #1, #4):
- Cache imported XSD schemas in tests/wsdl_cache/imports/ (general.xsd,
  generalclients.xsd, adextensiontypes.xsd). Add IMPORTED_XSD_REGISTRY
  (namespace -> filename) and fetch_imported_xsd() in wsdl_coverage.py.
- Resolve <xsd:import> nested types in get_operation_request_schema() via
  _load_schema_contexts(), and walk extension/@base inheritance so item_fields
  for types like gc:NotificationAdd and gc:ClientUpdateItem now contain
  resolved nested + inherited fields (e.g. Notification.EmailSubscriptions,
  Clients[].Settings).
- Add find_nested_schema_violations() in wsdl_coverage.py (recursive walk of
  body keys against the resolved schema).
- Add 9th schema gate bucket nested_schema_violations: dry-runs every entry
  in PAYLOAD_CASES (excluding DRY_RUN_PAYLOAD_EXCLUSIONS) and flags any
  unknown nested key.
- Extract get_first_advideo_probe_id from tests/test_integration.py into
  reusable direct_cli/_smoke_probes.py module with CLI entry point
  (python3 -m direct_cli._smoke_probes advideo). Honors
  YANDEX_DIRECT_TEST_ADVIDEO_ID env override; returns None on any failure
  so smoke scripts can skip without falsely passing. Includes graceful
  handling of create_client failures (auth/network).
- Update scripts/test_safe_commands.sh to use run_advideos_probe_get with
  [INFO] skip-on-no-id semantics instead of dummy --ids 1.
- Extend refresh_all_caches() and scripts/check_wsdl_drift.py to also
  refresh and monitor IMPORTED_XSD_REGISTRY entries.

Tests added (9 new):
PR-A: registry shape, block invocation (real + --dry-run paths), schema gate
flags unguarded entry.
PR-B: imported XSD cache↔registry parity, resolves imports
(Notification.{Email,Lang,EmailSubscriptions}), resolves extension/@base
fields (clients.update Clients[].Notification/Settings), payload nested
violation flagging, smoke probe returns None gracefully without credentials.

Schema gate goes from 7 to 9 buckets, all green: schema_parity_ok=True.
Full test suite: 338 passed, 29 skipped (integration, no creds), 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…rsions

Older click (Python 3.9) returns exit_code=0 with usage/error text in
stdout when a missing CLI group is invoked, while newer click returns
exit_code=2. The previous logic took two paths (exit-code-based
dry_run_failed vs. JSON-parse-based invalid_json), so the same input
produced different bucket kinds across versions.

Collapse to a single dry_run_failed kind: if either the exit code is
non-zero or the output is not parseable JSON, the case is reported as
dry_run_failed with the relevant error text. invalid_json was redundant
and not asserted anywhere.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@axisrow

axisrow commented Apr 28, 2026

Copy link
Copy Markdown
Owner Author

@claude review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production.

@claude

claude Bot commented Apr 28, 2026

Copy link
Copy Markdown

Claude finished @axisrow's task in 5m 27s —— View job


PR Review

  • Read changed files
  • Analyze for bugs, security issues, logical errors, data loss risks, performance problems
  • Post findings

Summary: The PR is well-structured and closes the stated ACs. Three issues worth fixing before merge — one is a clear functional bug, one is a design fragility that will bite the next person who adds a deprecated method, and one is a minor hardcoded-URL risk.


🐛 Bug: find_nested_schema_violations reports unknown nested keys twice

File: direct_cli/wsdl_coverage.py, lines 548–579

For both the list-item branch and the dict branch, unknown keys at the first nesting level are appended explicitly and then also emitted again by the recursive call to the same function — which independently checks the same level.

Dict branch (lines 571–579):

elif isinstance(value, dict):
    for nested_key in value:
        if nested_key not in nested_expected:
            violations.append(f"{path}.{key}.{nested_key}")   # ← added here
    violations.extend(
        find_nested_schema_violations(
            {"params": value}, nested_schema, path=f"{path}.{key}"  # ← also added here
        )
    )

The recursive call with path="{path}.{key}" re-checks every key in value against the same nested_expected and appends the same path string for any unknown key. This is confirmed by the test test_payload_cases_validate_nested_imported_fieldsBogusNestedKey ends up in violations twice. The test only asserts any("BogusNestedKey" in v ...), so it passes despite the duplication.

The gate still correctly fails when violations exist (the list is non-empty), but the report produced by scripts/build_api_coverage_report.py will contain duplicate entries, making it harder to diagnose real problems.

Fix: Remove the explicit inner loop for direct unknown keys and let the recursive call handle them. Fix this →


⚠️ Design fragility: deprecation guard only runs after Click processes required options

File: direct_cli/commands/agencyclients.py, line 138 / scripts/build_api_coverage_report.py, lines 597–600

assert_not_runtime_deprecated is placed inside the Click command function body:

def add(ctx, login, first_name, ...):
    try:
        assert_not_runtime_deprecated("agencyclients", "add")
        ...

Click processes and validates required options before calling the function body. This means if required options are missing, Click raises UsageError("Missing option '--login'.") before the deprecation guard ever runs.

The gate test in _validate_runtime_deprecated_methods therefore relies entirely on RUNTIME_DEPRECATED_CAPTURE_FIXTURES providing every required option — otherwise Click's missing-option error fires first, producing a UsageError that lacks the replacement hint, and the gate flags it as a violation (reason: "UsageError did not include replacement hint"). The gate would appear to catch an "unguarded" method when the method IS guarded but the fixture is incomplete.

The current fixture for agencyclients.add is complete and correct. But there is no automated check that keeps RUNTIME_DEPRECATED_CAPTURE_FIXTURES in sync with RUNTIME_DEPRECATED_METHODS. The next person who adds a deprecated method with required options and forgets to update the fixture will get a misleading gate failure and no guidance on why.

Suggested mitigations (pick one or both):

  1. Add a docstring/comment to RUNTIME_DEPRECATED_CAPTURE_FIXTURES explicitly stating it must include all required options for the function body to be reached.
  2. Consider moving the guard to a @click.pass_context decorator or eager=True callback so it fires before option processing — though that's a larger refactor.

🔍 Minor: hardcoded XSD refresh URL on a different domain with no constant

File: direct_cli/wsdl_coverage.py, line 346

url = f"https://soap.direct.yandex.ru/v5/{filename}"

The WSDL base URL uses api.direct.yandex.com; the imported XSD refresh URL uses soap.direct.yandex.ru. The two domains serve different content and presumably this is intentional, but:

  • There is no named constant for the XSD base URL (unlike WSDL_BASE_URL)
  • There is no comment explaining why it's a different domain
  • If the URL changes, refresh_all_caches() and check_wsdl_drift.py silently break for XSDs; this would only be discovered when a developer tries to refresh the cache

Suggested fix: extract a XSD_BASE_URL constant beside WSDL_BASE_URL with a brief comment.


✅ Everything else looks good

  • The _load_schema_contexts recursion guard (import_ns not in contexts) is correct and handles mutual namespace imports without infinite recursion.
  • The _CapturedClient / captured: dict = {} inside the for mode, argv in modes: loop is correct; the # noqa: B023 is justified — the lambda is only invoked within the same iteration.
  • advideo_probe_id correctly catches create_client() failures and returns None.
  • The assert_not_runtime_deprecated in agencyclients.add is correctly placed before dry_run — the PR description specifically calls out that both real-call and --dry-run paths are rejected, which is confirmed by the tests.
  • PAYLOAD_COVERED_COMMANDS keying (using argv[0].argv[1]) is consistent with DRY_RUN_PAYLOAD_EXCLUSIONS keying.
  • The test_all_canonical_dry_run_commands_have_payload_coverage_or_exclusion test correctly keeps the registry in sync with the actual CLI.

- Dedupe nested-schema violation reports: the recursive call already
  walks the same expected-key logic for each child, so the inline loop
  that appended unknown direct keys produced duplicate entries
  (clients.update Email/ClientId would each show up twice in the gate
  output). Delegate unknown-key detection entirely to the recursion.
- Add XSD_BASE_URL constant for tests/wsdl_cache/imports/ refresh; the
  hardcoded soap.direct.yandex.ru URL is now grep-able and documented
  alongside WSDL_BASE_URL.
- Add test_runtime_deprecated_methods_have_capture_fixture to enforce
  that every RUNTIME_DEPRECATED_METHODS entry has a complete
  RUNTIME_DEPRECATED_CAPTURE_FIXTURES record covering all of the target
  command's required options. Without this, Click would fail with
  "Missing option ..." before the deprecation guard runs and the gate
  would misclassify a guarded method as unguarded. Added a docstring on
  RUNTIME_DEPRECATED_CAPTURE_FIXTURES referencing the same invariant.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@axisrow

axisrow commented Apr 28, 2026

Copy link
Copy Markdown
Owner Author

Addressed all three review findings in db4d475.

1. Double-reporting in find_nested_schema_violations — fixed. The inline loop that appended unknown direct children duplicated the recursive call's work. Removed both inline loops; the recursive call now handles direct and deeply nested unknown keys uniformly. Verified manually: clients.update (with bogus payload) used to emit ['params.Clients[0].ClientId', 'params.Clients[0].Email', 'params.Clients[0].ClientId', 'params.Clients[0].Email']; now emits each path exactly once.

2. Deprecation guard fixture brittleness — addressed by test_runtime_deprecated_methods_have_capture_fixture. The test walks RUNTIME_DEPRECATED_METHODS, looks up each command's required options via the live Click app, and asserts that RUNTIME_DEPRECATED_CAPTURE_FIXTURES has a matching entry. Future contributors who add a deprecated method with required options will get a clear failure pointing at the missing fixture. Also added a docstring on RUNTIME_DEPRECATED_CAPTURE_FIXTURES explicitly stating the invariant.

3. Hardcoded XSD URL — fixed. Extracted XSD_BASE_URL = "https://soap.direct.yandex.ru/v5/{filename}" next to WSDL_BASE_URL, with an inline comment explaining the cross-domain split (api.direct.yandex.com for WSDL vs. soap.direct.yandex.ru for XSDs) so future failures are grep-able.

Verification: 250 passed, schema_parity_ok=True with all 9 buckets empty.

Note on the third Codex/external finding (clients.update / agencyclients.update payload contract divergence) — that was tracked separately as #120; not in scope for this PR.

@axisrow

axisrow commented Apr 28, 2026

Copy link
Copy Markdown
Owner Author

@claude review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production.

@claude

claude Bot commented Apr 28, 2026

Copy link
Copy Markdown

Claude finished @axisrow's task in 4m 11s —— View job


PR Review

  • Read changed files
  • Analyze for bugs, security issues, logical errors, data loss risks, performance problems
  • Post findings

Summary: All three findings from the previous review pass are correctly fixed. This pass found no new critical bugs, security issues, or data loss risks. One minor performance note worth awareness below.


✅ Previous findings — all addressed correctly

1. Double-reporting in find_nested_schema_violations
Fixed cleanly in direct_cli/wsdl_coverage.py:536-581. Both inline loops that duplicated the recursive call were removed. The comment explaining the delegation is correct and the logic is clean.

2. Deprecation guard fixture brittleness
test_runtime_deprecated_methods_have_capture_fixture (tests/test_api_coverage.py:1238) correctly walks RUNTIME_DEPRECATED_METHODS, introspects required options via the live Click app, and asserts a fixture entry exists. The RUNTIME_DEPRECATED_CAPTURE_FIXTURES docstring in scripts/build_api_coverage_report.py:457-462 makes the invariant explicit. The gate test test_runtime_deprecated_gate_rejects_parser_error_without_replacement further verifies that a Click parser error without the replacement hint is correctly flagged as a violation.

3. Hardcoded XSD URL
XSD_BASE_URL = "https://soap.direct.yandex.ru/v5/{filename}" at direct_cli/wsdl_coverage.py:20 with an inline comment explaining the cross-domain split. Clean.


⚠️ Minor: _field_name_operations calls get_operation_field_name_enums twice per operation

File: scripts/build_api_coverage_report.py, lines 83–90

def _field_name_operations(fetch_wsdl_func, api_service: str) -> dict[str, dict]:
    wsdl_xml = _fetch_wsdl(fetch_wsdl_func, api_service)
    return {
        operation: get_operation_field_name_enums(wsdl_xml, operation)
        for operation in parse_wsdl_operations(wsdl_xml)
        if get_operation_field_name_enums(wsdl_xml, operation)  # ← called again
    }

Each get_operation_field_name_enums call invokes both get_operation_request_schema and parse_wsdl_field_enums, both of which call _load_schema_contexts (which parses XML and resolves imported XSDs). For a service with N operations this is called 2×N times; across ~27 services with ~5 ops each, _load_schema_contexts runs ~270 times per report build. The XSD reads are cached (_cached_imported_xsd has lru_cache), but WSDL XML is parsed from string each time.

This is a pre-existing pattern (the double-call was there before this PR; what's new is _load_schema_contexts being called by both get_operation_request_schema and parse_wsdl_field_enums). No correctness issue — only CI build-time impact. Fix this →


✅ Everything else looks correct

  • _load_schema_contexts recursion guard is correct: target_ns in contexts is checked first inside add_schema, and import_ns not in contexts is checked before recursing. Handles mutual namespace imports without infinite recursion.
  • _collect_complex_type_fields seen: frozenset correctly prevents cycles in xsd:extension inheritance chains.
  • _advideo_get_accepts passes string IDs to Ids — this is consistent with how advideos.get CLI command itself constructs the body (line 38: [x.strip() for x in ids.split(",")]), so no type mismatch vs. the API.
  • assert_not_runtime_deprecatedexcept click.UsageError: raise in agencyclients.add correctly propagates the guard's error past the generic except Exception handler.
  • _CapturedClient lambda + # noqa: B023 is justified: captured is a fresh dict = {} per loop iteration and the lambda is only invoked within that iteration's try block.
  • XSD_BASE_URL.format(filename=filename) in fetch_imported_xsd is safe: filename comes from IMPORTED_XSD_REGISTRY.values(), a hardcoded dict of three known filenames — no user-controlled input reaches this format call.

The _field_name_operations comprehension called the function twice per
operation: once in the if-filter and once as the value. Each call walks
get_operation_request_schema → _load_schema_contexts → parse_wsdl_field_enums,
so for ~27 services × ~5 operations this was ~270 redundant
schema-context loads per coverage report build. Compute the dict once,
then filter empty values.

No behavior change. Pre-existing inefficiency that became more visible
once _load_schema_contexts started resolving imported XSD types.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@axisrow axisrow merged commit b2f2d6b into main Apr 28, 2026
6 checks passed
@axisrow axisrow deleted the fix/issue-112-imported-xsd-and-runtime-deprecated branch April 28, 2026 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Audit WSDL/schema coverage drift for nested fields, deprecated methods, and ID discovery

1 participant