fix: close #112 — imported XSD validation, runtime-deprecated registry, advideos discovery#119
Conversation
…stry, advideos discovery Closes #112 by addressing the two open bug classes from the audit (the other three were resolved on main). PR-A — Runtime-deprecated method registry (closes AC #3): - Add RUNTIME_DEPRECATED_METHODS to direct_cli/wsdl_coverage.py (registry of WSDL-present methods that Yandex rejects at runtime, e.g. agencyclients.add with error_code=3500). - Add assert_not_runtime_deprecated() helper in direct_cli/utils.py raising click.UsageError with a stable replacement hint. - Wire the assertion into agencyclients.add so both real-call and --dry-run paths fail with a clear message before request construction. - Add 8th schema gate bucket runtime_deprecated_unguarded in scripts/build_api_coverage_report.py: walks RUNTIME_DEPRECATED_METHODS, uses the captured-client harness to verify each entry's CLI command actually rejects with UsageError before reaching the API. Bucket flips schema_parity_ok to False; mirrored into report["schema"] and a top-level report["runtime_deprecated_methods"] section for machine-readable parity. PR-B — Imported XSD validation + AdVideos smoke discovery (closes AC #1, #4): - Cache imported XSD schemas in tests/wsdl_cache/imports/ (general.xsd, generalclients.xsd, adextensiontypes.xsd). Add IMPORTED_XSD_REGISTRY (namespace -> filename) and fetch_imported_xsd() in wsdl_coverage.py. - Resolve <xsd:import> nested types in get_operation_request_schema() via _load_schema_contexts(), and walk extension/@base inheritance so item_fields for types like gc:NotificationAdd and gc:ClientUpdateItem now contain resolved nested + inherited fields (e.g. Notification.EmailSubscriptions, Clients[].Settings). - Add find_nested_schema_violations() in wsdl_coverage.py (recursive walk of body keys against the resolved schema). - Add 9th schema gate bucket nested_schema_violations: dry-runs every entry in PAYLOAD_CASES (excluding DRY_RUN_PAYLOAD_EXCLUSIONS) and flags any unknown nested key. - Extract get_first_advideo_probe_id from tests/test_integration.py into reusable direct_cli/_smoke_probes.py module with CLI entry point (python3 -m direct_cli._smoke_probes advideo). Honors YANDEX_DIRECT_TEST_ADVIDEO_ID env override; returns None on any failure so smoke scripts can skip without falsely passing. Includes graceful handling of create_client failures (auth/network). - Update scripts/test_safe_commands.sh to use run_advideos_probe_get with [INFO] skip-on-no-id semantics instead of dummy --ids 1. - Extend refresh_all_caches() and scripts/check_wsdl_drift.py to also refresh and monitor IMPORTED_XSD_REGISTRY entries. Tests added (9 new): PR-A: registry shape, block invocation (real + --dry-run paths), schema gate flags unguarded entry. PR-B: imported XSD cache↔registry parity, resolves imports (Notification.{Email,Lang,EmailSubscriptions}), resolves extension/@base fields (clients.update Clients[].Notification/Settings), payload nested violation flagging, smoke probe returns None gracefully without credentials. Schema gate goes from 7 to 9 buckets, all green: schema_parity_ok=True. Full test suite: 338 passed, 29 skipped (integration, no creds), 0 failed. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…rsions Older click (Python 3.9) returns exit_code=0 with usage/error text in stdout when a missing CLI group is invoked, while newer click returns exit_code=2. The previous logic took two paths (exit-code-based dry_run_failed vs. JSON-parse-based invalid_json), so the same input produced different bucket kinds across versions. Collapse to a single dry_run_failed kind: if either the exit code is non-zero or the output is not parseable JSON, the case is reported as dry_run_failed with the relevant error text. invalid_json was redundant and not asserted anywhere. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
@claude review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production. |
|
Claude finished @axisrow's task in 5m 27s —— View job PR Review
Summary: The PR is well-structured and closes the stated ACs. Three issues worth fixing before merge — one is a clear functional bug, one is a design fragility that will bite the next person who adds a deprecated method, and one is a minor hardcoded-URL risk. 🐛 Bug:
|
- Dedupe nested-schema violation reports: the recursive call already walks the same expected-key logic for each child, so the inline loop that appended unknown direct keys produced duplicate entries (clients.update Email/ClientId would each show up twice in the gate output). Delegate unknown-key detection entirely to the recursion. - Add XSD_BASE_URL constant for tests/wsdl_cache/imports/ refresh; the hardcoded soap.direct.yandex.ru URL is now grep-able and documented alongside WSDL_BASE_URL. - Add test_runtime_deprecated_methods_have_capture_fixture to enforce that every RUNTIME_DEPRECATED_METHODS entry has a complete RUNTIME_DEPRECATED_CAPTURE_FIXTURES record covering all of the target command's required options. Without this, Click would fail with "Missing option ..." before the deprecation guard runs and the gate would misclassify a guarded method as unguarded. Added a docstring on RUNTIME_DEPRECATED_CAPTURE_FIXTURES referencing the same invariant. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
Addressed all three review findings in db4d475. 1. Double-reporting in 2. Deprecation guard fixture brittleness — addressed by 3. Hardcoded XSD URL — fixed. Extracted Verification: 250 passed, schema_parity_ok=True with all 9 buckets empty. Note on the third Codex/external finding ( |
|
@claude review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production. |
|
Claude finished @axisrow's task in 4m 11s —— View job PR Review
Summary: All three findings from the previous review pass are correctly fixed. This pass found no new critical bugs, security issues, or data loss risks. One minor performance note worth awareness below. ✅ Previous findings — all addressed correctly1. Double-reporting in 2. Deprecation guard fixture brittleness 3. Hardcoded XSD URL
|
The _field_name_operations comprehension called the function twice per operation: once in the if-filter and once as the value. Each call walks get_operation_request_schema → _load_schema_contexts → parse_wsdl_field_enums, so for ~27 services × ~5 operations this was ~270 redundant schema-context loads per coverage report build. Compute the dict once, then filter empty values. No behavior change. Pre-existing inefficiency that became more visible once _load_schema_contexts started resolving imported XSD types. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Closes #112.
Summary
Closes the two open bug classes from the issue #112 audit (the other three were already resolved on
main):advideos.getintegration uses a real probe ID viadirect_cli._smoke_probes, never a magic--ids 1.What's in PR-A (runtime-deprecated registry)
RUNTIME_DEPRECATED_METHODSindirect_cli/wsdl_coverage.py(registry of WSDL-present methods that fail at runtime, e.g.agencyclients.add→ error_code=3500).assert_not_runtime_deprecated()helper indirect_cli/utils.pyraisingclick.UsageErrorwith replacement hint.agencyclients.add: both real-call and--dry-runpaths reject before request construction.runtime_deprecated_unguardedinscripts/build_api_coverage_report.py— uses captured-client harness to verify each registry entry actually rejects withUsageErrorbefore the API call. Mirrored intoreport["schema"]and top-levelreport["runtime_deprecated_methods"].What's in PR-B (imported XSD + smoke probes)
tests/wsdl_cache/imports/(general.xsd, generalclients.xsd, adextensiontypes.xsd) withIMPORTED_XSD_REGISTRY(namespace → filename) andfetch_imported_xsd().get_operation_request_schema()resolves<xsd:import>nested types andextension/@baseinheritance —Notification.EmailSubscriptions,Clients[].Settingsand similar nested fields are now visible to the validator.nested_schema_violations— dry-runs everyPAYLOAD_CASESentry, flags unknown nested keys.direct_cli/_smoke_probes.pymodule:advideo_probe_id()(env overrideYANDEX_DIRECT_TEST_ADVIDEO_ID, fallback throughcreatives.get, returnsNoneon any failure including auth) + CLI entrypython3 -m direct_cli._smoke_probes advideo.scripts/test_safe_commands.shusesrun_advideos_probe_getwith[INFO] skip-on-no-idsemantics — never falsely passes on a dummy ID.refresh_all_caches()andscripts/check_wsdl_drift.pyto also refresh/monitor imported XSDs (cron picks up XSD drift the same way as WSDL drift).Tests (9 new)
PR-A:
test_runtime_deprecated_registry_shapetest_runtime_deprecated_methods_block_invocationtest_runtime_deprecated_methods_block_dry_run_tootest_schema_gate_flags_runtime_deprecated_without_cli_guardPR-B:
test_imported_xsd_cache_matches_registrytest_get_operation_request_schema_resolves_importstest_get_operation_request_schema_resolves_extension_base_fieldstest_payload_cases_validate_nested_imported_fieldstest_smoke_probes_advideo_returns_none_without_credentials(caught a real bug —advideo_probe_idwas not catchingcreate_client()failures; fixed in this PR)Verification
pytest -q tests/— 338 passed, 29 skipped (integration, no creds), 0 failed.python3 scripts/build_api_coverage_report.py | jq '.summary.schema_parity_ok, .schema | keys'—true, 9 buckets all empty.Acceptance criteria mapping (issue #112)
nested_schema_violationsbucketAgencyClients.Notification.EmailSubscriptions)mainRUNTIME_DEPRECATED_METHODS+runtime_deprecated_unguardedbucketAdVideos.getreal probe ID)_smoke_probes.advideo_probe_id+ bash wrappermain🤖 Generated with Claude Code