Skip to content

fix(docs): point removed-doc services at live WSDL (closes #463)#464

Merged
axisrow merged 3 commits into
mainfrom
fix/463-docs-url-drift
May 29, 2026
Merged

fix(docs): point removed-doc services at live WSDL (closes #463)#464
axisrow merged 3 commits into
mainfrom
fix/463-docs-url-drift

Conversation

@axisrow

@axisrow axisrow commented May 29, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes the 4 stale RESOURCE_MAPPING_V5 docs URLs that hard-failed the release preflight (dynamicads, dynamicfeedadtargets, smartadtargets, vcards).

Root cause (confirmed via browser): Yandex removed the human-readable doc pages for these services around September 2025. Both the /ru/ and /en/ pages now 404, and the services are gone from the docs navigation ("Основные объекты" menu no longer lists them; ref-v5/<svc>/<svc>.html redirects to a 404/default). This is a deletion, not a slug move — which is why no canonical replacement page exists.

The services themselves remain live — their WSDL endpoints return HTTP 200:

api.direct.yandex.com/v5/{dynamictextadtargets,dynamicfeedadtargets,smartadtargets,vcards}?wsdl  → 200

Change

  • RESOURCE_MAPPING_V5: repoint the 4 docs URLs at their WSDL endpoints — the only authoritative source Yandex still serves for them — with a comment explaining the Sep-2025 removal.
  • check_all_docs_urls.py: validate ?wsdl/.wsdl URLs as real WSDL (XML + wsdl:definitions) rather than the 30 KB HTML-page threshold (WSDL payloads are 14–21 KB).
  • audit_wire_shape.py: treat a WSDL docs base as a single __group__ target instead of deriving per-method URLs (which would produce malformed …?wsdl/get). Regression test added.

Functional impact (corrected)

The docs field is read at runtime: _register_command (cli.py:316) calls get_docs_url(...) and appends Documentation: <url> to each group's --help epilog. So after this change, direct dynamicads|dynamicfeedadtargets|smartadtargets|vcards --help print a WSDL link instead of the old (now-dead 404) doc URL. This is a strict improvement — a live WSDL link beats a broken page — but it is a real user-facing help-text change, not "zero impact" as an earlier draft of this description claimed.

Why WSDL and not a doc page

Investigated thoroughly: the doc pages are gone on every language/host; the WSDL is the only live, authoritative description still served. WSDL does not contain links to doc pages, so there is nothing else to point at.

Note

Preflight now depends on api.direct.yandex.com/v5/<svc>?wsdl being publicly fetchable without a token (holds today; flagged for awareness — if Yandex ever gates WSDL behind auth, preflight would need adjusting).

Testing

  • python scripts/check_all_docs_urls.pyall URLs canonical (the 4 now report valid WSDL).
  • Full suite green: 2036 passed, 47 skipped; ruff check . clean.

Closes #463

🤖 Generated with Claude Code

Yandex removed the human-readable doc pages for DynamicTextAdTargets,
DynamicFeedAdTargets, SmartAdTargets and VCards in September 2025 — the
/ru/ and /en/ pages now 404 and the services are gone from the docs
navigation (confirmed via browser; the services are absent from the
"Основные объекты" menu). The services themselves stay live.

- RESOURCE_MAPPING_V5: repoint the 4 `docs` URLs at their WSDL endpoints
  (api.direct.yandex.com/v5/<service>?wsdl), the only authoritative source
  still served, with a comment explaining the Sep-2025 removal.
- check_all_docs_urls.py: validate ?wsdl/.wsdl URLs as real WSDL (XML +
  wsdl:definitions) instead of the 30 KB HTML-page threshold, so preflight
  passes on the smaller XML payloads.

The `docs` field is preflight-only (never used at runtime — requests build
from `resource`), so this has no functional impact. Full suite green
(2035 passed); `check_all_docs_urls.py` reports all URLs canonical.

Closes #463

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@axisrow axisrow added this to the 0.4.1 milestone May 29, 2026
@axisrow

axisrow commented May 29, 2026

Copy link
Copy Markdown
Owner Author

@claude review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production.

@claude

claude Bot commented May 29, 2026

Copy link
Copy Markdown

Claude finished @axisrow's task in 2m 49s —— View job


Review: PR #464 — repoint removed-doc services at live WSDL

  • Read changed files
  • Analyze RESOURCE_MAPPING_V5 URL changes
  • Analyze check_all_docs_urls.py WSDL validation
  • Check downstream consumers of the docs field
  • Post review

The core fix is correct: the 4 doc pages are genuinely 404, the WSDL endpoints are the only authoritative source still served, and they live inside the registry so the "no URL literals outside the registry" invariant is respected. The WSDL-validation branch in check_all_docs_urls.py (XML + wsdl:definitions/<definitions marker) is sound. No security or data-loss concerns.

Two issues, both low severity (neither blocks CI or the documented preflight), plus one note.


1. The PR's "never read at runtime / zero functional impact" claim is incorrect

docs is read at runtime. cli.py:314-321 looks up get_docs_url(command.name) for every command group and appends it to the --help epilog:

docs_url = get_docs_url(command.name or "")   # cli.py:316
if docs_url:
    docs_line = f"\b\nDocumentation: {docs_url}"

So direct dynamicads --help, direct vcards --help, direct smartadtargets --help, and direct dynamicfeedadtargets --help will now print:

Documentation: https://api.direct.yandex.com/v5/dynamictextadtargets?wsdl

This is not a functional break — the old doc URLs are dead 404s, so a live WSDL link is strictly better than a broken one — but the stated rationale is wrong, and there is a real user-facing help-text change. Worth correcting the PR description so reviewers/release notes reflect that --help now points these 4 groups at a WSDL file.

2. scripts/audit_wire_shape.py now derives malformed URLs for these services

audit_v5 builds per-method doc URLs via _v5_per_method_url(docs_base, op) (audit_wire_shape.py:588). With the new WSDL base, for dynamictextadtargets?wsdl + get it returns:

https://api.direct.yandex.com/v5/dynamictextadtargets?wsdl/get

— a garbage URL that will fetch-fail and emit docs_unreachable findings for all 4 services × every method. _v5_per_method_url's tail-chop heuristic assumes a …/ru/<svc> doc-page shape and doesn't recognize a ?wsdl query base.

This is not run in CI or the documented preflight (only the helper functions are unit-tested), so it doesn't fail the build — but anyone running the manual wire-shape audit will now get false "unreachable" noise for these 4 groups. Consider short-circuiting ?wsdl/.wsdl bases in audit_v5 (treat as a single __group__ target, skip per-method derivation). Fix this →

3. Note (not blocking): WSDL preflight depends on unauthenticated access

check_all_docs_urls.py now relies on api.direct.yandex.com/v5/<svc>?wsdl being publicly fetchable without a token. That holds today (author verified 200), but if Yandex ever gates the WSDL behind auth, preflight would report GONE/SMALL and hard-fail the release. No action needed now — just be aware the preflight surface moved from the docs host to the API host.


Verdict: The change fixes #463 correctly and is safe to merge. Recommend (a) correcting the "never read at runtime" claim in the PR body, and (b) optionally guarding the audit tool (#2) in this or a follow-up PR.
· branch fix/463-docs-url-drift

After #463 repointed 4 services' `docs` at WSDL endpoints, audit_v5's
_v5_per_method_url derived malformed URLs (`…?wsdl/get`) for them,
emitting false docs_unreachable findings for every method. Now a WSDL
base (`?wsdl`/`.wsdl`) is audited as one __group__ target instead of
deriving per-method URLs. Regression test added.

Addresses review note on PR #464.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@axisrow

axisrow commented May 29, 2026

Copy link
Copy Markdown
Owner Author

@claude review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production.

@claude

claude Bot commented May 29, 2026

Copy link
Copy Markdown

Claude finished @axisrow's task in 1m 49s —— View job


Review: PR #464 — repoint removed-doc services at live WSDL

  • Read changed files
  • Analyze RESOURCE_MAPPING_V5 URL changes
  • Analyze check_all_docs_urls.py WSDL validation
  • Analyze audit_wire_shape.py __group__ change + test
  • Post review

This revision resolves both findings from the prior review: the PR body now correctly states the --help epilog impact (no more "zero functional impact" claim), and audit_wire_shape.py now short-circuits WSDL bases instead of deriving the garbage …?wsdl/get URLs. The fixes are correct and safe to merge. No bugs, security issues, or data-loss risks.

I confirmed the only runtime consumer of docs is cli.py:316 (get_docs_url--help epilog); test_cli.py:86 reads the value dynamically via get_docs_url(...), so it won't break. The parametrize inputs at test_audit_wire_shape.py:146-153 are function-level test cases for _v5_per_method_url, not registry assertions — also unaffected.

Verified specifics:

  • audit_wire_shape.py:578docs_base = info.get("docs", "") defaults to "", so the .rstrip("/") on line 588 can't hit None. WSDL short-circuit logic is sound.
  • check_all_docs_urls.py:107-109 runs the captcha check before the WSDL branch, so a rate-limit gate is still caught (returns None → retry) rather than misclassified as "WSDL markers absent".

One non-blocking observation (low severity)

check_all_docs_urls.py:114-117 validates WSDL purely by marker presence (wsdl:definitions / <definitions) with no minimum-size floor, whereas the repo's documented freshness guard (wsdl_coverage.fetch_wsdl, per CLAUDE.md "Docs/cache freshness guard") rejects WSDL bodies <3 KB. A truncated or partial response that still contains <definitions would pass here as OK. Real Yandex WSDLs are 14–21 KB, so this is unlikely to bite, but adding a small floor would keep the preflight consistent with the freshness-guard philosophy used elsewhere:

if url.rstrip("/").endswith("?wsdl") or url.endswith(".wsdl"):
    if ("wsdl:definitions" in lower or "<definitions" in lower) and len(body) >= 3072:
        return ("OK", f"GET {resp.status_code}, valid WSDL, {len(body)} bytes")
    return ("SMALL", f"WSDL markers/size insufficient ({len(body)} bytes)")

Apply this →


Verdict: ✅ Correctly fixes #463 and addresses the prior review. Safe to merge. The size-floor note is optional hardening, not a blocker.
· branch fix/463-docs-url-drift

WSDL preflight validation passed on marker presence alone; a truncated
body containing <definitions would have reported OK. Add the same 3 KB
floor wsdl_coverage.fetch_wsdl uses (CLAUDE.md freshness guard) so a
partial WSDL reports SMALL. Real Yandex WSDLs are 14-21 KB.

Addresses non-blocking review note on PR #464.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@axisrow

axisrow commented May 29, 2026

Copy link
Copy Markdown
Owner Author

@claude review. Focus on critical issues: bugs, security vulnerabilities, logical errors, data loss risks, performance problems. Do NOT nitpick style, naming conventions, minor formatting, or subjective preferences — only flag issues that could break functionality or cause real harm in production.

@claude

claude Bot commented May 29, 2026

Copy link
Copy Markdown

Claude finished @axisrow's task in 1m 19s —— View job


Review: PR #464 — repoint removed-doc services at live WSDL

  • Read changed files (current state of branch)
  • Re-check WSDL validation in check_all_docs_urls.py (incl. <3 KB floor)
  • Re-check audit_wire_shape.py __group__ short-circuit + test
  • Verify RESOURCE_MAPPING_V5 URL changes
  • Post review

This is the third pass. Both prior findings (the misleading "zero functional impact" claim and the malformed …?wsdl/get audit URLs) are resolved, and the optional <3 KB WSDL floor from the last review has been added (MIN_WSDL_BYTES = 3_072, commit 72a3c0e). No bugs, security issues, logical errors, or data-loss risks. Safe to merge.

I re-verified the points that could plausibly still break:

check_all_docs_urls.py — order of checks in _probe is correct:

  • Captcha markers (line 110-112) run before the WSDL branch, so a SmartCaptcha gate still returns None (retry) instead of being misread as "markers absent".
  • HEAD-200 / non-redirect WSDL responses fall through to the GET (lines 82-86) and then into the WSDL branch — no premature OK same-path redirect short-circuit.
  • The <3 KB floor (line 121) closes the truncated-body gap; markers-without-size now correctly reports SMALL. Consistent with wsdl_coverage.fetch_wsdl.

audit_wire_shape.py:578-604docs_base = info.get("docs", "") defaults to "", so "".rstrip("/").endswith("?wsdl") is safe (no None/crash); empty base falls into the else and is handled by the existing if not targets and docs_base guard. WSDL bases produce exactly one ("__group__", docs_base) target. The new test test_audit_v5_treats_wsdl_base_as_single_group_target asserts both the single-target shape and the absence of any ?wsdl/ URL — good regression coverage.

resource_mapping.py — the 4 URLs are repointed inside the registry, so the "no URL literals outside the registry" invariant (CLAUDE.md) holds. The only runtime consumer is get_docs_url--help epilog (cli.py:316), which the PR body now correctly documents.

Non-blocking observations (no action required)

  1. Preflight depends on unauthenticated WSDL access (already flagged by the author in the PR "Note"). api.direct.yandex.com/v5/<svc>?wsdl must be fetchable without a token; holds today. If Yandex ever gates WSDL behind auth, preflight would report GONE/SMALL and hard-fail a release. Awareness only.

  2. ?wsdl detection is string-suffix based, not query-param parsing. …/vcards?wsdl matches; a hypothetical …/vcards?wsdl&v=5 would not. None of the registry entries use such a form, so this is purely theoretical — not worth adding parsing for.


Verdict: ✅ Correctly fixes #463 and incorporates all prior review feedback. No blocking issues. Recommend merge.

Note: I could not run the test suite locally (sandbox blocked pytest); this review is from code inspection. CI / the author's reported 2036 passed, 47 skipped covers execution.
· branch fix/463-docs-url-drift

@axisrow axisrow merged commit b15498e into main May 29, 2026
6 checks passed
@axisrow axisrow deleted the fix/463-docs-url-drift branch May 29, 2026 16:46
axisrow added a commit that referenced this pull request Jun 3, 2026
…#547)

The tapi-yandex-direct 2026.5.29 vendor update (1ee8c2a) silently reverted
the #464 fix: it restored the dead `…/dev/direct/doc/ru/<service>` HTML doc
URLs for dynamicads, dynamicfeedadtargets, smartadtargets and vcards. Yandex
removed those pages in September 2025 — they now 404, so preflight
(check_all_docs_urls.py) failed. The vendor sync is rm -rf + cp -R from the
fork, which overwrote the manual edit.

- resource_mapping.py: restore the 4 `docs` URLs to their live WSDL endpoints
  (api.direct.yandex.com/v5/<service>?wsdl), matching b15498e exactly.
- test_audit_wire_shape.py: add an offline regression guard
  (test_removed_doc_services_pin_wsdl_url) that fails in CI before the network
  preflight if any of the 4 reverts to an HTML doc page.
- CHANGELOG: document the regression and re-fix.

The fork itself is fixed in a separate PR so the next vendor update no longer
re-introduces the dead URLs.

Preflight now reports all URLs canonical; full suite green (2181 passed).

Co-authored-by: axisrow <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs-URL drift: dynamicads/dynamicfeedadtargets/smartadtargets/vcards return 404 in RESOURCE_MAPPING_V5

1 participant