Skip to content

Transition 7.71.2 7.78.2 - #354

Merged
LouisParkin merged 5 commits into
stackstate-7.78.2from
transition-7.71.2-7.78.2
Jun 9, 2026
Merged

LouisParkin merged 5 commits into
stackstate-7.78.2from
transition-7.71.2-7.78.2

Conversation

@LouisParkin

Copy link
Copy Markdown
Contributor

Step 1: Link to Jira issue

Step 2: Description of changes

Step 3: Did you add / update tests for your changes in the right area?

  • Unit/Component test
  • Integration test

Step 4: I'm confident that everything is properly tested:

I got a PO / QA Approval by:

  • Name

Step 5: Can we ship this feature to production?

  • Yes, I'm proud of my work. Ship it! 🚢

LouisParkin and others added 5 commits May 18, 2026 15:56
The agent embeds Python 3.13.13 (omnibus/config/software/python3.rb). Our dev
target was 3.11.10, so dep failures that only manifest under 3.13 (e.g. ddtrace
3.12.5 Cython compile of removed PyInt_Check) slipped past integrations CI and
broke agent build_deb. Bump dev/CI Python to match runtime.

Python:
  .python-version              3.11.10 -> 3.13.13
  .gitlab-ci.yml PYTHON_VERSION 3.11.14 -> 3.13.13

Shared deps where DD 7.78.2 is ahead of us (take DD's pin):
  boto3              1.34.11 -> 1.42.54
  ddtrace            3.12.5  -> 3.19.5    (also fixes Cython issue at Py3.13)
  kubernetes         33.1.0  -> 35.0.0
  orjson             3.11.6  -> 3.11.7
  prometheus-client  0.19.0  -> 0.24.1
  protobuf           5.29.6  -> 6.33.5    (major bump — verify proto contracts)
  pydantic           2.9.1   -> 2.12.5
  PyJWT              2.12.0  -> 2.12.1

Kept STS pins where we're ahead:
  cryptography 46.0.7 (DD: 46.0.6)

STS-only deps left alone (orionsdk, pg8000, zeep, vsphere-automation-sdk,
pynag, requests-ntlm, flatten-dict, iso8601, simplejson, uuid, pytz,
deprecated, enum34, pywin32, psycopg2-binary).

Image tag in .gitlab-ci.yml still says ...:20241120-py311. conda creates
envs at PYTHON_VERSION on demand so the bump should work as-is, but the
runner image should be rebuilt as py313 in a follow-up to avoid drift.
… shared dep bumps.

Follow-up to f78ad27 — that commit only updated the central
stackstate_checks_base/stackstate_checks/base/data/agent_requirements.in.
This commit sweeps every other requirements.in / requirements-dev.txt for
the same shared-dep pins (boto3, ddtrace, kubernetes, orjson, prometheus-client,
protobuf, pydantic, pyjwt) and applies the same DD-7.78.2-derived bumps.

Files touched:
  stackstate_checks_base/requirements.in           (6 pins)
  splunk_base/requirements.in                      (pyjwt)
  splunk_metric/requirements.in                    (pydantic + pyjwt)
  dynatrace_base/requirements-dev.txt              (pyjwt)
  dynatrace_health/requirements-dev.txt            (pyjwt)
  dynatrace_topology/requirements-dev.txt          (pyjwt)
Follow-up to 61e9d4f — the pipeline (2534428169) surfaced two blockers
the earlier sweep missed:

1. stackstate_checks_dev/setup.py:36 still hard-pinned pydantic==2.9.1,
   causing dep resolution conflicts with stackstate_checks_base[deps]==2.12.5
   in every check that has a tox env (11 of 12 failing test_* jobs).
   setup.py files weren't in the earlier sweep grep — only requirements*.in/txt
   were. Now bumped to 2.12.5.

2. Pydantic 2.9 -> 2.12 changed the docs URL embedded in ValidationError
   messages from errors.pydantic.dev/2.9/v/... to .../2.12/v/...
   Tests in 7 files asserted against the literal expected message string,
   so all 16 assertions in test_agent_check.py/test_agent_check_v2.py broke
   (with same brittleness in 5 other test files that never ran due to #1).
   Updated all 7 to /2.12/. NOTE: these are still version-pinned and will
   break again on the next pydantic bump — version-agnostic assertion
   helper would be the durable fix, deferred to a separate refactor.
…le growth.

prometheus-client 0.24 added a `native_histogram` field to the Sample
namedtuple (0.19: 5 fields, 0.24: 6). The test in test_kubelet.py:362
positional-unpacks Sample as a 5-tuple, breaking under the bump. Use a
star-rest pattern so future prometheus-client releases that add fields
don't break this assertion again.

Note: this is the only positional-unpack call site for metric.samples in
the repo — every other usage iterates as `for sample in metric.samples:`
and accesses fields by attribute name, which is already version-robust.
… failure.

Two related improvements to the openmetrics base check, prompted by a real
sandbox finding: ZooKeeper's Prometheus metrics provider serializes
synced_observers as the literal string "null" on non-Leader replicas. The
prometheus_client parser correctly rejects "null" as not-a-float, but its
exception aborts the entire payload parse — every metric emitted after the
bad line is silently lost (~200 valid ZK metrics per scrape on a typical
3-node ensemble), and the only operator-visible signal is a Python
traceback that names neither the scrape target nor the offending metric.

Change 1: promote `_text_filter_blacklist` from internal-only to a public
instance-config key `text_filter_blacklist`. The pre-parse line filter
already existed (originally implemented for the kubelet check) but was
never wired through to instance config. Now any openmetrics scrape can
defend against malformed exposition payloads by listing the metric names
to drop before parsing:

  instances:
  - prometheus_url: http://%%host%%:9141/metrics
    metrics: ["*"]
    text_filter_blacklist: ["synced_observers"]

Default-instances and instance entries are concatenated, mirroring the
behavior of exclude_labels / extra_headers / etc.

Change 2: when the prometheus parser raises mid-payload, log an ERROR
identifying the scrape endpoint, the offending metric name (parsed from
the first whitespace-separated token of the bad line), the raw line, the
parser error, the impact (downstream metrics lost), and the remediation
(add to text_filter_blacklist). Implemented via a small generator wrapper
that captures the most recently yielded line, then catches ValueError /
KeyError around the text_fd_to_metric_families iteration. The original
exception is re-raised so existing check-level error handling continues
unchanged; the new log is purely additive context.

Tests: 4 new pytest functions covering the public config wiring (instance,
default-instance, merge, default-empty), the real-shape ZK-style
reproducer (parse aborts without filter, recovers with filter), and the
ERROR-log content (endpoint + metric name + remediation hint all present).
Full openmetrics suite (40 tests) and openmetrics_base_check suite (3
tests) pass cleanly. Flake8 clean on the touched files.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
@LouisParkin
LouisParkin merged commit 10f47ac into stackstate-7.78.2 Jun 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants