Skip to content

ref(pure_eval): Use data_collection in test - #7687

Merged
sentrivana merged 1 commit into
major/3.0from
ivana/major/drop-pii-pure-eval
Sep 23, 2026
Merged

ref(pure_eval): Use data_collection in test#7687
sentrivana merged 1 commit into
major/3.0from
ivana/major/drop-pii-pure-eval

Conversation

@sentrivana

@sentrivana sentrivana commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

@linear-code

linear-code Bot commented Sep 23, 2026

Copy link
Copy Markdown

PY-2884

@sentrivana
sentrivana marked this pull request as ready for review September 23, 2026 13:33
@sentrivana
sentrivana requested a review from a team as a code owner September 23, 2026 13:33
@github-actions

github-actions Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Codecov Results 📊

✅ 65246 passed | ❌ 1 failed | ⏭️ 2971 skipped | Total: 68218 | Pass Rate: 95.64% | Execution Time: 187m 16s

📊 Comparison with Base Branch

Metric Change
Total Tests 📉 -68683
Passed Tests 📉 -64507
Failed Tests 📈 +1
Skipped Tests 📉 -4177

➕ New Tests (1)

View new tests
  • test_input_attributes_nonstreaming_chat[True-messages0-expected_system_instructions0-expected_input_messages0]
    • File: tests.integrations.mistral.test_mistral
    • Status: ❌ Failing

❌ Failed Tests

test_input_attributes_nonstreaming_chat[True-messages0-expected_system_instructions0-expected_input_messages0]

File: tests.integrations.mistral.test_mistral
Suite: py3.10-mistral-v2.0.5
Error: KeyError: 140165622772160

Stack Trace 
tests/integrations/mistral/test_mistral.py:380: in test_input_attributes_nonstreaming_chat
    sentry_init(
tests/conftest.py:349: in inner
    client = sentry_sdk.Client(*a, **kw)
sentry_sdk/client.py:278: in __init__
    self._init_impl()
sentry_sdk/client.py:428: in _init_impl
    self.integrations = setup_integrations(
sentry_sdk/integrations/__init__.py:216: in setup_integrations
    for integration_cls in iter_default_integrations(
sentry_sdk/integrations/__init__.py:44: in iter_default_integrations
    yield getattr(import_module(module), cls)
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1024: in _find_and_load
    ???
<frozen importlib._bootstrap>:171: in __enter__
    ???
<frozen importlib._bootstrap>:123: in acquire
    ???
E   KeyError: 140165622772160

✅ Patch coverage is 100.00%. Project has 2062 uncovered lines.
✅ Project coverage is 90.22%. Comparing base (03253ee) to head (affd66e).

Coverage diff 
@@            Coverage Diff             @@
##        master       #PR       +/-##
==========================================
+ Coverage    90.22%    90.22%        —%
==========================================
  Files          194       180       -14
  Lines        25997     21076     -4921
  Branches      9662      7596     -2066
==========================================
+ Hits         23454     19014     -4440
- Misses        2543      2062      -481
- Partials      1451      1197      -254

Generated by Codecov Action

@sentrivana
sentrivana merged commit 3ef8c1f into major/3.0 Sep 23, 2026
145 of 147 checks passed
@sentrivana
sentrivana deleted the ivana/major/drop-pii-pure-eval branch September 23, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexander-alderman-webb alexander-alderman-webb alexander-alderman-webb approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

See More

Successfully merging this pull request may close these issues.

2 participants

@sentrivana @alexander-alderman-webb