Skip to content

Add agentic CI doctor workflow#4258

Merged
dondonz merged 10 commits into
masterfrom
add-agentic-ci
Mar 7, 2026
Merged

Add agentic CI doctor workflow#4258
dondonz merged 10 commits into
masterfrom
add-agentic-ci

Conversation

@dondonz
Copy link
Copy Markdown
Member

@dondonz dondonz commented Feb 22, 2026 •
edited
Loading

Adds the ci-doctor agentic workflow, which automatically investigates CI failures. This is one of GitHub's own agentic pipelines from their documentation.

When a monitored workflow fails, ci-doctor analyses failing builds and either comments on the PR or opens an issue (in the unlikely case of a master build failure)

It's not easy to test this as a dry-run. The agentic pipeline will only show up in the list of pipelines after merging into master. My plan is to give it a test on a deliberately failing pull request after this is merged into master.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 22, 2026 •
edited
Loading

Test Results

  335 files  ±0    335 suites  ±0   5m 4s ⏱️ +2s
5 378 tests ±0  5 370 ✅ +1  8 💤  - 1  0 ❌ ±0 
5 467 runs  ±0  5 459 ✅ +1  8 💤  - 1  0 ❌ ±0 

Results for commit 7bb6455. ± Comparison against base commit bdd53e6.

This pull request removes 196 and adds 172 tests. Note that renamed tests count towards both. 
	?

	, expected: combo-\"\\\b\f\n\r\t, #4]
                __schema { types { fields { args { type { name fields { name }}}}}}
                __schema { types { fields { type { name fields { name }}}}}
                __schema { types { inputFields { type { inputFields { name }}}}}
                __schema { types { interfaces { fields { type { interfaces { name } } } } } }
                __schema { types { name} }
                __type(name : "t") { name }
                a1: __schema { types { name} }
                a1: __type(name : "t") { name }
…

graphql.AssertTest ‑ assertFalse with different number of error args but false does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_21prov0_closure23@1e60b459 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0]
graphql.AssertTest ‑ assertFalse with different number of error args but false does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_21prov0_closure24@7c0777b5 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2, #1]
graphql.AssertTest ‑ assertFalse with different number of error args but false does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_21prov0_closure25@29ebbdf4 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2 arg3, #2]
graphql.AssertTest ‑ assertFalse with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_20prov0_closure20@6ba060f3 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0]
graphql.AssertTest ‑ assertFalse with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_20prov0_closure21@411a5965 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2, #1]
graphql.AssertTest ‑ assertFalse with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_20prov0_closure22@18a25bbd delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2 arg3, #2]
graphql.AssertTest ‑ assertNotNull with different number of  error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_5prov0_closure3@5a0bef24 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0]
graphql.AssertTest ‑ assertNotNull with different number of  error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_5prov0_closure4@17092fff delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2, #1]
graphql.AssertTest ‑ assertNotNull with different number of  error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_5prov0_closure5@22c75c01 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2 arg3, #2]
graphql.AssertTest ‑ assertNotNull with different number of error args with non null does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_6prov0_closure6@63d5874f delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0]
…

♻️ This comment has been updated with latest results.

@dondonz dondonz marked this pull request as draft February 22, 2026 20:06
@dondonz
Copy link
Copy Markdown
Member Author

dondonz commented Feb 22, 2026

Keeping this in draft for now, I want to give this a test, even though the downside risk is minimal

dondonz
dondonz commented Feb 22, 2026
View reviewed changes
Comment thread .gitignore
@@ -1,3 +1,4 @@
.claude/worktrees/
Copy link
Copy Markdown
Member Author

@dondonz dondonz Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And adding a slightly off topic change: I've found Claude & git worktrees very handy for multiple PRs at the same time

@dondonz
Copy link
Copy Markdown
Member Author

dondonz commented Feb 22, 2026 •
edited
Loading

An idea: we don't read our performance pipeline output often, and until now I was manually reading a JSON diff whenever we had a "big" change worth measuring!

What if we had an agentic pipeline constantly monitoring for performance regressions?

We can also have a pipeline to explain errors, although everyone can click through the build output, it's handy to have the right paragraph already extracted.

dondonz
dondonz commented Feb 22, 2026
View reviewed changes
Comment thread .github/workflows/ci-doctor.md Outdated
types:
- completed
branches:
- main
Copy link
Copy Markdown
Member Author

@dondonz dondonz Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be master

@dondonz dondonz marked this pull request as ready for review February 24, 2026 20:28
bbakerman
bbakerman approved these changes Mar 7, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@bbakerman bbakerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ummmm I guess I trust it ??

@dondonz
Copy link
Copy Markdown
Member Author

dondonz commented Mar 7, 2026

Awkwardly I have to test this one after merging. If the error messages are low quality I'll bin it

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 7, 2026

Test Report

Test Results

Java Version Total Passed Failed Errors Skipped
Java 11 5671 (±0) 5614 (±0) 0 (±0) 0 (±0) 57 (±0)
Java 17 5671 (±0) 5613 (±0) 0 (±0) 0 (±0) 58 (±0)
Java 21 5671 (±0) 5613 (±0) 0 (±0) 0 (±0) 58 (±0)
Java 25 5671 (±0) 5613 (±0) 0 (±0) 0 (±0) 58 (±0)
jcstress 32 (±0) 32 (±0) 0 (±0) 0 (±0) 0 (±0)
Total 22716 (±0) 22485 (±0) 0 (±0) 0 (±0) 231 (±0)

Code Coverage (Java 25)

Metric Covered Missed Coverage vs Master
Lines 28698 3126 90.2% ±0.0%
Branches 8331 1511 84.6% ±0.0%
Methods 7681 1224 86.3% ±0.0%

Changed Class Coverage (2 classes)

Class Line Branch Method
g.e.i.d.ExhaustedDataLoaderDispatchStrategy -1.2% 🔴 -7.7% 🔴 ±0.0%
g.e.i.d.PerLevelDataLoaderDispatchStrategy +0.7% 🟢 +2.0% 🟢 ±0.0%

Full HTML report: build artifact jacoco-html-report

Updated: 2026-03-07 07:34:24 UTC

@dondonz dondonz merged commit ef5b6f8 into master Mar 7, 2026
9 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bbakerman bbakerman bbakerman approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

25.1 bugfix & non-breaking changes

Development

Successfully merging this pull request may close these issues.

2 participants

@dondonz @bbakerman