Skip to content

feat(cookbooks): add live CrewAI e2e recipe through openinference adapter - #5

Merged
lostmartian merged 1 commit into
mainfrom
feat/crewai-e2e-cookbook
Aug 22, 2026
Merged

lostmartian merged 1 commit into
mainfrom
feat/crewai-e2e-cookbook

Conversation

@lostmartian

Copy link
Copy Markdown
Collaborator

What

Adds cookbooks/live_crewai.py: a live, end-to-end CrewAI example. It runs a real one-agent crew (custom BaseTool + gpt-4o-mini), instruments it with openinference-instrumentation-crewai + openinference-instrumentation-openai, and diffs two real crew executions through the existing openinference adapter — no manual trace reshaping.

Why

Roadmap item J3 — real end-to-end framework examples. CrewAI is the second most-adopted agent framework (5.2M downloads/mo per WS-A); this proves AgentDiff ingests its telemetry natively and gates regressions on real crew output (verified live).

How

  • Baseline: single NY lookup. Candidate: NY+CA+TX scope creep → 3 tool iterations.
  • Integration gotchas discovered and baked into the recipe:
    • Tool must be a BaseTool subclass overriding _run — @tool-decorated tools override run() and bypass the instrumentor's patch (no tool spans).
    • CrewAI 1.x dropped litellm; its LLM layer calls the OpenAI SDK directly, so OpenAIInstrumentor provides LLM spans.
    • CrewAI product telemetry / interactive trace prompt disabled (CREWAI_DISABLE_TELEMETRY, CREWAI_TRACING_ENABLED, tracing=False).
  • Live verification (2026-08-22): TDI 0.429, loop flag across 3 LLM+tool iterations, tokens +218%, gate BLOCKED on divergence AND loops. Evidence recorded in context/LIVE-VERIFICATION.md.
  • Updates cookbooks/README.md + CHANGELOG [Unreleased].

Testing

  • Ran the recipe live against the real API: gate blocked as designed
  • make lint -> pass; uv run python -m pytest -> 205 passed

Checklist

  • Exactly one logical change in this PR
  • make lint passes
  • uv run python -m pytest is green (205 passed)
  • Docs updated if user-facing
  • CHANGELOG.md entry added under [Unreleased]
  • Conventional title

@lostmartian
lostmartian merged commit a2e8d48 into main Aug 22, 2026
7 checks passed
@lostmartian
lostmartian deleted the feat/crewai-e2e-cookbook branch August 22, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant