Task definitions for coder_eval. Each *.yaml is one task (see docs/TASK_DEFINITION_GUIDE.md for the schema). Files are organized by purpose, not just dumped flat — this README is the map. New here? The curated examples in samples/ are the best place to start.

Every task file must carry at least one tags: entry (enforced by tests/test_tags.py). Tags are how the runner selects subsets, e.g. coder-eval run tasks/*.yaml --tags smoke-pass.

Curated, runnable real-world tasks — a genuine evaluation rather than a toy, and the best first stop. Each is self-contained and documented in its own samples/README.md.

Minimal tasks that each demonstrate one framework feature; referenced from the docs.

Per-agent capability probes (hello-world, streaming, sub-agents, parallel tool calls, skills discovery, disallowed-tools). Grouped by the claude_ / codex_ / antigravity_ / subagent_ prefix. Several need a specific backend or extra (--type codex, GEMINI_API_KEY, etc.), so they are not in the CI smoke buckets — run them individually or with a -e experiment.

Tiny tasks the CI and make targets run to prove the framework itself works. These stay at the root because CI and the Makefile select them with the single-level glob tasks/*.yaml --tags <bucket> — moving them into a subdir would drop them from those runs.

Members: hello_date, agentless_smoke_test, byod_smoke_test, dataset_example, smoke_agent_judge, smoke_llm_judge, smoke_negative_path, smoke_budget_exceeded, smoke_cost_budget_exceeded, smoke_task_timeout, smoke_variants, token_check.