feat: ForgeRL v2 M0–M1 vector-batch runtime and formal acceptance - #1

Summary

This draft PR implements ForgeRL v2 M0 and the complete M1 code path. PyTorch remains responsible for tensors, autograd, CUDA, AMP, DDP and future FSDP2; ForgeRL owns RL sampling, inference, experience and orchestration.

master remains untouched and this PR remains draft.

Formal M1 subject

The production local topology is now one consistent subject across throughput and learning evidence:

C++/vectorized EnvRunner
  -> one contiguous observation batch
  -> VectorBatchInferenceRuntime
  -> one policy forward
  -> checked per-Actor/per-environment output views

Timing-based rendezvous, event-driven shared memory and process-mailbox services remain regression or fallback paths. They are not used to prove the formal vectorized M1 topology.

Completed M1 implementation

• fixed transition semantics for terminated, truncated and infrastructure faults;
• lossless trajectory fragmentation and transition identity audits;
• bounded on-policy experience queue and versioned policy registry;
• atomic active/staging policy replicas;
• stable C ABI for vectorized C++ environments;
• producer-defined VectorBatchInferenceRuntime;
• identical vector-batch v2 subject in the PPO CartPole/Pendulum learning gate;
• C++ VectorEnv -> vector-batch -> TransitionBatch -> ExperienceQueue -> PPO update integration test;
• explicit FP32 tolerances for batched-versus-sliced checks on Python 3.11 and 3.12;
• controlled three-repeat vector-batch stability gate;
• pinned self-hosted formal workflow that combines throughput and five-seed learning evidence into m1-final.json.

Validation

The latest standard GitHub Actions run is green:

• Python 3.11 core;
• Python 3.12 core, 46 tests passed;
• C++ vector-environment closed-loop test;
• vector-batch PPO learning smoke for CartPole and Pendulum;
• two-process Gloo learner test;
• vector-batch stability screening with one warmup and three measured repeats.

Hosted screening reported stable vector-batch speedups well above the documented 2x threshold, but hosted results are not formal acceptance evidence.

Remaining formal execution gates

M1 is not described as accepted until the manual self-hosted workflow on a runner labeled forgerl-benchmark produces all three artifacts:

m1-learning-formal.json
m1-vector-batch-controlled.json
m1-final.json

The final artifact must have:

{"status": "GO"}

The formal gate requires five paired CartPole/Pendulum seeds, all target reaches, v2 time-to-target no worse than 1.05x v1, normalized AUC at least 0.95x v1, stable >=2x valid-row throughput, and all correctness audits passing.

Milestone order

M2 is frozen until M1 produces the persisted GO report, unless the user explicitly changes the milestone order. No merge or M1 acceptance claim is made by this update.