Standalone C++ inference engine for SAM-3D-Body — zero Python dependency at runtime.

Takes a BGR image and produces per-person MHR body pose parameters, camera translation, and optionally full 3D mesh vertices + 70 body/hand keypoints, all via ONNX Runtime + ggml.

Also includes Python frontends that call the compiled shared library via ctypes, and a CSV exporter for the 70 MHR keypoints.

--bvh PATH writes a standard BVH motion-capture file per detected person (p_0.bvh, p_1.bvh, …). Identities are kept stable across frames by a built-in 2D-bbox IoU tracker, each file's joint OFFSETs are auto-resized to the actor's measured bone lengths, and the output drops straight into Blender / BVHTester / any DCC. A bundled blender/blender_bvh_plugin.py drives a MakeHuman-rigged character from the result. See BVH export for details.

./fast_sam_3dbody_run --from clip.mp4 --bvh ./p.bvh --headless
# → p_0.bvh, p_1.bvh, …

Pre-built ONNX / GGUF / LBS model files are hosted on HuggingFace:

https://huggingface.co/AmmarkoV/SAM3DBody-cpp-onnx-models

Download SAM3DBody-cpp-onnx-models.zip, extract it, and place the resulting onnx/ directory at the root of this repository:

# Download and extract
wget https://huggingface.co/AmmarkoV/SAM3DBody-cpp-onnx-models/resolve/main/SAM3DBody-cpp-onnx-models.zip
unzip SAM3DBody-cpp-onnx-models.zip
# onnx/ is now at the repo root — ready to build

CMake will warn at configure time if neither onnx/ nor the zip is found.

BGR image
  │
  ▼  yolo.onnx            ONNX Runtime (CUDA EP)   person bboxes + 17 COCO keypoints
  │
  ▼  backbone.onnx        ONNX Runtime (CUDA EP)   feature map  [B, 1280, 32, 32]
  │   DINOv3-ViT-H/14+
  │
  ▼  decoder.onnx         ONNX Runtime (CUDA EP)   pose token   [B, 1024]
  │   6-layer PromptableDecoder
  │
  ▼  pipeline.gguf        CPU matmul (ggml)         MHR params [B, 519] + camera [B, 3]
  │   MHR head + camera head weights
  │
  ▼  body_model.lbs       native C LBS (optional)   vertices [18439, 3] in metres
      extracted once by tools/extract_lbs_data.py

Note: body_model.onnx export is blocked on PyTorch ≥ 2.x (torch.export rejects TorchScript modules). The native C LBS path reads body_model.lbs directly and produces identical output to Python mhr_forward (body model stores data in cm; mhr_lbs_compute applies ×0.01 to match Python's /100 conversion).

Per-person output (MHRResult / FsbResult):

SAM3DBody-cpp/
├── CMakeLists.txt
├── body_mesh.tri                     SMPL-like body mesh for the GL renderer
├── fast_sam_3dbody_frontend.py       Python lightweight frontend (ctypes, no extra deps)
├── fast_sam_3dbody_frontend-3D.py    Python 3D frontend (ctypes + Python body model)
├── fast_sam_3dbody_dump_csv.py       Python CSV exporter – 70 MHR keypoints per frame
├── two_pass.py                       Second-pass temporal smoother
├── ros_demo_webcam.py                ROS demo
├── onnx/                             Runtime model files – download from HuggingFace (see above)
│   ├── backbone.onnx + .data         ~4.8 GB  DINOv3-ViT-H/14+ encoder
│   ├── decoder.onnx                  ~93 MB   6-layer PromptableDecoder
│   ├── pipeline.gguf                 ~5 MB    MHR + camera heads
│   ├── yolo.onnx                     ~81 MB   YOLO11m-pose
│   ├── body_model.lbs                ~27 MB   native C LBS data
│   ├── correctives.bin               ~33 MB   pose corrective blend shapes
│   └── keypoint_mapping.bin          ~8 KB    MHR-70 keypoint index map
├── GraphicsEngine/
│   ├── System/glx3.{h,c}            GLX window management
│   └── ModelLoader/                  .tri mesh loader + LBS joint transform
├── AmMatrix/                         Lightweight C matrix / quaternion library
├── render/
│   ├── fast_sam_3dbody_render.cpp    OpenGL mesh overlay renderer
│   └── mhr_pose_driver.h             LBS driver (camera matrices, vertex update)
├── scripts/
│   └── build.sh / setup.sh / webcam.sh / video.sh
└── src/
    ├── fast_sam_3dbody.h             C++ public API
    ├── fast_sam_3dbody.cpp           Pipeline implementation
    ├── fast_sam_3dbody_capi.h        Plain C API (for ctypes)
    ├── fast_sam_3dbody_capi.cpp
    ├── preprocess.hpp                Crop, normalise, ray_cond, NMS, pose conversion
    ├── bvh_writer.h / bvh_writer.cpp BVH motion-capture exporter (multi-person, name-mapped to MHR joints)
    ├── mhr_joint_table.h             Generated by scripts/build_joint_table.py — MHR joint names + parents
    └── main.cpp                      CLI executable (--bvh, --out CSV, live overlay window)

Developer tools (ONNX/GGUF export, LBS extraction, debug scripts, Python training env) live in the parent project: https://github.com/AmmarkoV/Fast-SAM-3D-Body

See the Models section above. After extracting the zip, onnx/ should be at the repo root.

To build models from source (requires the original Python training environment), see Fast-SAM-3D-Body.

Requirements: CMake ≥ 3.18, C++17 compiler, OpenCV (core/imgproc/videoio/highgui/dnn), optional CUDA Toolkit.

cd fast_sam_3dbody_cpp
mkdir -p build && cd build

cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

CMake handles dependencies automatically:

• ONNX Runtime 1.20.1 – downloaded from GitHub releases if not found; point to an existing install with -DONNX_RUNTIME_DIR=/path/to/onnxruntime
• ggml – fetched via FetchContent from GitHub
• CUDA – auto-detected; set -DCMAKE_CUDA_ARCHITECTURES=86 (or 75, 89, etc.) for your GPU; falls back to CPU-only if not found

Outputs in build/:

cd fast_sam_3dbody_cpp/build

# Single image – prints pose params to stdout
./fast_sam_3dbody_run \
    --onnx-dir ../onnx \
    --gguf     ../onnx/pipeline.gguf \
    --yolo     ../onnx/yolo.onnx \
    --from     ../../assets/teaser.png

# Webcam (device 0)
./fast_sam_3dbody_run \
    --onnx-dir ../onnx --gguf ../onnx/pipeline.gguf --yolo ../onnx/yolo.onnx \
    --from 0

# Video file
./fast_sam_3dbody_run \
    --onnx-dir ../onnx --gguf ../onnx/pipeline.gguf --yolo ../onnx/yolo.onnx \
    --from /path/to/video.mp4

# Fastest mode – skip LBS body model (no vertices, just pose params)
./fast_sam_3dbody_run ... --skip-body

# CPU-only
./fast_sam_3dbody_run ... --cuda -1

Full option list:

--onnx-dir PATH    Directory with backbone/decoder/body_model ONNX files
--gguf     PATH    pipeline.gguf (MHR + camera heads)
--yolo     PATH    YOLO pose model (.onnx)
--from     SRC     Webcam index (0,1,..) or path to image/video
-o / --out PATH    Write 70-joint 3D keypoints to CSV per frame
--bvh      PATH    Write BVH motion capture file(s) to PATH (see "BVH export" below)
--bvh-template P   BVH skeleton template (default: ./body.bvh)
--cuda     DEVICE  CUDA device index (default 0; -1 = CPU)
--skip-body        Skip body model (no vertices / keypoints)
--headless         No display window
--thresh   T       YOLO person confidence threshold (default 0.50)
--nms      T       YOLO NMS IoU threshold (default 0.45)
--fx / --fy F      Camera focal length x/y in pixels (0 = image width)
--cx / --cy F      Principal point (0 = image centre)
--render-size W H  Override display window size
--size W H         Webcam capture resolution
--fps Z            Webcam capture framerate
--butterworth      Apply Butterworth low-pass filter to MHR output vectors
--bw-cutoff HZ     Butterworth cutoff frequency in Hz (default 6.0)
--rot-clamp DEG    Max global_rot change per frame in degrees (default 15; 0=off)
--info             Print pipeline info and exit
--help             Show this message

Draws COCO 2D skeletons and a pose-bar panel. Requires only opencv-python and numpy — no PyTorch.

# From the repo root:
python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend.py --from assets/teaser.png

# Webcam, cap at 3 persons
python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend.py --from 0 --max-skeletons 3

# Save output image / video
python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend.py \
    --from assets/teaser.png --out out.jpg

python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend.py \
    --from video.mp4 --headless --out out.mp4

Key options (same as CLI, plus):

--max-skeletons N  Cap persons drawn per frame (0 = unlimited)
--headless         No display window
--out PATH         Write result to image or video file

Full 3D mesh rendering identical to demo_webcam.py: four-panel output [original | 2D skeleton | front mesh | side mesh].

Uses the C engine for the fast path (YOLO → backbone → decoder → MHR FFN heads), then calls the Python MHR body model (mhr_model.pt) for LBS skinning to produce mesh vertices. Requires the full Python environment (PyTorch, sam_3d_body package, pyrender).

python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend-3D.py --from assets/teaser.png

# Webcam
python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend-3D.py --from 0 --max-skeletons 3

# Custom checkpoint paths
python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend-3D.py \
    --from assets/teaser.png \
    --checkpoint ./checkpoints/sam-3d-body-dinov3/model.ckpt \
    --mhr-model  ./checkpoints/sam-3d-body-dinov3/assets/mhr_model.pt

# Save result
python fast_sam_3dbody_cpp/fast_sam_3dbody_frontend-3D.py \
    --from assets/teaser.png --out result_3d.jpg

Key options (same as lightweight frontend, plus):

--checkpoint PATH  Path to model.ckpt (default: checkpoints/sam-3d-body-dinov3/model.ckpt)
--mhr-model  PATH  Path to mhr_model.pt
--device     STR   PyTorch device for body model: cuda or cpu (default: auto)

The CLI can apply a second-order Butterworth low-pass filter to the MHR output vectors on every frame, reducing per-frame jitter without introducing ripple in the passband.

# Enable with the default 6 Hz cutoff
./fast_sam_3dbody_run --from video.mp4 --butterworth

# Lower cutoff for smoother (more lag) output
./fast_sam_3dbody_run --from video.mp4 --butterworth --bw-cutoff 3.0

# Higher cutoff to preserve faster motion
./fast_sam_3dbody_run --from video.mp4 --butterworth --bw-cutoff 10.0

The filter is applied in-place to each detected person's result immediately after inference, so all downstream consumers (CSV writer, BVH writer, display overlay) receive filtered data.

global_rot uses wrap-corrected frame rejection rather than Butterworth because Euler angles wrap at ±π — a Butterworth filter interpolates through the discontinuity and produces a visible flip. Clamping the delta only delays the flip; if the model keeps predicting the flipped orientation the output still slowly drifts there.

Instead, the per-frame wrapped delta is compared to --rot-clamp. If any component exceeds the threshold the frame is rejected and the previous value is held. Genuine rotation (small delta per frame) passes through unchanged; flips and ambiguous orientation jumps (large delta) are frozen out entirely.

The sampling rate is taken from --fps when specified, otherwise 30 Hz is assumed. Each person slot maintains its own independent filter bank; new person slots are initialised with a warm-up pass on their first frame so the filter starts from the measured value rather than zero.

Implemented in src/outputFiltering.h — a header-only, dependency-free, C-compatible Butterworth filter. Each channel is a ButterWorth struct initialised with initButterWorth(sensor, fs, fc) and stepped with filter(sensor, value).

Exports the per-frame MHR pose as one or more standard BVH motion-capture files. The hierarchy is taken from a BVH template (default ./body.bvh, a MocapNET/MakeHuman T-pose skeleton); the motion comes from the MHR pipeline.

# Single image / video / webcam → one or more <name>_<id>.bvh files
./fast_sam_3dbody_run --from boom.mp4 --bvh ./p.bvh --headless
# produces ./p_0.bvh, ./p_1.bvh, … (one per tracked person)

For each detected person, every frame:

• Root joint — translation from pred_cam_t (×100 to convert metres → cm) and orientation from MHR's global rotation, in the BVH root's Zrotation Yrotation Xrotation channel order.
• Body joints matched by name to MHR (~50 joints incl. spine, arms, legs, fingers — full table in src/bvh_writer.cpp NAME_MAP). The local rotation is computed in MHR's frame as inv(delta[parent]) · delta[self], where delta[j] = R_global_mhr[j] · R_global_mhr_rest[j]⁻¹, then decomposed to the joint's BVH channel order (Zrotation Xrotation Yrotation).
• Unmapped BVH joints (toes, face details, BVH metacarpals, etc.) stay at zero rotation — MHR doesn't predict angles for them.
• OFFSETs are rewritten at close-time to each person's median observed bone length so the template T-pose proportions match the actual subject. Bone direction is preserved (changing it disrupts the T-pose look the BVH file was authored with).

Multiple skeletons in the same scene are exported as separate BVH files, one per tracked identity. Identity persistence is handled by a built-in bbox-IoU greedy tracker:

• IoU threshold 0.10; tracks are retired after 90 frames missing (≈ 3 s at 30 fps).
• Detections with degenerate bboxes (anchored at (0, 0) or near-zero area) are dropped before they hit the tracker — these are common YOLO failure modes that otherwise would spawn spurious tracks.
• While a track is alive but missing this frame, its previous pose is duplicated so the BVH timeline stays continuous through brief occlusions.

Filenames are derived from --bvh PATH:

Each file is fully independent — drop into Blender / BVHTester / any DCC.

# Render a 3D-keypoints CSV at the same time, then compare hip-relative
# joint positions between the BVH and MHR's own 3D keypoints.
./fast_sam_3dbody_run --from boom.mp4 \
    --bvh ./p.bvh --out /tmp/boom_mhr.csv --headless

source venv/bin/activate
python3 scripts/verify_bvh_motion.py ./p_0.bvh /tmp/boom_mhr.csv

A clean run prints per-joint median / p90 / max error in cm; numbers around 2–5 cm on trunk joints and 5–15 cm on extremities are the expected range (the larger residual on hands/feet partly reflects MHR's 70-keypoint surface landmarks not coinciding with the LBS rotation-centre joints).

The MHR body model uses 127 named joints (body_world, root, l_uparm, r_thumb1, …) — names that don't appear in body_model.lbs but are exported from the JIT model:

# (Re)generate src/mhr_joint_table.h from a checkpoint
source venv/bin/activate
python3 scripts/build_joint_table.py
# pass MHR_MODEL_PT=/path/to/mhr_model.pt to override the default location

bvh_writer.cpp then matches each BVH joint name to an MHR name via the hand-authored NAME_MAP table. To support a new BVH template just add entries to that table (and rebuild).

• The BVH I/O comes from a vendored, trimmed-down copy of MotionCaptureLoader in GraphicsEngine/MotionCaptureLoader/ (plus TrajectoryParser/InputParser_C). We use bvh_loadBVH for parsing the template hierarchy and dumpBVHToBVH for serialising the result.
• The per-person motion buffer lives in std::vector<float> and is transplanted into mc->motionValues at close-time, then detached before bvh_free() so the library doesn't try to free our std::vector storage.
• If you only have one person in the scene you'll get exactly one file (<name>_0.bvh) — there is no single-file mode for backwards compatibility.

blender/blender_bvh_plugin.py is a Blender add-on (by AmmarkoV) that plays one of these BVHs onto a MakeHuman-rigged character via the mpfb / MakeHuman plugin for Blender. It maps the BVH joints onto the MakeHuman armature with copy-rotation bone constraints (body / hands / feet / face are independently selectable), and exposes a "MocapNET BVH Animation Helper" panel in the 3D viewport's N-panel.

End-to-end workflow:

# 1) Generate one BVH per detected person
./fast_sam_3dbody_run --from clip.mp4 --bvh ./p.bvh --headless
# → p_0.bvh, p_1.bvh, …

# 2) (One-time) install a known-good Blender + open the plugin
cd blender && ./downloadAndInstallBlender.sh
# downloads blender-3.4.1, launches it with the plugin loaded;
# on first run the plugin will offer to fetch + install the mpfb2 MakeHuman addon

Inside Blender:

1. Use the MakeHuman (mpfb) tab to create or load a skinned character.
2. File → Import → Motion Capture (.bvh) and pick one of the p_<id>.bvh files. The BVH skeleton appears as a separate armature.
3. Open the MocapNET BVH Animation Helper panel, point Source BVH at the imported armature, tick the body parts you want driven (body / hands / feet / face) and click Apply — the plugin adds copy-rotation constraints on every matching MakeHuman bone.
4. Scrub or render the timeline as usual; the character now follows the exported motion.

The plugin auto-handles naming variants between the BVH skeleton (body.bvh, which is what we export against) and the MakeHuman armature, so the p_<id>.bvh files coming out of --bvh work without any further renaming.

#include "fast_sam_3dbody.h"

fsb::PipelineConfig cfg;
cfg.onnx_dir        = "./onnx";
cfg.gguf_path       = "./onnx/pipeline.gguf";
cfg.yolo_path       = "./onnx/yolo.onnx";
cfg.cuda_device     = 0;       // -1 = CPU only
cfg.skip_body_model = true;    // faster: no vertices
cfg.max_persons     = 4;       // 0 = unlimited

fsb::Pipeline pipeline;
pipeline.load(cfg);

// BGR uint8 pointer, width, height
std::vector<fsb::MHRResult> results =
    pipeline.process_bgr(bgr_ptr, width, height);

for (const auto& r : results) {
    // r.bbox          [4]   x1 y1 x2 y2 (original image pixels)
    // r.global_rot    [3]   Euler ZYX global orientation
    // r.body_pose     [133] joint angles
    // r.shape         [45]  identity betas
    // r.pred_cam_t    [3]   raw camera head: [s, tx, ty]
    // r.focal_length        estimated focal length (pixels)
    // r.keypoints_yolo[51]  COCO 17 × [x,y,conf]
    // r.pred_vertices [18439*3]  (empty when skip_body_model=true)
}

pipeline.free();

#include "fast_sam_3dbody_capi.h"

FsbHandle h = fsb_create();

FsbConfig cfg = {
    .onnx_dir        = "./onnx",
    .gguf_path       = "./onnx/pipeline.gguf",
    .yolo_path       = "./onnx/yolo.onnx",
    .cuda_device     = 0,
    .skip_body_model = 1,
    .person_thresh   = 0.5f,
    .person_nms_iou  = 0.45f,
    .max_persons     = 0,
};
fsb_load(h, &cfg);

FsbResult results[32];
int n = fsb_process_bgr(h, bgr, width, height, results, 32);

for (int i = 0; i < n; i++) {
    // results[i].bbox, .body_pose, .yolo_kps, ...
}

fsb_destroy(h);

• Backbone is the bottleneck; it dominates end-to-end latency.
• Use --skip-body unless 3D vertices are required.
• For higher throughput, batch multiple crops in a single backbone forward pass (already done when multiple persons are detected).

The official PyTorch SAM 3D Body repository from Meta Superintelligence Labs is :

https://github.com/facebookresearch/sam-3d-body

If you use this software repository in your research or work, please cite:

@misc{qammaz2026sam3dbodycpp,
  author       = {Qammaz, Ammar},
  title        = {{SAM3DBody-cpp}: Standalone {C++} Inference Engine for {SAM-3D-Body}},
  year         = {2026},
  howpublished = {\url{https://github.com/AmmarkoV/SAM3DBody-cpp}},
  note         = {Zero-dependency runtime: ONNX Runtime + ggml, with BVH export and Python ctypes frontends}
}

as well as the Meta AI team behind the awesome paper that proposes the SAM 3D Body method.

@article{yang2026sam3dbody,
  title={SAM 3D Body: Robust Full-Body Human Mesh Recovery},
  author={Yang, Xitong and Kukreja, Devansh and Pinkus, Don and Sagar, Anushka and Fan, Taosha and Park, Jinhyung and Shin, Soyong and Cao, Jinkun and Liu, Jiawei and Ugrinovic, Nicolas and Feiszli, Matt and Malik, Jitendra and Dollar, Piotr and Kitani, Kris},
  journal={arXiv preprint arXiv:2602.15989},
  year={2026}
}