Investigate other salient object detection models

InSPyReNet (MIT licence) is an excellent salient object detection model supported by Tracefinity. It's been out for some time now (ACCV 2022), so it's worth reviewing the current state of the art to see if we want to support additional models, giving users a choice between speed/portability and maximum quality.

Current model

• InSPyReNet - MIT licence, lightweight, runs well on CPU. Our current default via transparent-background pip package. Still actively maintained (latest release May 2025). Supports MPS on Apple Silicon. For a local-first tool like Tracefinity where users may not have a GPU, this remains the strongest default.

Candidates worth evaluating

BiRefNet (Bilateral Reference Network)

• Licence: MIT
• Repo: https://github.com/ZhengPeng7/BiRefNet
• Why it's interesting: Current state of the art for mask quality. Uses a dual-pathway architecture (global context + local detail) for edge-accurate mask generation. Multiple variants: general, high-res (2048x2048), lite, portrait, matting, dynamic-resolution. Very actively maintained. Cloudflare chose BiRefNet for their production background removal after benchmarking against U2-Net, IS-Net, and SAM.
• Tradeoffs: Significantly heavier than InSPyReNet (~973MB for SwinL backbone). Even the ONNX version takes ~165ms on an A100 GPU. CPU inference would be very slow -- users on the original GitHub repo reported it as "extremely slow" compared to lighter models. The BiRefNet-lite (SwinT backbone) variant helps (~93.8ms on A100 via ONNX) but is still not in the same league as InSPyReNet for CPU use.
• Best for: Users with a GPU who want maximum quality, particularly for complex backgrounds, fine hair detail, and high-resolution images.

BEN2 (Background Erase Network v2)

• Licence: MIT (base model only -- the refiner network is commercial/API-only)
• Repo: https://github.com/PramaLLC/BEN2
• Why it's interesting: Good results for hair matting and 4K processing. Trained on DIS5k + 22K proprietary dataset.
• Tradeoffs: The best quality requires the refiner, which is not open source. Base model alone may not justify the switch from BiRefNet. Less of a community/ecosystem around it compared to BiRefNet.
• Best for: Possibly worth benchmarking, but BiRefNet likely wins on quality at the same licence terms.

U2-Net

• Licence: Not explicitly stated in the original repo -- needs verification before inclusion
• Repo: https://github.com/xuebinqin/U-2-Net
• Why it's interesting: Extremely lightweight. The full model is 176.3MB, but the small variant (U2-Net†) is only 4.7MB and still achieves competitive results. Runs at 30-40 FPS on a GTX 1080Ti. No pretrained backbone dependency. Won the 2020 Pattern Recognition Best Paper Award.
• Tradeoffs: Older (2020), lower accuracy than InSPyReNet or BiRefNet on current benchmarks. Licence ambiguity is a blocker until clarified.
• Best for: Could be interesting as an ultra-lightweight option if licence is confirmed permissive.

Ruled out

RMBG 2.0 (BRIA AI)

• Licence: CC BY-NC 4.0 (commercial use requires paid agreement with BRIA)
• Repo: https://github.com/Bria-AI/RMBG-2.0
• Built on BiRefNet architecture with proprietary fine-tuning. Claims superior results to vanilla BiRefNet. However, the non-commercial licence makes this unsuitable for Tracefinity.

Samba / Samba+ (CVPR 2025)

• Licence: Unclear/research
• Repo: https://github.com/Jia-hao999/Samba
• Pure Mamba-based (state space model) architecture. Interesting research direction but not a practical drop-in replacement at this stage. Worth watching for the future.

Recommendation

InSPyReNet should remain the default model. Its CPU performance, MIT licence, lightweight footprint, and Apple Silicon (MPS) support make it the best fit for Tracefinity's local-first approach.

BiRefNet is the strongest candidate for an optional "high quality" mode aimed at users with GPU access. Same MIT licence, clearly superior mask quality, very active development.

Suggested next steps:

1. Benchmark BiRefNet (general + lite variants) against InSPyReNet on our existing test images to quantify the quality gap
2. Measure BiRefNet CPU inference time to confirm it's impractical as a default (and quantify how bad it is)
3. If the quality difference justifies it, add BiRefNet as a selectable model option (e.g. --model inspyrenet vs --model birefnet) with InSPyReNet remaining the default
4. Investigate BiRefNet-lite specifically as a possible middle ground between quality and speed

References

• Cloudflare background removal evaluation: https://blog.cloudflare.com/background-removal/
• BiRefNet paper: https://arxiv.org/abs/2401.03407
• BiRefNet inference speed discussion: inference speed extremely slow ZhengPeng7/BiRefNet#26
• BRIA RMBG 2.0 benchmark: https://blog.bria.ai/benchmarking-blog/brias-new-state-of-the-art-remove-background-2.0-outperforms-the-competition
• InSPyReNet transparent-background package: https://pypi.org/project/transparent-background/