Add GPU-backed BiRefNet General tracer#38
Conversation
| rembg>=2.0.50 # birefnet-lite, isnet, paper detection | ||
| onnxruntime>=1.17.0 # rembg runtime | ||
| rembg>=2.0.75 # birefnet, isnet, paper detection | ||
| onnxruntime-gpu[cuda,cudnn]>=1.23.0 # rembg runtime with CUDA provider |
There was a problem hiding this comment.
Don't make GPU the default. onnxruntime-gpu[cuda,cudnn] only ships wheels for Linux x86_64 / Windows, breaks pip install on Mac, and pulls a few GB of NVIDIA libs even when no GPU. I reckon GPU users can opt in via a separate requirements-gpu.txt (or extras)
|
Sure, I’ll update and resubmit. On 5 May 2026, at 6:25 am, Jason Madigan ***@***.***> wrote:
@jasonmadigan commented on this pull request.
In backend/requirements.txt:
@@ -16,7 +16,7 @@ Pillow>=10.0.0
pillow-heif>=0.16.0
# local model support
transparent-background>=1.3.0 # inspyrenet
-rembg>=2.0.50 # birefnet-lite, isnet, paper detection
-onnxruntime>=1.17.0 # rembg runtime
+rembg>=2.0.75 # birefnet, isnet, paper detection
+onnxruntime-gpu[cuda,cudnn]>=1.23.0 # rembg runtime with CUDA provider
Don't make GPU the default. onnxruntime-gpu[cuda,cudnn] only ships wheels for Linux x86_64 / Windows, breaks pip install on Mac, and pulls a few GB of NVIDIA libs even when no GPU. I reckon GPU users can opt in via a separate requirements-gpu.txt (or extras)
In backend/app/config.py:
@@ -33,7 +33,7 @@ def available_tracers(self) -> list[str]:
# auto-detect
if self.google_api_key or self.openrouter_api_key:
return ["gemini"]
- return ["isnet", "birefnet-lite", "inspyrenet"]
+ return ["isnet", "birefnet-lite", "inspyrenet", "birefnet-general"]
related:
If GPU is opt-in, birefnet-general shouldn't be in the auto-detected default. Belongs behind opt-in (TRACERS=...,birefnet-general or only when a GPU runtime is installed)
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>
|
|
Updated the change to make gpu usage opt-in |
| logger.warning("failed to preload ONNX Runtime CUDA DLLs: %s", exc) | ||
|
|
||
|
|
||
| def get_onnx_providers(require_gpu: bool = False): |
There was a problem hiding this comment.
require_gpu is never passed True by any caller -- ai_tracer.py and image_processor.py both call get_onnx_providers() with no args. Either wire it up for birefnet-general loads where GPU is expected, or remove it.
| @@ -0,0 +1,105 @@ | |||
| from __future__ import annotations | |||
There was a problem hiding this comment.
This file has zero test coverage. test_tracer_config.py only exercises Settings.available_tracers. At minimum, need tests for: invalid TRACEFINITY_ONNX_PROVIDER raises ValueError, cpu mode returns ["CPUExecutionProvider"], auto mode branches correctly.
| for _tid in settings.available_tracers: | ||
| _get_tracer(_tid) | ||
|
|
||
| polygon_scaler = PolygonScaler() |
There was a problem hiding this comment.
The eager preload loop is gone, so a broken tracer config (bad model name, missing weights) will only surface on first user request. Consider a lightweight validation step at startup -- e.g. check configured tracer names against _REMBG_MODELS keys without actually loading models.
| - cuda: require CUDAExecutionProvider | ||
| - cpu: CPUExecutionProvider only | ||
| """ | ||
| import onnxruntime as ort |
There was a problem hiding this comment.
_preload_onnxruntime_cuda_dlls above catches ImportError gracefully when onnxruntime is missing, but this bare import will crash. Should be consistent -- either both handle it or both let it propagate.
| @@ -151,12 +153,16 @@ def _load_local_model(self): | |||
| label = self._LOCAL_MODEL_LABELS.get(name, name) | |||
There was a problem hiding this comment.
nit: docstring on _load_local_model (line 149) says "at startup" but loading is now lazy (preload loop removed from routes.py).
|
thanks @noobydp just a few comments then I think we're in good shape to merge. |
|
all that stuff is pretty nit-picky tbh, just approving. thanks a lot @noobydp |
Add GPU-backed BiRefNet General tracer
2 participants
This PR adds BiRefNet General as a GPU-backed local tracing option so higher-quality background removal can run locally without forcing every model to load at startup. It also wires rembg/ONNX Runtime provider selection and Windows CUDA DLL discovery so NVIDIA setups can use CUDA when available while retaining CPU fallback.
Code and PR drafted by Codex
Summary
Verification
pm run build