Pinned Loading

1. fastertransformer_backend fastertransformer_backend Public

   Forked from triton-inference-server/fastertransformer_backend

   Python

2. onnxruntime onnxruntime Public

   Forked from microsoft/onnxruntime

   ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

   C++

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 94.1k 23.5k

4. huggingface/text-embeddings-inference huggingface/text-embeddings-inference Public

   A blazing fast inference solution for text embeddings models

   Rust 5.1k 462

5. vllm-project/llm-compressor vllm-project/llm-compressor Public

   State-of-the-art LLM compression, built for production inference with vLLM

   Python 3.9k 698