avifenesh/

bw24

GitHub Website

From-scratch Rust+CUDA inference engine, bit-exact by construction — NVFP4, MoE, MTP speculative decoding, tuned against measured limits of one RTX 5090 Laptop (sm_120a).

260stars

31forks

4issues

MITlicense

2026since

Star historydaily snapshots by VibeCrowd

Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).

Alternatives & relatedmatched by topic overlap

vllm-project/

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

86k19kPython

sgl-project/

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

30k7.1kPython

flashinfer-ai/

flashinfer

FlashInfer: Kernel Library for LLM Serving

5.9k1.1kPython

Michael-A-Kuykendall/

shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

5.6k542Rust

AlexsJones/

llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

29k1.8kRust

gpustack/

gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

5.3k566Python

Comparebw24 vs vllm bw24 vs sglang bw24 vs flashinfer

SharePost on X LinkedIn