flashinfer-ai/

flashinfer

FlashInfer: Kernel Library for LLM Serving

5.9kstars

1.1kforks

771issues

Apache-2.0license

2023since

Star historydaily snapshots by VibeCrowd

Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).

Alternatives & relatedmatched by topic overlap

vllm-project/

A high-throughput and memory-efficient inference and serving engine for LLMs

86k19kPython

sgl-project/

SGLang is a high-performance serving framework for large language models and multimodal models.

30k7.1kPython

gpustack/

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

5.3k566Python

beam-cloud/

Ultrafast serverless GPU inference, sandboxes, and background jobs

1.7k145Go

ray-project/

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

43k7.8kPython

superduper-io/

Superduper: End-to-end framework for building custom AI applications and agents.

5.3k542Python