vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

View on GitHub
Python
Stars 97.0k
Forks 24.9k
License Apache-2.0
Open Issues 10436
Updated 13m ago