omgitsbase/llmash

llmash a replacement for ollama. llmash uses advanced kernel optimizations to bring it to speeds far past vllm, on any model. Moving from ollama to llmash is nearly automatic, and the speed improvements are nearly (7% under) linear to power consumption improvements.

View on GitHub
C++
Stars 11
Forks 0
License MIT
Open Issues 1
Updated 1d ago