High-throughput LLM serving framework for production.
Free / Open Source
Not sure if vLLM is right for you?