Skip to content

About

A high-throughput and memory-efficient inference and serving engine for LLMs

Resources

Code of conduct

Contributing

Security policy

Stars

123 stars

Watchers

8 watching

Forks

Releases

Packages

Contributors

Languages