# ggml\-org/llama.cpp

llama.cpp: C/C\+\+ LLM Inference Engine

- HackerByte page: <https://hackerbyte.net/en/projects/llama-cpp-c-c-llm-inference-engine-612354784>
- GitHub: <https://github.com/ggml-org/llama.cpp>
- Programming language: C\+\+

LLM inference in C/C\+\+

## AI-generated summary

llama.cpp is a high\-performance inference engine for Large Language Models \(LLMs\) and Vision\-Language Models \(VLMs\) written in plain C/C\+\+. It aims for minimal setup and runs efficiently across a wide range of hardware, including Apple Silicon, x86 architectures with AVX support, and various GPUs.

## Key points

- Plain C/C\+\+ implementation with no external dependencies.
- Optimized for Apple Silicon via ARM NEON, Accelerate, and Metal.
- Supports various quantization methods \(1.5\-bit to 8\-bit\) for faster inference and reduced memory usage.
- Offers custom CUDA kernels for NVIDIA GPUs, and support for AMD GPUs via HIP and Moore Threads GPUs via MUSA.
- Includes a built\-in OpenAI\-compatible API server and a command\-line interface.