ashhart/TensorFold
TensorFold: High-Performance LLM Inference Engine for Metal, CUDA, and Vulkan
LLM Inference Engine for Metal, CUDA and Vulkan.
AI-generated summary
TensorFold is a high-performance LLM inference engine written in Zig, designed to run on Apple Silicon (Metal) and NVIDIA GPUs (CUDA, Vulkan). It allows serving language models directly from a Zig binary without requiring Python or MLX installations, offering a streamlined experience for developers.
Language: Zig