# karpathy/nanochat

nanochat: Train and interact with a GPT\-2 quality LLM for under $100

- HackerByte page: <https://hackerbyte.net/en/projects/nanochat-train-and-interact-with-a-gpt-2-quality-llm-for-under-100-1075431749>
- GitHub: <https://github.com/karpathy/nanochat>
- Programming language: Python

The best ChatGPT that $100 can buy.

## AI-generated summary

nanochat is an experimental, minimalist Python framework for training Large Language Models \(LLMs\) on a single GPU node. It covers the entire LLM lifecycle, from tokenization and pretraining to fine\-tuning, evaluation, and inference. The project aims to make training models with GPT\-2 capabilities accessible, costing as little as $48 on cloud instances.

## Key points

- Enables training a GPT\-2 capability LLM for under $100, significantly reducing the cost compared to previous efforts.
- Features a 'Time\-to\-GPT\-2 Leaderboard' tracking community progress in achieving GPT\-2 grade capability.
- Offers a single, hackable codebase designed for ease of understanding, modification, and forking.
- Supports training on various hardware including NVIDIA GPUs \(A100/H100\) and Apple Silicon \(MPS\), with explicit precision management.
- Provides a CLI to chat with the trained models, demonstrating their capabilities.