jingyaogong/minimind
MiniMind: Train a 64M LLM from Scratch in 2 Hours
🧠Train a 64M-parameter LLM from scratch in just 2h!
AI-generated summary
MiniMind is an open-source project that allows developers to train a 64 million parameter Large Language Model (LLM) from scratch with minimal cost and time. It provides a complete training pipeline, including code for data cleaning, pre-training, fine-tuning, and reinforcement learning techniques. The project aims to lower the barrier to entry for understanding and building LLMs.
Language: Python