HackerByte

jingyaogong/minimind

MiniMind: Train a 64M LLM from Scratch in 2 Hours

🧠 Train a 64M-parameter LLM from scratch in just 2h!

AI-generated summary

MiniMind is an open-source project that allows developers to train a 64 million parameter Large Language Model (LLM) from scratch with minimal cost and time. It provides a complete training pipeline, including code for data cleaning, pre-training, fine-tuning, and reinforcement learning techniques. The project aims to lower the barrier to entry for understanding and building LLMs.

Language: Python

View the original GitHub repository