I ran 120B MoE models on my 32GB M1 Max using under 16 GB
A user shares running 120B parameter MoE models on a 32GB M1 Max with low memory usage (<16GB).
Hacker News submitter: tarassenkov
Submitted to Hacker News:
A user shares running 120B parameter MoE models on a 32GB M1 Max with low memory usage (<16GB).
Hacker News submitter: tarassenkov
Submitted to Hacker News: