HackerByte

I ran 120B MoE models on my 32GB M1 Max using under 16 GB

A user shares running 120B parameter MoE models on a 32GB M1 Max with low memory usage (<16GB).

Hacker News submitter: tarassenkov

Submitted to Hacker News:

Original source Hacker News discussion