#mixture of experts
5 articles tagged mixture of experts. At twelve articles this topic starts lifting the whole site in search.
Timeline
-
Inkling 2 Small — Mira Murati's 594B model jumps from 13 to 31 on the AI index, weights after safety tests
-
Mistral Large 4: a 1-trillion-parameter open-weight model, weights due late October
-
Reflection AI Beam: a 501B open-weight model aimed squarely at China's lead
-
What mixture of experts (MoE) is — how a 125B model runs on a gaming GPU
All articles
-
Tech · 3 min readInkling 2 Small — Mira Murati's 594B model jumps from 13 to 31 on the AI index, weights after safety tests
Model — 594B-parameter MoE (17B active), text and image input, adjustable reasoning
-
Tech · 3 min readMistral Large 4: a 1-trillion-parameter open-weight model, weights due late October
Scale — ~1T total, ~49B active parameters per token; trained on ~4,000 Grace Blackwell GPUs over two months
-
Tech · 2 min readReflection AI Beam: a 501B open-weight model aimed squarely at China's lead
Launch — October 5: 501B total / 23B active MoE, 1M-token context, 23.8T training tokens
-
Tech · 2 min readWhat mixture of experts (MoE) is — how a 125B model runs on a gaming GPU
Structure — many 'experts' plus a router that picks a few per token: big in total, small in compute
-
Tech · 4 min readDeepSeek V4-Pro launches — 1.6 trillion parameters, 49 billion switched on
V4-Pro-0813: 1.6tn total parameters, 49bn active per token, 1M-token context, 384K max output