Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each

NVIDIA explains how mixture-of-experts models like Nemotron 3.5 Lightning activate only 3B parameters despite having 30B total parameters.

How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the...