SegMoE: Segmind Mixture of Diffusion Experts

Hugging Face Blog · 2 years ago

Segmind released SegMoE, a framework that creates Mixture-of-Experts diffusion models by combining multiple expert models through a router network that selectively activates them during inference. Three pre-built models are available on Hugging Face (SegMoE-2x1, SegMoE-4x2, and SegMoE-SD-4x2), with SegMoE-4x2 requiring 24GB of VRAM in half-precision. Users can now create custom MoE diffusion models by configuring a YAML file specifying base and expert models, though inference speed decreases when multiple experts process tokens simultaneously.