AI glossary / How it works

Mixture of experts

How it works, also called MoE

A model design with many specialist parts, where only the relevant few switch on for each question. It saves computing power.

More words in How it works

• Model collapseWhat can happen when AI is trained on too much AI-made content: quality and variety gradually decline.
• Multi-agent systemSeveral AI agents working together, each with its own job, such as one researching, one writing and one checking.
• Neural networkSoftware loosely inspired by the brain: layers of simple connected units that strengthen or weaken their links as they learn.
• OverfittingWhen a model memorises its practice examples so closely that it does badly on anything new.
• ParametersThe millions or billions of internal dials a model adjusts as it learns. More can mean more capable, but not always.