AI glossary / How it works

Distillation

How it works

Training a small, cheap model to copy a big, expensive one so you get most of the skill at a fraction of the cost.

More words in How it works

• EmbeddingA way of turning words or images into lists of numbers so a computer can tell which things have similar meanings.
• Emergent behaviourAbilities that show up in large models without anyone specifically training for them.
• Fine-tuningTaking a ready-made AI model and giving it extra training on specific material so it gets good at one particular job.
• GANGenerative adversarial network. An older way of making fake images, where one network creates and another tries to catch it out, so both improve.
• GPTGenerative pre-trained transformer. The type of model behind ChatGPT: it generates text, was trained in advance on lots of writing, and uses the transformer design.