AI glossary / How it works

Overfitting

How it works

When a model memorises its practice examples so closely that it does badly on anything new.

Think of it as: Memorising last year's exam answers instead of learning the subject.

More words in How it works

• ParametersThe millions or billions of internal dials a model adjusts as it learns. More can mean more capable, but not always.
• Post-trainingEverything done to a model after its main training to make it helpful and safe, including instruction tuning and RLHF.
• Pre-trainingThe first and most expensive stage of building a model, where it reads enormous amounts of text to learn how language and the world work.
• QuantizationShrinking a model by storing its numbers less precisely, so it runs on smaller devices with a small loss in quality.
• Reasoning modelA model built to think through a problem step by step before it answers. Slower, but stronger on maths, logic and planning.