AI glossary / How it works

Neural network

How it works, one of the 17 words to learn first

Software loosely inspired by the brain: layers of simple connected units that strengthen or weaken their links as they learn.

More words in How it works

• OverfittingWhen a model memorises its practice examples so closely that it does badly on anything new.
• ParametersThe millions or billions of internal dials a model adjusts as it learns. More can mean more capable, but not always.
• Post-trainingEverything done to a model after its main training to make it helpful and safe, including instruction tuning and RLHF.
• Pre-trainingThe first and most expensive stage of building a model, where it reads enormous amounts of text to learn how language and the world work.
• QuantizationShrinking a model by storing its numbers less precisely, so it runs on smaller devices with a small loss in quality.