← World of AI
Deep Learning
Attention
A mechanism that lets models focus on the most relevant parts of an input.
Attention calculates how strongly one element should consider other elements when producing a representation. A model can therefore emphasise relevant words, image regions or features instead of treating every part equally.
The mechanism improved translation and sequence modelling by helping networks handle long-range dependencies. It later became the foundation of transformer architectures.
Also in Deep Learning
JOIN NOW
Begin the first module
It is free, it is the real curriculum, and if it is not for you, you have lost nothing but an evening.
Join any time · Build AI skills at your pace