← World of AI
Deep Learning
Self-Attention
Attention that models relationships between elements within the same sequence.
Self-attention allows every token or element in a sequence to compare itself with all the others. These comparisons determine which elements are most relevant to its meaning.
Because relationships can be modelled directly rather than through repeated recurrent steps, self-attention supports parallel computation. It is the central operation behind modern transformers and large language models.
Also in Deep Learning
JOIN NOW
Begin the first module
It is free, it is the real curriculum, and if it is not for you, you have lost nothing but an evening.
Join any time · Build AI skills at your pace