Deep Learning

Mixture of Experts

An architecture that activates selected specialist networks for each input.

A mixture-of-experts model contains several specialised neural components called experts. A routing network decides which experts should process each token or example.

Only a small subset of experts is activated at one time, allowing the model to contain many parameters without using all of them for every input. Efficient routing and balanced expert usage are important challenges.

Also in Deep Learning

CNN RNN LSTM Transformers Auto Encoders Deep RL

JOIN NOW

Begin the first module

It is free, it is the real curriculum, and if it is not for you, you have lost nothing but an evening.

See the curriculum

Join any time · Build AI skills at your pace