Deep Learning

Mixture of Experts

An architecture that activates selected specialist networks for each input.

A mixture-of-experts model contains several specialised neural components called experts. A routing network decides which experts should process each token or example.

Only a small subset of experts is activated at one time, allowing the model to contain many parameters without using all of them for every input. Efficient routing and balanced expert usage are important challenges.

Also in Deep Learning

CNN RNN LSTM Transformers Auto Encoders Deep RL

Apply

Begin the first module

Become AI native, it is the real deal today, and if it is not for you, you have lost nothing but learnt a new skill.

Apply