Attention Mechanisms in Deep Neural Networks

Source: https://wsai.iitm.ac.in/themes/attention-mechanisms-in-deep-neural-networks/ Parent: https://wsai.iitm.ac.in/research/

• Home
• Research Themes
• Attention Mechanisms in Deep Neural Networks

Attention Mechanisms in Deep Neural Networks

Tags

neural networks reinforcement learning attention mechanism

Recently the Deep Learning community has shown great interest in attention mechanisms to train neural networks - the network pays attention to only certain parts of the input or to certain parts of the network structure to learn at a given instant. However, to make this work well, we need to develop efficient algorithms for jointly learning the network parameters as well as the attention mechanism. The main work in the proposal will be three-fold.

Better algorithms for attention, possibly based on reinforcement learning; transfer learning using attention; and more efficient implementations of attention mechanisms