02 · Model Architectures
RNN
Recurrent Neural Network
A neural-network architecture for sequences that processes elements in order while carrying forward hidden state. RNNs can struggle with long-range dependencies and parallel training, limitations that helped motivate attention-based architectures.
Concrete example
Early phone autocomplete that often forgot the start of your sentence by the end.