LSTM

In the previous post, we talked about RNN, and how performing Backpropagation through time (BPTT) on an unrolled RNN with many time steps can lead to the problems of vanishing / exploding gradients, and difficulties in learning long term dependencies. In this post, we’re going to look at a the LSTM (Long Short Term Memory) […]

RNN and Vanishing/Exploding Gradients

In this post, we’re going to be looking at: Recurrent Neural Networks (RNN) Weight updates in an RNN Unrolling an RNN Vanishing/Exploding Gradient Problem Recurrent Neural Networks A Recurrent Neural Network (RNN) is a variant of neural networks, where in each neuron, the outputs cycle back to themselves, hence being recurrent. This means that each […]