In this blog, we will see what is Recurrent Neural Network and how to implement its forward propagation from scratch using Python and numpy i.e, without using libraries like tensorflow, keras etc.
As we are implementing this machine learning development technique from scratch, you can also change the structure of vanilla RNN which we will implement in this blog.
Recurrent neural network is a sequence to sequence model i.e, output of the next is dependent on previous input. RNNs are extensively used for data along with the sequential structure. Whenever, the semantics of the data are changed, via any arbitrary permutation, the developers will get a sequential dataset. Additionally, Recurrent Neural Network is often used to solve the issues.
Some of the common applications of RNN are Speech to text, Music generation etc.
Let’s see the structure of RNN first. It is the network of neural network you can say.
RNN has a major role to play in various fields. They are becoming quite popular in the deep learning space environment. Therefore, RNN seems to have a decent demand in the coming future. Composition of music, Speech Recognition, Translation via Machines, Recognizing the Handwriting, For Learning Grammar etc. are a few of the real-world apps of RNN.
Using the above figure and its terminologies we will write equations of vanilla RNN.
Let’s consider initial input as
Structure of RNN
So, as you can interpret from above equations that o/p i.e, value of y is indirectly dependent on previous x.
Here g1 : activation function used for computing hidden neuron. Mostly it’s tanh
Here g2 : activation function used for computing output neuron. Mostly it’s softmax.
Implementation in Python
In this blog, we have seen what is Recurrent Neural Network and implement its forward propagation. The main aim of the article was to help you easily implement RNN from the start. You would be able to implement it from scratch. Although, once you will learn the process, you would be able to do it quite easily, without any hassle. Therefore, the information mentioned above will help you to enhance your RNN related skills.
There are more lot to go in its implementation too and this is the vanilla implementation of sequence to sequence model.
Read More :