Contents
Why is RNN used for machine translation?
The loop makes the neural network go back and check what happened in all of the previous words before deciding what the current word actually means. A RNN can be thought of as copy-pasting the same network over and over again, with each new copy-paste adding a bit more information than the previous one.
Which type of RNN can be used for machine translation?
The simplest idea is to use a basic RNN such as the following diagram below. In this diagram the RNN is unrolled to make it easier to understand whats happening. This type of RNN is a sequence to sequence RNN (seq2seq).
Why might RNN based models have difficulty translating long sentences?
Training on long sentences is difficult because few available training corpora include sufficiently many long sentences, and because the computational over- head of each update iteration in training is linearly correlated with the length of training sentences.
How can RNNs be used for machine translation?
Since we have a basic understanding of LSTMs and RNNs, let’s try and apply some of this into developing a machine translation model using Keras. This will demonstrate how a sequence to sequence LSTM network can be used to translate text from English to French. Data can be found here.
How are sequence models used in machine translation?
Sequence models are a special class of deep neural networks that have applications in machine translation, speech recognition, image captioning, music generation, etc. Sequence problems can be of varying types where the input X and output Y might both be sequences with either the same length or different lengths.
How are the different types of RNNs expressed?
Different types of RNNs are usually expressed using the following diagrams: As discussed in the Learn article on Neural Networks, an activation function determines whether a neuron should be activated. The nonlinear functions typically convert the output of a given neuron to a value between 0 and 1 or -1 and 1.
Is the output of a recurrent neural network independent?
While traditional deep neural networks assume that inputs and outputs are independent of each other, the output of recurrent neural networks depend on the prior elements within the sequence.