Contents
What is a feasible sequence length for an RNN to model?
It totally depends on the nature of your data and the inner correlations, there is no rule of thumb. However, given that you have a large amount of data a 2-layer LSTM can model a large body of time series problems / benchmarks. Furthermore, you don’t backpropagate-through-time to the whole series but usually to (200-300) last steps.
What’s the goal of a RNN and a LSTM?
The goal of any RNN (LSTM/GRU) is to be able to encode the entire sequence into a final hidden state which it can then pass on to the next layer.
Why do you need a larger sample size for CNN?
The size of the data is not very important. What is important is the variations in the samples you have. That is why data augmentation increases the accuracy and the generalization of a CNN model. However, you need to have additional training data if your model is over-fitting.
Is there a minimum or maximum sample size?
Hence, there is no minimum or maximum for training sample size. Generally the more sample for training can ensure better performance of system but make sure dont get over trained your network.
What’s the average batch size of a RNN?
Since you now have a basic idea, let’s break down the execution process with an example. Say your batch size is 6, RNN size is 7, the number of time steps/segments you would include in one input line is 5 and the number of features in one time step is 3.
When do you feed a batch of data into the RNN cell?
When you feed a batch of data into the RNN cell it starts the processing from the 1st line of input. Likewise, the RNN cell will sequentially process all the input lines in the batch of data that was fed and give one output at the end which includes all the outputs of all the input lines.
How to train RNN with examples of different lengths in keras?
In the keras documentation, it says the input to an RNN layer must have shape (batch_size, timesteps, input_dim). This suggests that all the training examples have a fixed sequence length, namely timesteps. But this is not especially typical, is it?