Contents
In general: The number of hidden layer neurons are 2/3 (or 70% to 90%) of the size of the input layer. The number of hidden layer neurons should be less than twice of the number of neurons in input layer. The size of the hidden layer neurons is between the input layer size and the output layer size.
Can a hidden layer neuron merge with an output neuron?
Fortunately, we are not required to add another hidden layer with a single neuron to do that job. The output layer neuron will do the task. Such neuron will merge the two lines generated previously so that there is only one output from the network.
One feasible network architecture is to build a second hidden layer with two hidden neurons. The first hidden neuron will connect the first two lines and the last hidden neuron will connect the last two lines. The result of the second hidden layer. The result of the second layer is shown in figure 9.
How to control the architecture of a neural network?
Artificial neural networks have two main hyperparameters that control the architecture or topology of the network: the number of layers and the number of nodes in each hidden layer. You must specify values for these parameters when configuring your network.
The size of the hidden layer is normally between the size of the input and output-.It should be should be 2/3 the size of the input layerplus the size of the o/p layer The number of hidden neurons should be less than twice the size of the input layer.
Which is the optimal size of the hidden layer?
There are some empirically-derived rules-of-thumb, of these, the most commonly relied on is ‘the optimal size of the hidden layer is usually between the size of the input and size of the output layers’. Jeff Heaton, author of Introduction to Neural Networks in Java offers a few more.
To demonstrate how this function works see the outputs below. Say we have 5 hidden layers, and the outermost layers have 50 nodes and 10 nodes respectively. Then the middle 3 layers should have 40, 30, and 20 nodes respectively, if we want a linear decrease in the number of nodes.
How to increase the number of hidden nodes?
Increase the hidden nodes number until you get a good performance. Only if not I would add further layers. Further, use cross validation and appropriate regularization. As Yoshua Bengio, Head of Montreal Institute for Learning Algorithms remarks:
Is it possible to keep the number of nodes constant?
It is also possible to keep the number of nodes constant (by keeping the number of nodes for the outermost layers equal). With this function, it is also possible to increase the number of nodes by keeping the number of nodes for the left outer hidden layer less than that of the right outer hidden layer, however, this is not advised.