How to improve generalization in deep reinforcement learning?
A very interesting paper called “ A Simple Randomization Technique for Generalization in Deep Reinforcement Learning ” presented a nice method to improve generalization over the standard regularization shown before. They suggest to add a convolutional layer just between the input image and the neural network policy, that transforms the input image.
Which is an example of a reinforcement learning environment?
An example of such an environment is CoinRun, introduced by OpenAI in the paper “Quantifying Generalization in Reinforcement Learning”. This environment can produce a large variety of levels with different layouts and visual appearance, and thus serves as a nice benchmark for generalization.
Is the concept of generalization in supervised learning hold?
Had that not been the case, the standard concept of generalization in supervised learning would not hold, and it would be difficult to justify our expectation that learning on the training set should yield good results on the test set as well.
What does generalization mean in the context of RL?
Several other papers suggest that RL policies can be brittle to very minor mismatches between the environment they learn on and the one they are expected to be deployed in, making adoption of RL in the real world very difficult. We would like our policies to Generalize as they do in supervised learning, but what does it mean in the context of RL?
Which is the main puzzle of deep learning?
Deep Learning stresses traditional approaches to generalization theory to the extreme: a main puzzle of deep networks revolves around the absence of overfitting despite large over-parameterization and despite the large capacity demonstrated by zero training error on randomly labeled data [1]. The overall idea of complexity is being revisited.
Which is the best technique for generalization in supervised learning?
Results were significantly better using the larger model. Regularization: the most common set of techniques used in supervised learning to improve generalization are things like L2 regularization, Dropout and Batch Normalization.
Which is better for generalization larger or smaller neural networks?
Size of neural network: another finding from the paper that resonates with current practices in supervised learning, is that larger neural networks can often attain better generalization performance than smaller ones.