Atari Performance for PPO2

Describe the bug PPO2 have bad performance on Atari games. For example, it does not converge on pong.

Code example Using https://github.com/araffin/rl-baselines-zoo:

python train.py --algo ppo2 --env PongNoFrameskip-v4

System Info Describe the characteristic of your environment: * Stable-Baselines v2.1.2 * GPU * Python 3.6 * Tensorflow 1.8

该提问来源于开源项目：hill-a/stable-baselines