强化学习训练中报错setting an array element with a sequence

Question

在强化学习训练中出现错误，请问如何解决？

e:\DQN_3D\Env3D.py:38: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  self.window = np.array([self.x0,self.y0,self.f0,self.rad0]).reshape(1,4)
e:\DQN_3D\Env3D.py:134: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 
'dtype=object' when creating the ndarray

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "e:/DQN_3D/TrainLibrary3D.py", line 126, in <module>
    train(rssRow)
  File "e:/DQN_3D/TrainLibrary3D.py", line 39, in train
    RL.learn()
  File "e:\DQN_3D\DeepQ.py", line 187, in learn
    self.learning_rate_step:self.learn_step_counter,
  File "D:\anaconda\Anaconda\envs\TF2.1\lib\site-packages\tensorflow\python\client\session.py", line 958, in run
    run_metadata_ptr)
  File "D:\anaconda\Anaconda\envs\TF2.1\lib\site-packages\tensorflow\python\client\session.py", line 1150, in _run
    np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
  File "D:\anaconda\Anaconda\envs\TF2.1\lib\site-packages\numpy\core\_asarray.py", line 83, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.

主要是函数learn这一块的问题：

self.sess.run(
            [self._train_op, self.loss],
            feed_dict={    
                self.s: batch_memory[:, :self.n_features],
                self.a: batch_memory[:, self.n_features],
                self.r: batch_memory[:, self.n_features + 1],
                self.s_: batch_memory[:, -self.n_features:],
                self.learning_rate_step:self.learn_step_counter,
            })

Answer 1

这个错误通常出现在使用numpy数组时，尝试将一个序列赋值给数组的单个元素，但数组的元素只能是标量。请检查你的代码，确保在向numpy数组中赋值时，将标量（单个值）赋值给单个元素。

另外，请注意，如果你在尝试将序列赋值给数组的单个元素时出现此错误，它可能意味着序列的维数与数组的维数不匹配。在这种情况下，请确保将序列的维度与数组的维度匹配。