openstack集群一次重启后cinder-volume服务一直是down的状态
[root@PNWC2-N1 ~]# openstack volume service list
+------------------+-----------------+------+---------+-------+----------------------------+
| Binary | Host | Zone | Status | State | Updated At |
+------------------+-----------------+------+---------+-------+----------------------------+
| cinder-scheduler | PNWC2-N3 | nova | enabled | up | 2021-03-18T03:25:11.000000 |
| cinder-scheduler | PNWC2-N2 | nova | enabled | up | 2021-03-18T03:25:04.000000 |
| cinder-scheduler | PNWC2-N1 | nova | enabled | up | 2021-03-18T03:25:03.000000 |
| cinder-volume | PNWC2-N3@rbd-1 | nova | enabled | down | 2021-03-09T09:42:13.000000 |
| cinder-volume | PNWC2-N2@rbd-1 | nova | enabled | down | 2021-03-09T09:42:13.000000 |
| cinder-volume | PNWC2-N1@rbd-1 | nova | enabled | down | 2021-03-09T09:42:12.000000 |
| cinder-volume | PNWC2-N9@rbd-1 | nova | enabled | down | 2021-03-09T09:42:13.000000 |
| cinder-volume | PNWC2-N10@rbd-1 | nova | enabled | down | 2021-03-09T09:42:12.000000 |
| cinder-volume | PNWC2-N6@rbd-1 | nova | enabled | down | 2021-03-09T09:42:13.000000 |
| cinder-volume | PNWC2-N4@rbd-1 | nova | enabled | down | 2021-03-11T06:42:04.000000 |
| cinder-volume | PNWC2-N5@rbd-1 | nova | enabled | down | 2021-03-09T09:42:13.000000 |
| cinder-volume | PNWC2-N8@rbd-1 | nova | enabled | down | 2021-03-09T09:42:13.000000 |
| cinder-volume | PNWC2-N7@rbd-1 | nova | enabled | down | 2021-03-09T09:42:13.000000 |
| cinder-backup | PNWC2-N5 | nova | enabled | up | 2021-03-18T03:25:05.000000 |
| cinder-backup | PNWC2-N4 | nova | enabled | up | 2021-03-18T03:25:04.000000 |
| cinder-backup | PNWC2-N10 | nova | enabled | up | 2021-03-18T03:25:04.000000 |
| cinder-backup | PNWC2-N9 | nova | enabled | up | 2021-03-18T03:25:05.000000 |
| cinder-backup | PNWC2-N6 | nova | enabled | up | 2021-03-18T03:25:02.000000 |
| cinder-backup | PNWC2-N8 | nova | enabled | up | 2021-03-18T03:25:06.000000 |
| cinder-backup | PNWC2-N7 | nova | enabled | up | 2021-03-18T03:25:02.000000 |
| cinder-backup | PNWC2-N2 | nova | enabled | up | 2021-03-18T03:25:02.000000 |
| cinder-backup | PNWC2-N3 | nova | enabled | up | 2021-03-18T03:25:09.000000 |
| cinder-backup | PNWC2-N1 | nova | enabled | up | 2021-03-18T03:25:03.000000 |
+------------------+-----------------+------+---------+-------+----------------------------+
重启也启动不了日志反复报如下错误
2021-03-10 08:53:09.009 18812 INFO cinder.service [-] Starting cinder-volume node (version 10.0.2)
2021-03-10 08:53:09.025 18812 INFO cinder.coordination [-] Coordination backend started successfully.
2021-03-10 08:53:09.026 18812 INFO cinder.volume.manager [req-2fb7587c-c519-459c-9648-12e4e135c24c - - - - -] Starting volume driver RBDDriver (1.2.0)
2021-03-10 08:53:09.468 18812 ERROR cinder.volume.drivers.rbd [req-2fb7587c-c519-459c-9648-12e4e135c24c - - - - -] error opening rbd image volume-c2d4fcb5-285e-4365-9d0e-66fec2c871d3
2021-03-10 08:53:09.468 18812 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
2021-03-10 08:53:09.468 18812 ERROR cinder.volume.drivers.rbd File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 119, in __init__
2021-03-10 08:53:09.468 18812 ERROR cinder.volume.drivers.rbd read_only=read_only)
2021-03-10 08:53:09.468 18812 ERROR cinder.volume.drivers.rbd File "rbd.pyx", line 1061, in rbd.Image.__init__ (rbd.c:8652)
2021-03-10 08:53:09.468 18812 ERROR cinder.volume.drivers.rbd ImageNotFound: error opening image volume-c2d4fcb5-285e-4365-9d0e-66fec2c871d3 at snapshot None
2021-03-10 08:53:09.468 18812 ERROR cinder.volume.drivers.rbd
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service [req-2fb7587c-c519-459c-9648-12e4e135c24c - - - - -] Error starting thread.
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service Traceback (most recent call last):
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_service/service.py", line 722, in run_service
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service service.start()
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/service.py", line 241, in start
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service service_id=Service.service_id)
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/manager.py", line 442, in init_host
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service self.driver.init_capabilities()
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/driver.py", line 719, in init_capabilities
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service stats = self.get_volume_stats(True)
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 423, in get_volume_stats
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service self._update_volume_stats()
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 409, in _update_volume_stats
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service self._get_usage_info()
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 364, in _get_usage_info
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service with RBDVolumeProxy(self, t, read_only=True) as v:
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 119, in __init__
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service read_only=read_only)
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service File "rbd.pyx", line 1061, in rbd.Image.__init__ (rbd.c:8652)
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service ImageNotFound: error opening image volume-c2d4fcb5-285e-4365-9d0e-66fec2c871d3 at snapshot None
2021-03-10 08:53:09.472 18812 ERROR oslo_service.service
2021-03-10 08:53:09.488 6 INFO oslo_service.service [req-eb769754-26fa-4f51-90c5-22d02d6c3eaf - - - - -] Child 18812 exited with status 1
提示打不开的那个volume-c2d4fcb5-285e-4365-9d0e-66fec2c871d3也不是个特殊卷,但就是一直提示,ceph中也能找到这个volume,但是不知道跟哪有关联,求助大神,如何启动cinder-volume这个服务。