clickhouse报DB::Exception: Failed to wait RAFT initialization

Question

问题遇到的现象和发生背景

clickhouse启动正常，执行clickhouse-client命令，报无法连接9000端口，日志报错原因：Application: DB::Exception: Failed to wait RAFT initialization

报错日志

2024.08.23 17:18:13.836623 [ 1496250 ] {} <Error> RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled
2024.08.23 17:18:13.848202 [ 1496235 ] {} <Error> Application: DB::Exception: Failed to wait RAFT initialization
 
2024.08.23 17:18:44.589182 [ 1497071 ] {} <Warning> KeeperLogStore: Removing all changelogs
2024.08.23 17:18:44.589266 [ 1497071 ] {} <Warning> KeeperLogStore: Moving broken logs to /data/clickhouse/keeper/log/detached/20240823T171844
2024.08.23 17:18:44.589341 [ 1497071 ] {} <Warning> KeeperLogStore: Removing changelog /data/clickhouse/keeper/log/changelog_1_100000.bin.zstd

config.xml新增配置，及metrika.xml配置

<keeper_server>
   <tcp_port>9181</tcp_port>
   <server_id>1</server_id>
   <log_storage_path>/data/clickhouse/keeper/log</log_storage_path>
   <snapshot_storage_path>/data/clickhouse/keeper/coordination/snapshots</snapshot_storage_path>
   <coordination_settings>
       <operation_timeout_ms>10000</operation_timeout_ms>
       <session_timeout_ms>30000</session_timeout_ms>
       <raft_logs_level>trace</raft_logs_level>
   </coordination_settings>
   <raft_configuration>
       <server>
           <id>1</id>
           <hostname>134.133.12.70</hostname>
           <port>9444</port>
       </server>
       <server>
           <id>2</id>
           <hostname>134.133.12.71</hostname>
           <port>9444</port>
       </server>
       <server>
           <id>3</id>
           <hostname>134.133.12.72</hostname>
           <port>9444</port>
       </server>
       <server>
           <id>4</id>
           <hostname>134.133.12.73</hostname>
           <port>9444</port>
       </server>
       <server>
           <id>5</id>
           <hostname>134.133.12.74</hostname>
           <port>9444</port>
       </server>
   </raft_configuration>
</keeper_server>
<listen_host>::</listen_host>
<include_from>/etc/clickhouse-server/metrika.xml</include_from>
<remote_servers incl="clickhouse_remote_servers" optional="true"/>
<zookeeper incl="zookeeper-servers" optional="true"/>
<macros incl="macros" optional="true"/>

<clickhouse>
   <zookeeper-servers>
       <node index="1">
           <host>134.133.12.70</host>
           <port>9181</port>
       </node>
       <node index="2">
           <host>134.133.12.71</host>
           <port>9181</port>
       </node>
       <node index="3">
           <host>134.133.12.72</host>
           <port>9181</port>
       </node>
       <node index="4">
           <host>134.133.12.73</host>
           <port>9181</port>
       </node>
       <node index="5">
           <host>134.133.12.74</host>
           <port>9181</port>
       </node>
       <node index="6">
           <host>134.133.12.75</host>
           <port>9181</port>
       </node>
       <node index="7">
           <host>134.133.12.76</host>
           <port>9181</port>
       </node>
       <node index="8">
           <host>134.133.12.77</host>
           <port>9181</port>
       </node>
       <node index="9">
           <host>134.133.12.78</host>
           <port>9181</port>
       </node>
       <node index="10">
           <host>134.133.12.79</host>
           <port>9181</port>
       </node>
   </zookeeper-servers>
   <clickhouse_remote_servers>
       <ck_hnids>
           <secret>foo</secret>
           <shard>
               <internal_replication>true</internal_replication>
               <replica>
                   <host>134.133.12.70</host>
                   <port>9000</port>
               </replica>
               <replica>
                   <host>134.133.12.75</host>
                   <port>9000</port>
               </replica>
           </shard>
           <shard>
               <internal_replication>true</internal_replication>
               <replica>
                   <host>134.133.12.71</host>
                   <port>9000</port>
               </replica>
               <replica>
                   <host>134.133.12.76</host>
                   <port>9000</port>
               </replica>
           </shard>
           <shard>
               <internal_replication>true</internal_replication>
               <replica>
                   <host>134.133.12.72</host>
                   <port>9000</port>
               </replica>
               <replica>
                   <host>134.133.12.77</host>
                   <port>9000</port>
               </replica>
           </shard>
           <shard>
               <internal_replication>true</internal_replication>
               <replica>
                   <host>134.133.12.73</host>
                   <port>9000</port>
               </replica>
               <replica>
                   <host>134.133.12.78</host>
                   <port>9000</port>
               </replica>
           </shard>
           <shard>
               <internal_replication>true</internal_replication>
               <replica>
                   <host>134.133.12.74</host>
                   <port>9000</port>
               </replica>
               <replica>
                   <host>134.133.12.79</host>
                   <port>9000</port>
               </replica>
           </shard>
       </ck_hnids>
   </clickhouse_remote_servers>
   <macros>
       <cluster>ck_hnids</cluster>
       <shard>04</shard>
       <replica>clickhouse4</replica>
   </macros>
   <networks>
       <ip>::/0</ip>
   </networks>
   <!--数据压缩配置 MergeTree-引擎表-->
   <clickhouse_compression>
       <case>
           <min_part_size>10000000000</min_part_size>
           <min_part_size_ratio>0.01</min_part_size_ratio>
           <method>lz4</method>
       </case>
   </clickhouse_compression>
</clickhouse>

运行结果

启动状态正常运行，日志报错

解决尝试

尝试重新安装clickhouse，但问题一样，求解答

Answer 1

执行 clickhouse-client 命令时无法连接到 9000 端口，并且日志中出现了 Failed to wait RAFT initialization 的错误。这通常与 RAFT 协议的初始化失败有关，可能是由于配置问题、网络问题或其他原因导致的。