cfpl12011124
cfpl12011124
2018-09-29 13:16

logstash全量同步mysql百万级别数据问题?

  • logstash
  • elasticsearch
  • mysql

mysql单表数据大概300万左右。全量同步的时候每次同步到20000的时候就会报错。

[WARN ][logstash.inputs.jdbc     ] Exception when executing JDBC query {:exception=>#<Sequel::DatabaseError: Java::JavaLang::IllegalArgumentException: HOUR_OF_DAY: 2 -> 3>}

在连接数据的URL后面加上时区UTC或者CST又会报错MONTH
不知道有没有大神碰到过这个问题
logstash,es版本均为6.4.1

 input {
  jdbc {
    jdbc_driver_library => "/opt/middleware/logstash-6.4.1/lib/mysql-connector-java-6.0.4.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/ad?characterEncoding=UTF-8&useSSL=false&serverTimezone=UTC"
    jdbc_user => "root"
    jdbc_password => "123456"
    statement_filepath => "/opt/middleware/logstash-6.4.1/config/alarm.sql"
    jdbc_paging_enabled => "true"
    jdbc_page_size => "5000"
    schedule => "* * * * *"
  }
}
 [WARN ][logstash.inputs.jdbc     ] Exception when executing JDBC query {:exception=>#<Sequel::DatabaseError: Java::JavaLang::IllegalArgumentException: MONTH>}
  • 点赞
  • 回答
  • 收藏
  • 复制链接分享

2条回答