weixin_39612122
weixin_39612122
2020-11-29 09:09

loader checker成功 导入失败

./checker -user root -password 密码 -host 127.0.0.1 -port 3306 DB
成功 ./loader -h 远程host -u root -P 4000 -t 4 -d /opt/backup/soeasy_business.sql/

2017/09/06 10:41:42 db.go:122: [debug] [exec][sql]USE soeasy_business; 2017/09/06 10:41:42 db.go:122: [debug] [exec][sql]INSERT INTO receipt_user VALUES(49822057,50005314853,2011856,1,800551,0,0,0,50005314843,170261,60087,"2016-07-04 21:57:19","2016-07-04 21:57:19",1,NULL,"2016-09-02 19:40:44","2016-09-18 11:05:43","2 2017/09/06 10:41:43 db.go:122: [debug] [exec][sql]UPDATE tidb_loader.checkpoint SET offset=45995889 WHERE id ='08b7c3' AND filename='soeasy_business.receipt_user.000000033.sql'; 2017/09/06 10:42:03 db.go:128: [warning] [exec][sql][USE soeasy_business; INSERT INTO receipt_user VALUES(49822057,50005314853,2011856,1,800551,0,0,0,50005314843,170261,600 UPDATE tidb_loader.checkpoint SET offset=45995889 WHERE id ='08b7c3' AND filename='soeasy_business.r][error]Error 1105: [try again later]: backoffer.maxSleep 20000ms is exceeded, errors: message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > 2017/09/06 10:42:03 db.go:99: [warning] transaction execution costs 20.943735 seconds 2017/09/06 10:42:03 db.go:67: [warning] exec sql retry 9 - [USE soeasy_business; INSERT INTO receipt_user VALUES(49822057,50005314853,2011856,1,800551,0,0,0,50005314843,170261,600 UPDATE tidb_loader.checkpoint SET offset=45995889 WHERE id ='08b7c3' AND filename='soeasy_business.r] 2017/09/06 10:42:21 db.go:122: [debug] [exec][sql]USE soeasy_business; 2017/09/06 10:42:21 db.go:122: [debug] [exec][sql]INSERT INTO receipt_user VALUES(49822057,50005314853,2011856,1,800551,0,0,0,50005314843,170261,60087,"2016-07-04 21:57:19","2016-07-04 21:57:19",1,NULL,"2016-09-02 19:40:44","2016-09-18 11:05:43","2 2017/09/06 10:42:22 db.go:122: [debug] [exec][sql]UPDATE tidb_loader.checkpoint SET offset=45995889 WHERE id ='08b7c3' AND filename='soeasy_business.receipt_user.000000033.sql'; 2017/09/06 10:42:42 db.go:128: [warning] [exec][sql][USE soeasy_business; INSERT INTO receipt_user VALUES(49822057,50005314853,2011856,1,800551,0,0,0,50005314843,170261,600 UPDATE tidb_loader.checkpoint SET offset=45995889 WHERE id ='08b7c3' AND filename='soeasy_business.r][error]Error 1105: [try again later]: backoffer.maxSleep 20000ms is exceeded, errors: message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > 2017/09/06 10:42:42 db.go:99: [warning] transaction execution costs 20.817224 seconds 2017/09/06 10:42:42 loader.go:111: [fatal] Error 1105: [try again later]: backoffer.maxSleep 20000ms is exceeded, errors: message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > message:"region is stale" stale_epoch: peers:\<3 store_id:1> peers:\<6 store_id:4> peers:\<7 store_id:5> > new_regions:\<1404 start_key: end_key: region_epoch: version:26> peers:\<1405 store_id:1> peers:\<1406 store_id:4> peers:\<1407 store_id:5> > > /home/jenkins/workspace/build_tidb_enterprise_tools_master/go/src/github.com/pingcap/tidb-enterprise-tools/loader/db.go:84: 1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>1407>1406>1405>1404>7>6>3>

该提问来源于开源项目:pingcap/docs-cn

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

10条回答

  • weixin_39981041 weixin_39981041 4月前

    『3个tikv server 』有几台物理机?请提供一下导入时每台 tikv 机器的 IO、CPU、内存状况

    点赞 评论 复制链接分享
  • weixin_39612122 weixin_39612122 4月前

    3台tikv 都是云主机 以下是导入时,服务器状态 1 cpu 1.5 - 4 % mem 64G use 20G Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn vdb 9.75 91.58 2069.11 117269022 2649522090 2 cpu. 4-6% mem 64G use 20G Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn vdb 9.95 117.98 2027.74 151068790 2596519421 3 cpu. 1.5 -4 % mem 64G use 20G Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn vdb 10.29 102.80 2128.67 131634294 2725826670

    点赞 评论 复制链接分享
  • weixin_39932330 weixin_39932330 4月前
    
    

    hi,看日志出现了 region is stale , 是应为 tikv 过于繁忙出现了。 - tidb 集群使用了几台机器? - 机器配置如何? - 物理机 or 虚拟机 ? - CPU 核数 and 内存大小 - 单台机器是否部署了多个 tikv - 磁盘性能如何? - dd bs=4k count=2560000 if=/dev/zero of=test oflag=direct

    点赞 评论 复制链接分享
  • weixin_39612122 weixin_39612122 4月前

    嗯,确实,整个环境,tikv ,tidb是只分了一台,PD分了3台。 机器配置是云主机,磁盘io性能基本等同于ssd ,应该是用ssd 对 磁盘做了加速。 在ansible-playbook bootstrap.yml 安装过程中确实提示了PD 的磁盘性能不足,io性能大概是推荐值的一半 。 我跳过了,直接进行了ansible-playbook deploy.yml 安装。

    现在的问题是,我是否可以通过增加tikv server ,或者调整其他参数的方式解决这个问题。 服务器目前无法拿到更好的了。

    点赞 评论 复制链接分享
  • weixin_39932330 weixin_39932330 4月前
    • 首先将 tikv 调整到 3 台以上,且不能与 pd 放置在同一块磁盘。
    • 导出的单个 sql 文件是多大?
    • cpu 核数与 内存大小具体是多少?
    • loader 线程数可以适当性降低。按照现在 tikv 1 台的情况下,使用 -t 1 试试吧。
    点赞 评论 复制链接分享
  • weixin_39612122 weixin_39612122 4月前

    好的,我试试,tikv 1台的情况下,我 -t 1 已经试过了,不行的。 整个库 mydumper 备份150G ,最大文件是64M ,我重新备份将单个文件sql放小如何? 服务器 是16cpu 64G内存。

    点赞 评论 复制链接分享
  • weixin_39932330 weixin_39932330 4月前

    数据导出可以参考这部分内容mydumper

    点赞 评论 复制链接分享
  • weixin_39612122 weixin_39612122 4月前

    重新安装后,现在3个tikv server 3个Pd server 导入任然失败 2017/09/13 17:56:04 db.go:99: [warning] transaction execution costs 1.004126 seconds 2017/09/13 17:56:11 db.go:150: [error] exec sqls[[USE soeasy_business; INSERT INTO bid_transfer_all VALUES(50002503073,2027009,1,800524,100058,"#65#","泰禾集团","XZW1UBOJRH" UPDATE tidb_loader.checkpoint SET offset=62981093 WHERE id ='24484c' AND filename='soeasy_business.b]] commit failed Error 1105: Information schema is out of date. 2017/09/13 17:56:11 db.go:99: [warning] transaction execution costs 7.121481 seconds 2017/09/13 17:56:11 db.go:67: [warning] exec sql retry 1 - [USE soeasy_business; INSERT INTO bid_transfer_all VALUES(50002503073,2027009,1,800524,100058,"#65#","泰禾集团","XZW1UBOJRH" UPDATE tidb_loader.checkpoint SET offset=62981093 WHERE id ='24484c' AND filename='soeasy_business.b] 2017/09/13 17:56:19 db.go:150: [error] exec sqls[[USE soeasy_business; INSERT INTO bid_transfer_all VALUES(50002503073,2027009,1,800524,100058,"#65#","泰禾集团","XZW1UBOJRH" UPDATE tidb_loader.checkpoint SET offset=62981093 WHERE id ='24484c' AND filename='soeasy_business.b]] commit failed Error 1105: Information schema is out of date. 2017/09/13 17:56:19 db.go:99: [warning] transaction execution costs 6.043543 seconds 2017/09/13 17:56:19 db.go:67: [warning] exec sql retry 2 - [USE soeasy_business; INSERT INTO bid_transfer_all VALUES(50002503073,2027009,1,800524,100058,"#65#","泰禾集团","XZW1UBOJRH" UPDATE tidb_loader.checkpoint SET offset=62981093 WHERE id ='24484c' AND filename='soeasy_business.b] 2017/09/13 17:56:30 db.go:150: [error] exec sqls[[USE soeasy_business; INSERT INTO bid_transfer_all VALUES(50002503073,2027009,1,800524,100058,"#65#","泰禾集团","XZW1UBOJRH" UPDATE tidb_loader.checkpoint SET offset=62981093 WHERE id ='24484c' AND filename='soeasy_business.b]] commit failed Error 1105: Information schema is out of date. 这个什么错误? 有Q 或者微信吗?

    点赞 评论 复制链接分享
  • weixin_39981041 weixin_39981041 4月前

    AND filename='soeasy_business.b]] commit failed Error 1105: Information schema is out of date.

    这种错误说明 TiKV 的负载太高了,可以看一下磁盘IO、CPU 是不是都打满了,推荐用更好的机器。

    点赞 评论 复制链接分享
  • weixin_39612122 weixin_39612122 4月前

    tikv 从top 观察来看,真看不出任何压力

    点赞 评论 复制链接分享