Kettle数据迁移MySQL连接关闭导致失败？

Question

使用的是kettle8.2，数据迁移用的是表输入-表输出，然后数据量现在到了200多万条后报错了，错误信息如下

2019/09/13 02:20:42 - 表输入.0 - linenr 2050000
2019/09/13 02:20:47 - 表输出.0 - linenr 2050000
2019/09/13 02:21:06 - 表输入.0 - linenr 2100000
2019/09/13 02:21:11 - 表输出.0 - linenr 2100000
2019/09/13 02:22:43 - 表输出.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Because of an error, this step can't continue:
2019/09/13 02:22:43 - 表输出.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException:
2019/09/13 02:22:43 - 表输出.0 - Unable to clear batch for prepared statement
2019/09/13 02:22:43 - 表输出.0 - No operations allowed after statement closed.
2019/09/13 02:22:43 - 表输出.0 -
2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.core.database.Database.clearBatch(Database.java:1342)
2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:337)
2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:125)
2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2019/09/13 02:22:43 - 表输出.0 - at java.lang.Thread.run(Thread.java:745)
2019/09/13 02:22:43 - 表输出.0 - Caused by: java.sql.SQLException: No operations allowed after statement closed.
2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:959)
2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:898)
2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:887)
2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:862)
2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.StatementImpl.checkClosed(StatementImpl.java:438)
2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.PreparedStatement.clearBatch(PreparedStatement.java:1007)
2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.core.database.Database.clearBatch(Database.java:1340)
2019/09/13 02:22:43 - 表输出.0 - ... 4 more
2019/09/13 02:22:43 - 表输入.0 - Finished reading query, closing connection.
2019/09/13 02:22:43 - 表输出.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Unexpected batch update error committing the database connection.

开始时间是 02:04，跑了大概18分钟后报错，但是什么都不修改重新启动就跑成功了，什么情况报错还没搞清楚。
是服务端把我的连接关闭了吗？
MySQL加testOnBorrow和testOnReturn暂时不允许加，要怎么修改呢？有没有用过kettle连接池的，能解决吗？

Answer 1

需要从优化数据库性能着手，大批量数据写入需要考虑使用多线程和批量插入的方式，减少数据库连接带来的性能影响。