linux 下kettle执行mysql抽数到oracle的job问题报错 5C

图片说明

            mysql驱动包已放在\data-integration\lib以及jdbc/driver
            执行job报错。
            驱动版本 5.1.47
            kettle版本 7.1

1个回答

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
Linux下,kettle连接MySQL资源库问题(数据库资源库,非文件资源库)
Linux下,部署完kettle之后,执行“./kitchen.sh”命令,已成功部署。 自动化执行文件如下图: ![图片说明](https://img-ask.csdn.net/upload/201704/28/1493350014_282102.jpg) 启动自动化调度后,报错,如下图: ![图片说明](https://img-ask.csdn.net/upload/201704/28/1493350048_288319.jpg) Processing stopped because of an error: Error connecting to the repository! Error occurred while trying to connect to the database Error connecting to database: (using class org.gjt.mm.mysql.Driver) Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. ERROR: Kitchen can't continue because the job couldn't be loaded. 注:mysql的jdbc驱动(mysql-connector-java-5.1.15-bin.jar)已经放到了 data-integration\lib下但仍然报错(在网上说这么好用,但是没有效果,还有说放到/app/kettle/data-integration/libext/JDBC目录下的,因为没有此目录,新建后放入此jar包,还是不管用),望大神赐教
kettle生产环境hive到mysql连接出错
1.kettle做数据转换 从hive到mysql,本地spoon.bat开发运行正常,上传到linux下用kitchen.sh执行不报错但是会卡住,debug模式下日志如图。 2.做了单个测试,job转换中只连接mysql执行个查询可以,或者只连接hive执行查询语句也可以。但是同时将两个步骤放到一个转换里 就出现跟上面一样的问题 ![图片说明](https://img-ask.csdn.net/upload/201910/30/1572405694_49226.jpg) na位路过的大佬帮忙看下。
linux 下 kettle Job 日志文件问题
启动脚本是 ``` /opt/kettle/data-integration/kitchen.sh -file=/opt/kettle/Job/syncToGreenCloud.kjb -level=basic >> /opt/kettle/rLog/syn_$(date +%Y%m%d).log & ``` JOB是每10分钟执行一次,日志的时间没有变过,永远在一个日志文件中,文件越来越大,占用资源。 如何能够将每次定时开启之前的日志清空。
kettle的JOB互相嵌套的作业,在windows上正常运行,迁移到Linux服务器就报错了
windows系统运行正常: ![图片说明](https://img-ask.csdn.net/upload/201907/29/1564389024_347344.png) Linux系统下运行报错: ``` /usr/local/kettle/data-integration/spoon.sh: line 140: ldconfig: command not found ####################################################################### WARNING: no libwebkitgtk-1.0 detected, some features will be unavailable Consider installing the package with apt-get or yum. e.g. 'sudo apt-get install libwebkitgtk-1.0-0' ####################################################################### Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 log4j:WARN Continuable parsing error 45 and column 76 log4j:WARN Element type "rollingPolicy" must be declared. log4j:WARN Continuable parsing error 52 and column 14 log4j:WARN The content of element type "appender" must match "(errorHandler?,param*,layout?,filter*,appender-ref*)". log4j:WARN Please set a rolling policy for the RollingFileAppender named 'pdi-execution-appender' 16:11:29,469 INFO [KarafBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled 16:11:29,646 INFO [KarafInstance] ******************************************************************************* *** Karaf Instance Number: 1 at /usr/local/kettle/data-integration/./system *** *** /karaf/caches/kitchen/data-1 *** *** FastBin Provider Port:52901 *** *** Karaf Port:8802 *** *** OSGI Service Port:9051 *** ******************************************************************************* Jul 29, 2019 4:11:32 PM org.apache.karaf.main.Main$KarafLockCallback lockAquired INFO: Lock acquired. Setting startlevel to 100 2019/07/29 16:11:36 - Kitchen - Start of run. 2019/07/29 16:11:36 - RepositoriesMeta - Reading repositories XML file: /usr/local/kettle/.kettle/repositories.xml 2019-07-29 16:11:53.034:INFO:oejs.Server:jetty-8.1.15.v20140411 2019-07-29 16:11:53.106:INFO:oejs.AbstractConnector:Started NIOSocketConnectorWrapper@0.0.0.0:9051 Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/core Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/configuration/beans Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/configuration/parameterized-types Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/configuration/security Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/wsdl/ Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://www.w3.org/2005/08/addressing Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/ws/2004/08/addressing Jul 29, 2019 4:11:58 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-management (195) [org.apache.cxf.management.InstrumentationManager] Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-wsdl (198) [org.apache.cxf.wsdl.WSDLManager] Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-bindings-xml (200) [org.apache.cxf.binding.xml.XMLBindingFactory, org.apache.cxf.binding.xml.wsdl11.XMLWSDLExtensionLoader] Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-bindings-soap (201) [org.apache.cxf.binding.soap.SoapBindingFactory, org.apache.cxf.binding.soap.SoapTransportFactory] Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/bindings/soap Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-transports-http (202) [org.apache.cxf.transport.http.HTTPTransportFactory, org.apache.cxf.transport.http.HTTPWSDLExtensionLoader, org.apache.cxf.transport.http.policy.HTTPClientAssertionBuilder, org.apache.cxf.transport.http.policy.HTTPServerAssertionBuilder, org.apache.cxf.transport.http.policy.NoOpPolicyInterceptorProvider] Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-policy (220) [org.apache.cxf.ws.policy.PolicyEngine, org.apache.cxf.policy.PolicyDataEngine, org.apache.cxf.ws.policy.AssertionBuilderRegistry, org.apache.cxf.ws.policy.PolicyInterceptorProviderRegistry, org.apache.cxf.ws.policy.PolicyBuilder, org.apache.cxf.ws.policy.PolicyAnnotationListener, org.apache.cxf.ws.policy.attachment.ServiceModelPolicyProvider, org.apache.cxf.ws.policy.attachment.external.DomainExpressionBuilderRegistry, org.apache.cxf.ws.policy.attachment.external.EndpointReferenceDomainExpressionBuilder, org.apache.cxf.ws.policy.attachment.external.URIDomainExpressionBuilder, org.apache.cxf.ws.policy.attachment.wsdl11.Wsdl11AttachmentPolicyProvider, org.apache.cxf.ws.policy.mtom.MTOMAssertionBuilder, org.apache.cxf.ws.policy.mtom.MTOMPolicyInterceptorProvider] Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/transports/http/configuration Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/simple Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-frontend-jaxws (204) [org.apache.cxf.jaxws.context.WebServiceContextResourceResolver] Jul 29, 2019 4:11:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/jaxws Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/jaxrs Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/jaxrs-client Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/binding/coloc Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-transports-local (216) [org.apache.cxf.transport.local.LocalTransportFactory] Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-bindings-object (217) [org.apache.cxf.binding.object.ObjectBindingFactory] Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/binding/object Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/policy Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://www.w3.org/ns/ws-policy Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://www.w3.org/2006/07/ws-policy Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/ws/2004/09/policy Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://www.w3.org/2000/09/xmldsig# Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702 Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-addr (237) [org.apache.cxf.ws.addressing.policy.AddressingAssertionBuilder, org.apache.cxf.ws.addressing.policy.UsingAddressingAssertionBuilder, org.apache.cxf.ws.addressing.policy.AddressingPolicyInterceptorProvider, org.apache.cxf.ws.addressing.impl.AddressingWSDLExtensionLoader, org.apache.cxf.ws.addressing.WSAddressingFeature$WSAddressingFeatureApplier, org.apache.cxf.ws.addressing.MAPAggregator$MAPAggregatorLoader] Jul 29, 2019 4:12:00 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/ws/addressing Jul 29, 2019 4:12:01 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-security (239) [org.apache.cxf.ws.security.policy.WSSecurityPolicyLoader, org.apache.cxf.ws.security.cache.CacheCleanupListener] Jul 29, 2019 4:12:01 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-rm (241) [org.apache.cxf.ws.rm.RMManager, org.apache.cxf.ws.rm.policy.RMPolicyInterceptorProvider, org.apache.cxf.ws.rm.policy.RM10AssertionBuilder, org.apache.cxf.ws.rm.policy.RM12AssertionBuilder, org.apache.cxf.ws.rm.policy.WSRMP12PolicyLoader, org.apache.cxf.ws.rm.policy.MC11PolicyLoader, org.apache.cxf.ws.rm.policy.RSPPolicyLoader] Jul 29, 2019 4:12:01 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://cxf.apache.org/ws/rm/manager Jul 29, 2019 4:12:01 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/ws/2005/02/rm/policy Jul 29, 2019 4:12:01 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-javascript (242) [org.apache.cxf.javascript.JavascriptServerListener] Jul 29, 2019 4:12:01 PM org.pentaho.caching.impl.PentahoCacheManagerFactory$RegistrationHandler$1 onSuccess INFO: New Caching Service registered SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/kettle/data-integration/launcher/../lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/kettle/data-integration/plugins/pentaho-big-data-plugin/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2019/07/29 16:12:08 - test - Start of job execution log4j:ERROR No output stream or file set for the appender named [pdi-execution-appender]. 2019/07/29 16:12:08 - test - Starting entry [job1] 2019/07/29 16:12:08 - test - Launched job entry [job1] in parallel. 2019/07/29 16:12:08 - test - Starting entry [job2] 2019/07/29 16:12:08 - test - Launched job entry [job2] in parallel. 2019/07/29 16:12:08 - job2 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Error running job entry 'job' : 2019/07/29 16:12:08 - job1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Error running job entry 'job' : 2019/07/29 16:12:08 - job2 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleException: 2019/07/29 16:12:08 - job2 - Unexpected error during job metadata load 2019/07/29 16:12:08 - job2 - at java.lang.Thread.run (Thread.java:748) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.Job$1.run (Job.java:798) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.Job.access$000 (Job.java:121) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.Job.execute (Job.java:680) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.execute (JobEntryJob.java:667) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1343) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1381) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository (JobEntryJob.java:1353) 2019/07/29 16:12:08 - job2 - 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1421) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1343) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.execute(JobEntryJob.java:667) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.Job.execute(Job.java:680) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.Job.access$000(Job.java:121) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.Job$1.run(Job.java:798) 2019/07/29 16:12:08 - job2 - at java.lang.Thread.run(Thread.java:748) 2019/07/29 16:12:08 - job2 - Caused by: java.lang.NullPointerException 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository(JobEntryJob.java:1353) 2019/07/29 16:12:08 - job2 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1381) 2019/07/29 16:12:08 - job2 - ... 6 more 2019/07/29 16:12:08 - job1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleException: 2019/07/29 16:12:08 - job1 - Unexpected error during job metadata load 2019/07/29 16:12:08 - job1 - at java.lang.Thread.run (Thread.java:748) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.Job$1.run (Job.java:798) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.Job.access$000 (Job.java:121) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.Job.execute (Job.java:680) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.execute (JobEntryJob.java:667) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1343) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1381) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository (JobEntryJob.java:1353) 2019/07/29 16:12:08 - job1 - 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1421) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1343) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.execute(JobEntryJob.java:667) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.Job.execute(Job.java:680) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.Job.access$000(Job.java:121) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.Job$1.run(Job.java:798) 2019/07/29 16:12:08 - job1 - at java.lang.Thread.run(Thread.java:748) 2019/07/29 16:12:08 - job1 - Caused by: java.lang.NullPointerException 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository(JobEntryJob.java:1353) 2019/07/29 16:12:08 - job1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1381) 2019/07/29 16:12:08 - job1 - ... 6 more 2019/07/29 16:12:08 - test - Job execution finished 2019/07/29 16:12:08 - Kitchen - Finished! 2019/07/29 16:12:08 - Kitchen - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Finished with errors 2019/07/29 16:12:08 - Kitchen - Start=2019/07/29 16:11:36.193, Stop=2019/07/29 16:12:08.071 2019/07/29 16:12:08 - Kitchen - Processing ended after 31 seconds. ``` Linux环境运行作业调用转换的就没问题,作业调用作业的就报错,望大神指点,哪儿有问题。 kettle版本为8.2,jdk版本为 java version "1.8.0_171" 如果需要什么其他的参数,我再上传
kettle连接MYSQL数据库问题
![图片说明](https://img-ask.csdn.net/upload/201909/24/1569314912_453856.png)![图片说明](https://img-ask.csdn.net/upload/201909/24/1569314922_599953.png) mysql的驱动放到kettle安装文件中的lib下,连不上一直报错,折腾一下午了 求大佬来看看呀!
Linux下,kettle连接MySQL资源库问题
Processing stopped because of an error: Error connecting to the repository! Error occurred while trying to connect to the database Error connecting to database: (using class org.gjt.mm.mysql.Driver) Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. ERROR: Kitchen can't continue because the job couldn't be loaded. ![图片说明](https://img-ask.csdn.net/upload/201704/27/1493278500_404035.jpg)
用kettle把数据从mysql数据库中抽取到oracle库中字段出现问题
字段得的值是空值,然后运行kettle报错说标识符无效。 这个是oracle的字段 ![图片说明](https://img-ask.csdn.net/upload/201906/28/1561705825_769569.png) 。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。 这个是Mysql中的字段 ![图片说明](https://img-ask.csdn.net/upload/201906/28/1561705850_478697.png) 。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。 这个是报错的问题。 ![图片说明](https://img-ask.csdn.net/upload/201906/28/1561705878_237143.png) 是什么原因???求各路大神路过能解决下
kettle循环执行job内存溢出
循环分页查询一天的数据,每页1000条,1000个ID拼成串并设置为变量,供后边查询使用,kettle版本7.1,尝试过调大内存,还是不行。也会触发gc,但是经历新生代后进入老年代,直到撑爆堆内存,jdk版本1.8
kettle job执行调用问题
刚刚开始学习Kettle,对于一些问题不是很明白。kettle是一种ETL工具,它肯定是为ETL服务的,比如我现在就要根据业务需求对数据进行ETL,忽略其他步骤,只说用kettle对数据进行清洗,那么我肯定会写很多的job,写完了测试也完成了,项目上线了这些job怎么调用或者说执行啊?我现在只会写完一个start一个?主要是这里不明白,求大神解惑!!!http://ask.csdn.net/?ref=toolbar#
求助!使用Kettle从impala中取数慢
CDH6.2 第一套环境:4C,64G 有3台 第二套环境:4C,256G有5台 使用kettle配置两个数据源,分别从impala中取数(两套环境的数据量一样)。 单独把sql在Impala上执行,两套环境的执行时间几乎一样(相差几秒) 但是使用Kettle抽取时间3台机子的抽取时间为20秒。 5台机子的抽取时间是45秒。
kettle连接oracle 11g 报错 IO Error: Connection reset
报错信息如下: Error occurred while trying to connect to the database Error connecting to database: (using class oracle.jdbc.driver.OracleDriver) IO Error: Connection reset ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : Error initializing step [CREWS IDE_SOFL_T] 我的kettle版本是7.0 oracle是11g 这个问题困扰我一个月了 求大神指导 网上找了好多解决办法都不行
kettle执行job成功,Java执行kettle报错,求大神指导
[INFO,DoKettleService,http-bio-8080-exec-5] 执行kettle开始 [INFO,StandardFileSystemManager,http-bio-8080-exec-5] Using "C:\Users\ADMINI~1\AppData\Local\Temp\vfs_cache" as temporary files store. 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 开始执行任务 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 开始项[数据追溯] 2017/03/27 09:28:11 - cetl_001 - ERROR (version 1.0.0, build 1 from 2015-07-31 09.59.35 by chenrh) : 一个数据库错误发生在从资源库文件读取转换时 2017/03/27 09:28:11 - cetl_001 - org.pentaho.di.core.exception.KettlePluginLoaderException: 2017/03/27 09:28:11 - cetl_001 - Unable to load class for step/plugin with id [trace 2017/03/27 09:28:11 - cetl_001 - ]. Check if the plugin is available in the plugins subdirectory of the Kettle distribution. 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - ERROR (version 1.0.0, build 1 from 2015-07-31 09.59.35 by chenrh) : org.pentaho.di.core.exception.KettleException: 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - Unexpected error during transformation metadata load 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 一个数据库错误发生在从资源库文件读取转换时 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - Unable to load class for step/plugin with id [trace 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - ]. Check if the plugin is available in the plugins subdirectory of the Kettle distribution. 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.pentaho.di.job.entries.trans.JobEntryTrans.getTransMeta(JobEntryTrans.java:1208) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:648) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.pentaho.di.job.Job.execute(Job.java:716) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.pentaho.di.job.Job.execute(Job.java:859) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.pentaho.di.job.Job.execute(Job.java:532) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.pentaho.di.job.Job.run(Job.java:424) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at com.wa.framework.quartz.service.DoKettleService.runJob(DoKettleService.java:52) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at com.wa.framework.quartz.service.DoKettleService$$FastClassBySpringCGLIB$$8a4f9c12.invoke(<generated>) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:708) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:644) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at com.wa.framework.quartz.service.DoKettleService$$EnhancerBySpringCGLIB$$3cedfde9.runJob(<generated>) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at com.udatech.common.taskManage.controller.CollectionTaskController.testDs(CollectionTaskController.java:310) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at java.lang.reflect.Method.invoke(Method.java:483) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:852) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at javax.servlet.http.HttpServlet.service(HttpServlet.java:621) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at com.wa.framework.log.AccessLogFilter.doFilter(AccessLogFilter.java:121) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:230) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at com.wa.framework.security.XssAndSqlFilter.doFilter(XssAndSqlFilter.java:43) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at com.wa.framework.filter.SecurityFilter.doFilter(SecurityFilter.java:106) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) 2017/03/27 09:28:11 - JOB_GLOBAL_RUN - at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
在程序中调用shell脚本执行kettle抽取报错
前提:服务器中间件用的是weblogic,操作系统用的是linux,kettle在服务器端执行抽取完全没有问题; 但是在程序中远程调用shell脚本执行却报错,以下是程序中调用代码: ... String[] command = { "/bin/sh", "-c", "XX.sh" }; Process ps = Runtime.getRuntime().exec(command); ... 报错信息为Error occured while trying to connect to the database! 已知:shell脚本路径没有问题、数据库一切正常、kettle数据库配置文件也没问题,kettle在服务器端执行抽取完全没有问题; 请各位大神帮忙分析下原因!谢谢! ![图片说明](https://img-ask.csdn.net/upload/201705/12/1494578158_306718.jpg)
kettle,把oracle的数据迁移到Mysql中是报错,感觉是字段方面的错误,求大神指教
2015/03/23 11:25:52 - 表输出.0 - Connected to database [local_hsps1] (commit=1000) 2015/03/23 11:25:52 - 表输出.0 - ERROR (version 5.3.0.0-213, build 1 from 2015-02-02_12-17-08 by buildguy) : Unexpected batch update error committing the database connection. 2015/03/23 11:25:52 - 表输出.0 - ERROR (version 5.3.0.0-213, build 1 from 2015-02-02_12-17-08 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseBatchException: 2015/03/23 11:25:52 - 表输出.0 - Error updating batch 2015/03/23 11:25:52 - 表输出.0 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'MAXVALUE, MINVALUE, MODIFIER, MODIFYTIME, NAME, RANK, RECORDTIME, RECORDER, STAT' at line 1 2015/03/23 11:25:52 - 表输出.0 - 2015/03/23 11:25:52 - 表输出.0 - at org.pentaho.di.core.database.Database.createKettleDatabaseBatchException(Database.java:1351) 2015/03/23 11:25:52 - 表输出.0 - at org.pentaho.di.core.database.Database.emptyAndCommit(Database.java:1340) 2015/03/23 11:25:52 - 表输出.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.dispose(TableOutput.java:571) 2015/03/23 11:25:52 - 表输出.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:96) 2015/03/23 11:25:52 - 表输出.0 - at java.lang.Thread.run(Thread.java:745) 2015/03/23 11:25:52 - 表输出.0 - Caused by: java.sql.BatchUpdateException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'MAXVALUE, MINVALUE, MODIFIER, MODIFYTIME, NAME, RANK, RECORDTIME, RECORDER, STAT' at line 1 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1815) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1277) 2015/03/23 11:25:52 - 表输出.0 - at org.pentaho.di.core.database.Database.emptyAndCommit(Database.java:1327) 2015/03/23 11:25:52 - 表输出.0 - ... 3 more 2015/03/23 11:25:52 - 表输出.0 - Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'MAXVALUE, MINVALUE, MODIFIER, MODIFYTIME, NAME, RANK, RECORDTIME, RECORDER, STAT' at line 1 2015/03/23 11:25:52 - 表输出.0 - at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2015/03/23 11:25:52 - 表输出.0 - at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2015/03/23 11:25:52 - 表输出.0 - at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2015/03/23 11:25:52 - 表输出.0 - at java.lang.reflect.Constructor.newInstance(Constructor.java:408) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.Util.handleNewInstance(Util.java:377) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.Util.getInstance(Util.java:360) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2530) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1907) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2141) 2015/03/23 11:25:52 - 表输出.0 - at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773) 2015/03/23 11:25:52 - 表输出.0 - ... 5 more 2015/03/23 11:25:52 - 表输出.0 - 完成处理 (I=0, O=0, R=15, W=0, U=0, E=1
kettle 如何设置只读权限
我在kettle资料库是放在Oracle, 想实现的效果是:别的用户可以登陆kettle的资料库,查看数仓的数据处理方式,但是不可以(修改)保存,可以通过登陆用户设置吗? ![图片说明](https://img-ask.csdn.net/upload/201911/26/1574754404_880968.jpg)
kettle配置连接mysql数据库已测可以通,但是后续获取字段步骤时就会卡住,也没报错信息
![图片说明](https://img-ask.csdn.net/upload/201910/30/1572402204_219510.png)mysql版本5.7,百度了这个版本的mysql连接OK的。表输入,也可以预览到具体数据。![图片说明](https://img-ask.csdn.net/upload/201910/30/1572402436_13799.png) 预览数据:![图片说明](https://img-ask.csdn.net/upload/201910/30/1572402884_737902.png)
在kettle的调试软件中执行没有问题,但是在java代码中调用就会报错,含有接口的调用。
![图片说明](https://img-ask.csdn.net/upload/201910/15/1571124184_733842.jpg) 这是我再调试器中的内容,然后用java代码调用这个任务的时候,就会报一下错误: ``` 2019/10/15 14:50:12 - JSON Input.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : 错误初始化步骤[JSON Input] 2019/10/15 14:50:12 - JSON Input.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : java.lang.NullPointerException 2019/10/15 14:50:12 - JSON Input.0 - at org.pentaho.di.trans.steps.fileinput.BaseFileInputStep.init(BaseFileInputStep.java:102) 2019/10/15 14:50:12 - JSON Input.0 - at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:69) 2019/10/15 14:50:12 - JSON Input.0 - at java.lang.Thread.run(Thread.java:745) ``` 我网上查找好像是因为转换里是并行执行的。所以会直接报空指针的错误。但是在调试器中为什么没有报错呢,是我java代码调用方式有问题吗?求大佬解惑 以下是我java调用代码:
Kettle数据迁移MySQL连接关闭导致失败?
使用的是kettle8.2,数据迁移用的是表输入-表输出,然后数据量现在到了200多万条后报错了,错误信息如下 2019/09/13 02:20:42 - 表输入.0 - linenr 2050000 2019/09/13 02:20:47 - 表输出.0 - linenr 2050000 2019/09/13 02:21:06 - 表输入.0 - linenr 2100000 2019/09/13 02:21:11 - 表输出.0 - linenr 2100000 2019/09/13 02:22:43 - 表输出.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Because of an error, this step can't continue: 2019/09/13 02:22:43 - 表输出.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException: 2019/09/13 02:22:43 - 表输出.0 - Unable to clear batch for prepared statement 2019/09/13 02:22:43 - 表输出.0 - No operations allowed after statement closed. 2019/09/13 02:22:43 - 表输出.0 - 2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.core.database.Database.clearBatch(Database.java:1342) 2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:337) 2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:125) 2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62) 2019/09/13 02:22:43 - 表输出.0 - at java.lang.Thread.run(Thread.java:745) 2019/09/13 02:22:43 - 表输出.0 - Caused by: java.sql.SQLException: No operations allowed after statement closed. 2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:959) 2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:898) 2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:887) 2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:862) 2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.StatementImpl.checkClosed(StatementImpl.java:438) 2019/09/13 02:22:43 - 表输出.0 - at com.mysql.jdbc.PreparedStatement.clearBatch(PreparedStatement.java:1007) 2019/09/13 02:22:43 - 表输出.0 - at org.pentaho.di.core.database.Database.clearBatch(Database.java:1340) 2019/09/13 02:22:43 - 表输出.0 - ... 4 more 2019/09/13 02:22:43 - 表输入.0 - Finished reading query, closing connection. 2019/09/13 02:22:43 - 表输出.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Unexpected batch update error committing the database connection. 开始时间是 02:04,跑了大概18分钟后报错,但是什么都不修改重新启动就跑成功了,什么情况报错还没搞清楚。 是服务端把我的连接关闭了吗? MySQL加testOnBorrow和testOnReturn暂时不允许加,要怎么修改呢?有没有用过kettle连接池的,能解决吗?
数据表中的唯一索引带有decode函数,Kettle同步数据‘插入/更新’时该怎样写
数据表中的唯一索引带有decode函数,在用Kettle同步数据时,插入/更新那个地方用来查询的关键字如何写?
相见恨晚的超实用网站
搞学习 知乎:www.zhihu.com 简答题:http://www.jiandati.com/ 网易公开课:https://open.163.com/ted/ 网易云课堂:https://study.163.com/ 中国大学MOOC:www.icourse163.org 网易云课堂:study.163.com 哔哩哔哩弹幕网:www.bilibili.com 我要自学网:www.51zxw
花了20分钟,给女朋友们写了一个web版群聊程序
参考博客 [1]https://www.byteslounge.com/tutorials/java-ee-html5-websocket-example
爬虫福利二 之 妹子图网MM批量下载
爬虫福利一:27报网MM批量下载    点击 看了本文,相信大家对爬虫一定会产生强烈的兴趣,激励自己去学习爬虫,在这里提前祝:大家学有所成! 目标网站:妹子图网 环境:Python3.x 相关第三方模块:requests、beautifulsoup4 Re:各位在测试时只需要将代码里的变量 path 指定为你当前系统要保存的路径,使用 python xxx.py 或IDE运行即可。
字节跳动视频编解码面经
引言 本文主要是记录一下面试字节跳动的经历。 三四月份投了字节跳动的实习(图形图像岗位),然后hr打电话过来问了一下会不会opengl,c++,shador,当时只会一点c++,其他两个都不会,也就直接被拒了。 七月初内推了字节跳动的提前批,因为内推没有具体的岗位,hr又打电话问要不要考虑一下图形图像岗,我说实习投过这个岗位不合适,不会opengl和shador,然后hr就说秋招更看重基础。我当时
Java学习的正确打开方式
在博主认为,对于入门级学习java的最佳学习方法莫过于视频+博客+书籍+总结,前三者博主将淋漓尽致地挥毫于这篇博客文章中,至于总结在于个人,实际上越到后面你会发现学习的最好方式就是阅读参考官方文档其次就是国内的书籍,博客次之,这又是一个层次了,这里暂时不提后面再谈。博主将为各位入门java保驾护航,各位只管冲鸭!!!上天是公平的,只要不辜负时间,时间自然不会辜负你。 何谓学习?博主所理解的学习,它
程序员必须掌握的核心算法有哪些?
由于我之前一直强调数据结构以及算法学习的重要性,所以就有一些读者经常问我,数据结构与算法应该要学习到哪个程度呢?,说实话,这个问题我不知道要怎么回答你,主要取决于你想学习到哪些程度,不过针对这个问题,我稍微总结一下我学过的算法知识点,以及我觉得值得学习的算法。这些算法与数据结构的学习大多数是零散的,并没有一本把他们全部覆盖的书籍。下面是我觉得值得学习的一些算法以及数据结构,当然,我也会整理一些看过
大学四年自学走来,这些私藏的实用工具/学习网站我贡献出来了
大学四年,看课本是不可能一直看课本的了,对于学习,特别是自学,善于搜索网上的一些资源来辅助,还是非常有必要的,下面我就把这几年私藏的各种资源,网站贡献出来给你们。主要有:电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意:文中提到的所有资源,文末我都给你整理好了,你们只管拿去,如果觉得不错,转发、分享就是最大的支持了。 一、电子书搜索 对于大部分程序员...
linux系列之常用运维命令整理笔录
本博客记录工作中需要的linux运维命令,大学时候开始接触linux,会一些基本操作,可是都没有整理起来,加上是做开发,不做运维,有些命令忘记了,所以现在整理成博客,当然vi,文件操作等就不介绍了,慢慢积累一些其它拓展的命令,博客不定时更新 顺便拉下票,我在参加csdn博客之星竞选,欢迎投票支持,每个QQ或者微信每天都可以投5票,扫二维码即可,http://m234140.nofollow.ax.
比特币原理详解
一、什么是比特币 比特币是一种电子货币,是一种基于密码学的货币,在2008年11月1日由中本聪发表比特币白皮书,文中提出了一种去中心化的电子记账系统,我们平时的电子现金是银行来记账,因为银行的背后是国家信用。去中心化电子记账系统是参与者共同记账。比特币可以防止主权危机、信用风险。其好处不多做赘述,这一层面介绍的文章很多,本文主要从更深层的技术原理角度进行介绍。 二、问题引入 假设现有4个人...
Python 基础(一):入门必备知识
目录1 标识符2 关键字3 引号4 编码5 输入输出6 缩进7 多行8 注释9 数据类型10 运算符10.1 常用运算符10.2 运算符优先级 1 标识符 标识符是编程时使用的名字,用于给变量、函数、语句块等命名,Python 中标识符由字母、数字、下划线组成,不能以数字开头,区分大小写。 以下划线开头的标识符有特殊含义,单下划线开头的标识符,如:_xxx ,表示不能直接访问的类属性,需通过类提供
这30个CSS选择器,你必须熟记(上)
关注前端达人,与你共同进步CSS的魅力就是让我们前端工程师像设计师一样进行网页的设计,我们能轻而易举的改变颜色、布局、制作出漂亮的影音效果等等,我们只需要改几行代码,不需...
国产开源API网关项目进入Apache孵化器:APISIX
点击蓝色“程序猿DD”关注我回复“资源”获取独家整理的学习资料!近日,又有一个开源项目加入了这个Java开源界大名鼎鼎的Apache基金会,开始进行孵化器。项目名称:AP...
程序员接私活怎样防止做完了不给钱?
首先跟大家说明一点,我们做 IT 类的外包开发,是非标品开发,所以很有可能在开发过程中会有这样那样的需求修改,而这种需求修改很容易造成扯皮,进而影响到费用支付,甚至出现做完了项目收不到钱的情况。 那么,怎么保证自己的薪酬安全呢? 我们在开工前,一定要做好一些证据方面的准备(也就是“讨薪”的理论依据),这其中最重要的就是需求文档和验收标准。一定要让需求方提供这两个文档资料作为开发的基础。之后开发
网页实现一个简单的音乐播放器(大佬别看。(⊙﹏⊙))
今天闲着无事,就想写点东西。然后听了下歌,就打算写个播放器。 于是乎用h5 audio的加上js简单的播放器完工了。 欢迎 改进 留言。 演示地点跳到演示地点 html代码如下`&lt;!DOCTYPE html&gt; &lt;html&gt; &lt;head&gt; &lt;title&gt;music&lt;/title&gt; &lt;meta charset="utf-8"&gt
Python十大装B语法
Python 是一种代表简单思想的语言,其语法相对简单,很容易上手。不过,如果就此小视 Python 语法的精妙和深邃,那就大错特错了。本文精心筛选了最能展现 Python 语法之精妙的十个知识点,并附上详细的实例代码。如能在实战中融会贯通、灵活使用,必将使代码更为精炼、高效,同时也会极大提升代码B格,使之看上去更老练,读起来更优雅。 1. for - else 什么?不是 if 和 else 才
数据库优化 - SQL优化
前面一篇文章从实例的角度进行数据库优化,通过配置一些参数让数据库性能达到最优。但是一些“不好”的SQL也会导致数据库查询变慢,影响业务流程。本文从SQL角度进行数据库优化,提升SQL运行效率。 判断问题SQL 判断SQL是否有问题时可以通过两个表象进行判断: 系统级别表象 CPU消耗严重 IO等待严重 页面响应时间过长
2019年11月中国大陆编程语言排行榜
2019年11月2日,我统计了某招聘网站,获得有效程序员招聘数据9万条。针对招聘信息,提取编程语言关键字,并统计如下: 编程语言比例 rank pl_ percentage 1 java 33.62% 2 c/c++ 16.42% 3 c_sharp 12.82% 4 javascript 12.31% 5 python 7.93% 6 go 7.25% 7
通俗易懂地给女朋友讲:线程池的内部原理
餐厅的约会 餐盘在灯光的照耀下格外晶莹洁白,女朋友拿起红酒杯轻轻地抿了一小口,对我说:“经常听你说线程池,到底线程池到底是个什么原理?”我楞了一下,心里想女朋友今天是怎么了,怎么突然问出这么专业的问题,但做为一个专业人士在女朋友面前也不能露怯啊,想了一下便说:“我先给你讲讲我前同事老王的故事吧!” 大龄程序员老王 老王是一个已经北漂十多年的程序员,岁数大了,加班加不动了,升迁也无望,于是拿着手里
经典算法(5)杨辉三角
杨辉三角 是经典算法,这篇博客对它的算法思想进行了讲解,并有完整的代码实现。
编写Spring MVC控制器的14个技巧
本期目录 1.使用@Controller构造型 2.实现控制器接口 3.扩展AbstractController类 4.为处理程序方法指定URL映射 5.为处理程序方法指定HTTP请求方法 6.将请求参数映射到处理程序方法 7.返回模型和视图 8.将对象放入模型 9.处理程序方法中的重定向 10.处理表格提交和表格验证 11.处理文件上传 12.在控制器中自动装配业务类 ...
腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹?
昨天,有网友私信我,说去阿里面试,彻底的被打击到了。问了为什么网上大量使用ThreadLocal的源码都会加上private static?他被难住了,因为他从来都没有考虑过这个问题。无独有偶,今天笔者又发现有网友吐槽了一道腾讯的面试题,我们一起来看看。 腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹? 在互联网职场论坛,一名程序员发帖求助到。二面腾讯,其中一个算法题:64匹
面试官:你连RESTful都不知道我怎么敢要你?
面试官:了解RESTful吗? 我:听说过。 面试官:那什么是RESTful? 我:就是用起来很规范,挺好的 面试官:是RESTful挺好的,还是自我感觉挺好的 我:都挺好的。 面试官:… 把门关上。 我:… 要干嘛?先关上再说。 面试官:我说出去把门关上。 我:what ?,夺门而去 文章目录01 前言02 RESTful的来源03 RESTful6大原则1. C-S架构2. 无状态3.统一的接
求小姐姐抠图竟遭白眼?痛定思痛,我决定用 Python 自力更生!
点击蓝色“Python空间”关注我丫加个“星标”,每天一起快乐的学习大家好,我是 Rocky0429,一个刚恰完午饭,正在用刷网页浪费生命的蒟蒻...一堆堆无聊八卦信息的网页内容慢慢使我的双眼模糊,一个哈欠打出了三斤老泪,就在此时我看到了一张图片:是谁!是谁把我女朋友的照片放出来的!awsl!太好看了叭...等等,那个背景上的一堆鬼画符是什么鬼?!真是看不下去!叔叔婶婶能忍,隔壁老王的三姨妈的四表...
为啥国人偏爱Mybatis,而老外喜欢Hibernate/JPA呢?
关于SQL和ORM的争论,永远都不会终止,我也一直在思考这个问题。昨天又跟群里的小伙伴进行了一番讨论,感触还是有一些,于是就有了今天这篇文。 声明:本文不会下关于Mybatis和JPA两个持久层框架哪个更好这样的结论。只是摆事实,讲道理,所以,请各位看官勿喷。 一、事件起因 关于Mybatis和JPA孰优孰劣的问题,争论已经很多年了。一直也没有结论,毕竟每个人的喜好和习惯是大不相同的。我也看
SQL-小白最佳入门sql查询一
不要偷偷的查询我的个人资料,即使你再喜欢我,也不要这样,真的不好;
项目中的if else太多了,该怎么重构?
介绍 最近跟着公司的大佬开发了一款IM系统,类似QQ和微信哈,就是聊天软件。我们有一部分业务逻辑是这样的 if (msgType = "文本") { // dosomething } else if(msgType = "图片") { // doshomething } else if(msgType = "视频") { // doshomething } else { // doshom...
致 Python 初学者
欢迎来到“Python进阶”专栏!来到这里的每一位同学,应该大致上学习了很多 Python 的基础知识,正在努力成长的过程中。在此期间,一定遇到了很多的困惑,对未来的学习方向感到迷茫。我非常理解你们所面临的处境。我从2007年开始接触 python 这门编程语言,从2009年开始单一使用 python 应对所有的开发工作,直至今天。回顾自己的学习过程,也曾经遇到过无数的困难,也曾经迷茫过、困惑过。开办这个专栏,正是为了帮助像我当年一样困惑的 Python 初学者走出困境、快速成长。希望我的经验能真正帮到你
“狗屁不通文章生成器”登顶GitHub热榜,分分钟写出万字形式主义大作
一、垃圾文字生成器介绍 最近在浏览GitHub的时候,发现了这样一个骨骼清奇的雷人项目,而且热度还特别高。 项目中文名:狗屁不通文章生成器 项目英文名:BullshitGenerator 根据作者的介绍,他是偶尔需要一些中文文字用于GUI开发时测试文本渲染,因此开发了这个废话生成器。但由于生成的废话实在是太过富于哲理,所以最近已经被小伙伴们给玩坏了。 他的文风可能是这样的: 你发现,...
程序员:我终于知道post和get的区别
是一个老生常谈的话题,然而随着不断的学习,对于以前的认识有很多误区,所以还是需要不断地总结的,学而时习之,不亦说乎
《程序人生》系列-这个程序员只用了20行代码就拿了冠军
你知道的越多,你不知道的越多 点赞再看,养成习惯GitHub上已经开源https://github.com/JavaFamily,有一线大厂面试点脑图,欢迎Star和完善 前言 这一期不算《吊打面试官》系列的,所有没前言我直接开始。 絮叨 本来应该是没有这期的,看过我上期的小伙伴应该是知道的嘛,双十一比较忙嘛,要值班又要去帮忙拍摄年会的视频素材,还得搞个程序员一天的Vlog,还要写BU...
相关热词 c#处理浮点数 c# 生成字母数字随机数 c# 动态曲线 控件 c# oracle 开发 c#选择字体大小的控件 c# usb 批量传输 c#10进制转8进制 c#转base64 c# 科学计算 c#下拉列表获取串口
立即提问