Hbase连接报错RpcRetryingCaller 5C

16/12/22 10:13:42 INFO client.RpcRetryingCaller: Call exception, tries=10, retries=35, retryTime=46200ms, msg=row 'factoryBatteryData,123456789,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=localhost,57557,1482371106831, seqNum=0

在hbase中插入数据时候,报错如上。

2个回答

应该是hots 文件中没有配置 虚拟机 ip 的映射 你试一下

全部日志啊,找帮助日志都不会取?无语
错误信息你都没有复制出来。。。。。。。。。。。。。。

tuyim7124
yyw_一骑绝尘 怎么都有问题?没有解决方法的分享
大约 2 年之前 回复
qq1977261418
懂什么 回复水田如雅: 怎么解决呀 谢谢了
大约 2 年之前 回复
qq_19968255
李孟lm 回复水田如雅: 怎么解决的?
大约 2 年之前 回复
xinxian13
xinxian13 回复水田如雅: 咋弄的
2 年多之前 回复
qq_34134798
qq_34134798 回复水田如雅: 求解决方法
2 年多之前 回复
qq_31816085
一只大黄 回复水田如雅: 在嘛,请问下这个问题是怎么解决的?如果方便可以回复我一下吗?
2 年多之前 回复
qq_32931423
七大罪 回复水田如雅: 这个怎么解决的 我是查询的时候出现这个问题
大约 3 年之前 回复
lhc2207221755
水田如雅 大哥,已经搞定啦
大约 3 年之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
hbase启动报错 hbase shell
请大神帮帮忙 hbase启动 OK hbase shell有下问题 # ./bin/hbase shell 2016-04-05 08:53:06,328 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts 2016-04-05 08:53:06,331 WARN [main] zookeeper.ZKUtil: hconnection-0x1f6917fb0x0, quorum=salve1:2181,master:2181,salve2:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:482) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450) at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362) at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:58) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169) at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:148) at org.jruby.RubyClass.newInstance(RubyClass.java:822) at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535) at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrNBlock.call(JavaMethod.java:249) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.__file__(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb:118) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.load(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb) at org.jruby.Ruby.runScript(Ruby.java:697) at org.jruby.Ruby.runScript(Ruby.java:690) at org.jruby.Ruby.runNormally(Ruby.java:597) at org.jruby.Ruby.runFromMain(Ruby.java:446) at org.jruby.Main.doRunFromMain(Main.java:369) at org.jruby.Main.internalRun(Main.java:258) at org.jruby.Main.run(Main.java:224) at org.jruby.Main.run(Main.java:208) at org.jruby.Main.main(Main.java:188) 2016-04-05 08:53:06,338 ERROR [main] zookeeper.ZooKeeperWatcher: hconnection-0x1f6917fb0x0, quorum=salve1:2181,master:2181,salve2:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:482) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450) at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362) at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:58) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169) at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:148) at org.jruby.RubyClass.newInstance(RubyClass.java:822) at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535) at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrNBlock.call(JavaMethod.java:249) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.__file__(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb:118) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.load(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb) at org.jruby.Ruby.runScript(Ruby.java:697) at org.jruby.Ruby.runScript(Ruby.java:690) at org.jruby.Ruby.runNormally(Ruby.java:597) at org.jruby.Ruby.runFromMain(Ruby.java:446) at org.jruby.Main.doRunFromMain(Main.java:369) at org.jruby.Main.internalRun(Main.java:258) at org.jruby.Main.run(Main.java:224) at org.jruby.Main.run(Main.java:208) at org.jruby.Main.main(Main.java:188) HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 1.0.3, rf1e1312f9790a7c40f6a4b5a1bab2ea1dd559890, Tue Jan 19 19:26:53 PST 2016
java连接hbase Kerberos 24小时报错问题
java连接hbase kerberos 24小时过期问题。求指点 ``` try { String krbStr = Thread.currentThread().getContextClassLoader().getResource("krb5.ini").getFile(); String keyStr = Thread.currentThread().getContextClassLoader().getResource(keytab).getFile(); System.setProperty("java.security.krb5.conf", krbStr); UserGroupInformation.setConfiguration(conf); UserGroupInformation.loginUserFromKeytab(principal, keyStr); } catch (IOException e) { log.error(e); } try { HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists(tableName)) { HTableDescriptor tableDescripter = new HTableDescriptor( tableName.getBytes()); tableDescripter.addFamily(new HColumnDescriptor("data")); admin.createTable(tableDescripter); } } catch (Exception e) { log.error( e); } ``` 以上代码刚运行时没问题,但是过了24小时左右admin.tableExists(tableName)这句就会报错,错误如下 org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Sat Jul 13 14:30:29 CST 2019, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68090: row 'p_rsdisk,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=xx-xx-xx-xx-xx.indata.com,16020,1560266314048, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:271) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:223) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155) at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811) at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602) at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:303) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:313) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:436) at org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:177) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=68090: row 'p_rsdisk,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=xx-xx-xx-xx-xx.indata.com,16020,1560266314048, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159) at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: java.io.IOException: Could not set up IO Streams to xx-xx-xx-xx-xx.indata.com/xx.xx.xx.xx:16020 at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:777) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651) at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:372) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:199) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:369) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:343) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) ... 4 more Caused by: java.lang.RuntimeException: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$1.run(RpcClientImpl.java:677) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:635) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:743) ... 17 more Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732) ... 17 more Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 26 more 求解原因,是因为票据过期吗?
hbase启动报错问题,求解
hbase集群启动通过ssh启动会爆下面这个错 java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2706) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2721) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2704) ... 5 more Caused by: java.io.IOException: No FileSystem for scheme: hdfs at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356) at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:1003) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:579) 通过自定义脚本或者 bin/start-hbase.sh 启动会报错,但通过$ bin/hbase-daemon.sh start master $ bin/hbase-daemon.sh start regionserver 这命令单独在各个机子上运行则不报,求解!!!!
hbase mapreduce 报错 java.lang.NullPointerException
http://bbs.csdn.net/topics/390865764 这篇文章出错相似,求教大牛们 2017-09-15 23:19:15 [WARN]-[] Your hostname, admin-PC resolves to a loopback/non-reachable address: fe80:0:0:0:0:5efe:c0a8:164%23, but we couldn't find any external IP address! 2017-09-15 23:19:15 [INFO]-[org.apache.hadoop.conf.Configuration.deprecation] session.id is deprecated. Instead, use dfs.metrics.session-id 2017-09-15 23:19:15 [INFO]-[org.apache.hadoop.metrics.jvm.JvmMetrics] Initializing JVM Metrics with processName=JobTracker, sessionId= Exception in thread "main" java.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010) at org.apache.hadoop.util.Shell.runCommand(Shell.java:487) at org.apache.hadoop.util.Shell.run(Shell.java:460) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:720) at org.apache.hadoop.util.Shell.execCommand(Shell.java:813) at org.apache.hadoop.util.Shell.execCommand(Shell.java:796) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:656) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:444) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:308) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:147) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325) at TestOnlyMapper.main(TestOnlyMapper.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) ----------------------分割线 代码------------------------------------------------- import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.Cell; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil; import org.apache.hadoop.hbase.mapreduce.TableMapper; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.lib.output.NullOutputFormat; import java.io.IOException; /** * Created by admin on 2017/9/15. */ public class TestOnlyMapper { public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.rootdir","hdfs://hadoop.master:8020/hdfs/hbase"); conf.set("hbase.zookeeper.quorum","hadoop.master,hadoop.slave11,hadoop.slave12"); conf.set("hbase.zookeeper.property.clientPort","2181"); Job job= new Job(conf,"test"); job.setJarByClass(TestOnlyMapper.class); Scan scan = new Scan(); job.setMapSpeculativeExecution(false); job.setReduceSpeculativeExecution(false); TableMapReduceUtil.initTableMapperJob("test11",scan,OMapper.class,null,null,job); job.setOutputFormatClass(NullOutputFormat.class); job.waitForCompletion(true); } } class OMapper extends TableMapper<Text, LongWritable> { @Override protected void map(ImmutableBytesWritable key, Result value, Context context) throws IOException, InterruptedException { for(Cell cell:value.listCells()) { System.out.println("---------------------"); System.out.println("cell.getQualifier()= "+cell.getQualifier().toString()); System.out.println("---------------------"); } } }
连接hbase Connection refused: no further information
server环境: 在virtualbox里安装了ubuntu,然后装了了hadoop2.7.5,hbase1.3.1,然后hadoop和hbase的环境都配置好了 启动了hbase,然后本机里通过scala访问虚拟机里的hbase时,connection能获取到,但是在执行tableExists时,卡一会后就报超时了,client的代码和server的配置如: server配置(其中zookeeper用的hbase自己管理的): hbase-site.xml ![图片说明](https://img-ask.csdn.net/upload/201804/02/1522641879_341754.png) client的代码: ``` class HbaseUtil { def GetHbaseConfiguration(ip:String):Unit={ var conf = HBaseConfiguration.create; //这个是远程hbase的ip地址 conf.set("hbase.zookeeper.quorum",ip) //conf.set("zookeeper.znode.parent", "/hbase-unsecure") //2181是hbase里zookeeper的默认端口号 conf.set("hbase.zookeeper.property.clientPort","2181") println(ip+":habse connention success...") val tableName = "jndata" conf.set(TableInputFormat.INPUT_TABLE, tableName) val con= ConnectionFactory.createConnection(conf) val hBaseAdmin = con.getAdmin//new HBaseAdmin(conf); val tn = TableName.valueOf(tableName); // hBaseAdmin.disableTable(tn) // println(tableName + " is exist,detele....") //创建表 if (hBaseAdmin.tableExists(tn)) { println(tableName + " is exist,....") }else{ println(tableName + " is not exist,....") } } } ``` 然后server端的zookeeper的log如下: ![图片说明](https://img-ask.csdn.net/upload/201804/02/1522640993_389812.png) client端的异常如下: ``` Exception in thread "main" org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Mon Apr 02 11:30:11 CST 2018, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=76915: Connection refused: no further information row 'jndata,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=simon-virtualbox,16201,1522631406769, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:276) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:210) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:314) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:289) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:164) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:159) at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:796) at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602) at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:408) at rxb.flinkDemo.hbase.HbaseUtil.GetHbaseConfiguration(HbaseUtil.scala:42) at rxb.flinkDemo.MyDemo$.main(FlinkDemo.scala:27) at rxb.flinkDemo.MyDemo.main(FlinkDemo.scala) Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=76915: Connection refused: no further information row 'jndata,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=simon-virtualbox,16201,1522631406769, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:171) at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnection(RpcClientImpl.java:416) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:722) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:909) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1244) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:35396) at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:404) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:211) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:65) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:364) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:338) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:137) ... 4 more ``` 好多办法都试过了还是不行: 1.server和client的防火墙都关了 2.client的host里也配置了server的ip和hostname
Hbase链接数据库时,在循环遍历result时,前四次遍历取数正常,第5次报错
代码如下: ![图片说明](https://img-ask.csdn.net/upload/201710/31/1509412787_260791.png) p://img.ask.csdn.net/upload/201710/31/1509412397_714565.png) ## 报错异常如下:,,,求大神帮忙 1. java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Tue Oct 31 09:03:17 CST 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60108: row '10.10.49.161_150935' on table 'HOST_RESRC' at region=HOST_RESRC,10.10.47.55_1507847553,1509329651091.3ede1b811762ae5d54b7575a55f26a5a., hostname=node1,16020,1509329624220, seqNum=63801830 at org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:97) at com.nantian.zabbix.service.impl.TestHbase.querybasic(TestHbase.java:135) at com.nantian.zabbix.service.impl.TestHbase.main(TestHbase.java:61) Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Tue Oct 31 09:03:17 CST 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60108: row '10.10.49.161_150935' on table 'HOST_RESRC' at region=HOST_RESRC,10.10.47.55_1507847553,1509329651091.3ede1b811762ae5d54b7575a55f26a5a., hostname=node1,16020,1509329624220, seqNum=63801830 at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:271) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:210) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:413) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:371) at org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:94) ... 2 more Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=60108: row '10.10.49.161_150935' on table 'HOST_RESRC' at region=HOST_RESRC,10.10.47.55_1507847553,1509329651091.3ede1b811762ae5d54b7575a55f26a5a., hostname=node1,16020,1509329624220, seqNum=63801830 at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:169) at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Call to node1/10.10.49.182:16020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=3, waitTime=60001, operationTimeout=60000 expired. at org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:284) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1238) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:328) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32831) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:215) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:63) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:364) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:338) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136) ... 4 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=3, waitTime=60001, operationTimeout=60000 expired. at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1212) ... 13 more
hive on hbase报错,在hive中创建映射表,关联到hbase上
hive on hbase,在hive中创建映射表,关联到hbase上,在hive中已经创建hbase中的row_key,还是报错FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=xlwang5, access=EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--T
通过hue 执行hivesql 报错
我用hue 执行hivesql 的时候系统报错, java.net.SocketTimeoutException:callTimeout=60000, callDuration=68043: row 'log,,00000000000000' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=node4,16020,1476410081203, seqNum=0:5:1", 'org.apache.hadoop.hbase.client.RpcRetryingCaller:callWithRetries:RpcRetryingCaller.java:159', 'org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture:run:ResultBoundedCompletionService.java:64', '*org.apache.hadoop.hbase.exceptions.ConnectionClosingException:Call to node4/192.168.127.1:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to node4/192.168.127.1:16020 is closing. Call id=9, waitTime=1:16:11', 'org.apache.hadoop.hbase.ipc.RpcClientImpl:wrapException:RpcClientImpl.java:1239', 'org.apache.hadoop.hbase.ipc.RpcClientImpl:call:RpcClientImpl.java:1210', 'org.apache.hadoop.hbase.ipc.AbstractRpcClient:callBlockingMethod:AbstractRpcClient.java:213', 'org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation:callBlockingMethod:AbstractRpcClient.java:287', 'org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub:scan:ClientProtos.java:32651', 'org.apache.hadoop.hbase.client.ScannerCallable:openScanner:ScannerCallable.java:372', 'org.apache.hadoop.hbase.client.ScannerCallable:call:ScannerCallable.java:199', 'org.apache.hadoop.hbase.client.ScannerCallable:call:ScannerCallable.java:62', 'org.apache.hadoop.hbase.client.RpcRetryingCaller:callWithoutRetries:RpcRetryingCaller.java:200', 'org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC:call:ScannerCallableWithReplicas.java:369', 'org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC:call:ScannerCallableWithReplicas.java:343', 'org.apache.hadoop.hbase.client.RpcRetryingCaller:callWithRetries:RpcRetryingCaller.java:126', '*org.apache.hadoop.hbase.exceptions.ConnectionClosingException:Connection to node4/192.168.127.1:16020 is closing. Call id=9, waitTime=1:3:2', 'org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection:cleanupCalls:RpcClientImpl.java:1037', 'org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection:close:RpcClientImpl.java:844', 'org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection:run:RpcClientImpl.java:572'], statusCode=3), results=None, hasMoreRows=None) 但是我用hive shell 执行 sql 一切都正常
求助:kettle连接cdh510-hbase报错问题
最近在用hbase与oracle的数据搬运工作,之前一直用sqoop,最近用kettle在做这个事情,在windows,os环境测试一切正常,包括客户端直接跑Job,和生成ktr脚本去跑,都没问题,但是我把生成的.ktr文件拿到ubuntu跑,就报错 org.pentaho.di.core.exception.KettleException: Unable to obtain a connection to HBase: null 如下图 ![图片说明](https://img-ask.csdn.net/upload/201707/12/1499870023_502959.png)
java调用Hbase java.net.SocketTimeoutException
hbase菜鸟,工作需要,请大神帮忙。java连接hbase出现的错误error如下: log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Thu Apr 07 09:43:39 CST 2016, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=80156: row 'gpsinfo,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=zpu-storage3,16020,1443175301611, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:271) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:207) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:401) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:364) at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:604) at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:392) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:402) at Test.main(Test.java:19) Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=80156: row 'gpsinfo,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=zpu-storage3,16020,1443175301611, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159) at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.net.UnknownHostException: unknown host: zpu-storage3 at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.<init>(RpcClientImpl.java:303) at org.apache.hadoop.hbase.ipc.RpcClientImpl.createConnection(RpcClientImpl.java:133) at org.apache.hadoop.hbase.ipc.RpcClientImpl.getConnection(RpcClientImpl.java:1283) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1191) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:222) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:323) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32831) at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:373) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:200) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:360) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:334) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) ... 4 more
Hbase zookeeper好像是连接报错,求帮忙= =
hbase(main):002:0> create 'test','info' 2017-11-26 11:07:32,602 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2017-11-26 11:07:49,822 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts 2017-11-26 11:07:49,825 WARN [main] zookeeper.ZKUtil: hconnection-0x42b6d0cc, quorum=liubaoxing-precision-workstation-t7500:2182,liubaoxing-slave:2182, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:839) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:642) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:411) at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:390) at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:271) at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:195) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.jruby.javasupport.JavaConstructor.newInstanceDirect(JavaConstructor.java:275) at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:91) at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:178) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:182) at org.jruby.java.proxies.ConcreteJavaProxy$2.call(ConcreteJavaProxy.java:48) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:182) at org.jruby.RubyClass.newInstance(RubyClass.java:829) at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535) at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrNBlock.call(JavaMethod.java:266) at org.jruby.java.proxies.ConcreteJavaProxy$3.call(ConcreteJavaProxy.java:144) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169) at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:255) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:223) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:342) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:212) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:216) at org.jruby.RubyClass.newInstance(RubyClass.java:836) at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535) at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrTwoOrNBlock.call(JavaMethod.java:283) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203) at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169) at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95) at org.jruby.ast.OpAsgnOrNode.interpret(OpAsgnOrNode.java:100) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:147) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:183) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135) at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:63) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:147) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:183) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135) at org.jruby.ast.VCallNode.interpret(VCallNode.java:86) at org.jruby.ast.CallSpecialArgNode.interpret(CallSpecialArgNode.java:57) at org.jruby.ast.DAsgnNode.interpret(DAsgnNode.java:110) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111) at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374) at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295) at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229) at org.jruby.runtime.Block.yieldSpecific(Block.java:99) at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144) at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153) at org.jruby.ast.FCallNoArgBlockNode.interpret(FCallNoArgBlockNode.java:32) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:255) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:223) at org.jruby.RubyClass.finvoke(RubyClass.java:611) at org.jruby.RubyBasicObject.send(RubyBasicObject.java:2787) at org.jruby.RubyKernel.send(RubyKernel.java:2113) at org.jruby.RubyKernel$s$send.call(RubyKernel$s$send.gen:65535) at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrTwoOrThreeOrNBlock.call(JavaMethod.java:300) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:352) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:237) at org.jruby.ast.FCallSpecialArgNode.interpret(FCallSpecialArgNode.java:43) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111) at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374) at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295) at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229) at org.jruby.runtime.Block.yieldSpecific(Block.java:99) at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
scala编写从hdfs文件取数据通过phoenix将数据批量插入到hbase,报错对象没有序列化
求大牛指导指导问题所在 批量插入的方法如下 class PhoenixClient_v2(host:String,port:String) extends Serializable { try{ Class.forName("org.apache.phoenix.jdbc.PhoenixDriver") } catch{ case e :ClassNotFoundException => println(e) } def InsertOfBatch(phoenixSQL:String,Key:Array[String],Column:Array[String],filepath:String=null,dataArr:Array[Map[String,Any]]=null) = { val sparkConf = new SparkConf().setMaster("local").setAppName("hdfs2hbase") val sc = new SparkContext(sparkConf) //从json格式的hdfs文件中读取 if (filepath != null) { val url = "jdbc:phoenix:" + host + ":" + port val conn = DriverManager.getConnection(url) val ps = conn.prepareStatement(phoenixSQL) try { //利用spark从文件系统或者hdfs上读取文件 val Rdd_file = sc.textFile(filepath) var HashKey:String = (hashCode()%100).toString //从rdd中取出相应数据到ps中 Rdd_file.map{y=> Key.foreach(x => if (x == "Optime") HashKey += "|"+new JSONObject(y).getString(x).split(" ")(0) else HashKey += "|"+new JSONObject(y).getString(x)) ps.setString(1,HashKey) Column.foreach(x => ps.setString(Column.indexOf(x) + 2, new JSONObject(y).getString(x))) ps.addBatch() } ps.executeBatch() conn.commit() } catch{ case e: SQLException => println("Insert into phoenix from file failed:" + e.toString) }finally { conn.close() ps.close() } } //读取Map[String,Any]映射的集合 if (dataArr != null){ val url = "jdbc:phoenix:" + host + ":" + port val conn = DriverManager.getConnection(url) val ps = conn.prepareStatement(phoenixSQL) try { val Rdd_Arr = sc.parallelize(dataArr) var HashKey:String = (hashCode()%100).toString Rdd_Arr.map{y=> Key.foreach(x => if (x == "Optime") HashKey += "|"+y.get(x).toString.split(" ")(0) else HashKey += "|"+y.get(x) ) ps.setString(1,HashKey) Column.foreach(x => ps.setString(Column.indexOf(x) + 2, y.get(x).toString)) ps.addBatch() } ps.executeBatch() conn.commit() } catch{ case e: SQLException => println("Insert into phoenix from file failed:" + e.toString) }finally { conn.close() ps.close() } } } } 调用的方式如下 object PhoenixTest{ def main(args: Array[String]): Unit = { val pc = new PhoenixClient_v2("zk1","2181") println("select result :"+pc.Select_Sql("select * from \"OP_record\"").toString) pc.CreateTable("create table if not exists \"OP_record\"(\"pk\" varchar(100) not null primary key, \"cf1\".\"curdonate\" varchar(20),\"cf1\".\"curcharge\" varchar(20),\"cf1\".\"uniqueid\" varchar(20))","OP_record") val KeyArr = Array("serverid","userid","Optime") val ColumnArr = Array("curdonate","curcharge","uniqueid") pc.InsertOfBatch("upsert into \"OP_record\" values(?,?,?,?)",KeyArr,ColumnArr,"hdfs://zk1:9000/chen/hive_OP_record_diamonds_20170702.log.1499028228.106") } } 报错信息如下: Exception in thread "main" org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:304) at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:294) at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:122) at org.apache.spark.SparkContext.clean(SparkContext.scala:2032) at org.apache.spark.rdd.RDD$$anonfun$map$1.apply(RDD.scala:318) at org.apache.spark.rdd.RDD$$anonfun$map$1.apply(RDD.scala:317) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) at org.apache.spark.rdd.RDD.withScope(RDD.scala:310) at org.apache.spark.rdd.RDD.map(RDD.scala:317) at scala.Hbase.PhoenixClient_v2.InsertOfBatch(PhoenixClient_v2.scala:123) at scala.Hbase.PhoenixTest$.main(PhoenixClient_v2.scala:230) at scala.Hbase.PhoenixTest.main(PhoenixClient_v2.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:674) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.io.NotSerializableException: org.apache.phoenix.jdbc.PhoenixPreparedStatement Serialization stack: - object not serializable (class: org.apache.phoenix.jdbc.PhoenixPreparedStatement, value: upsert into "OP_record" values(?,?,?,?)) - field (class: scala.Hbase.PhoenixClient_v2$$anonfun$InsertOfBatch$1, name: ps$1, type: interface java.sql.PreparedStatement) - object (class scala.Hbase.PhoenixClient_v2$$anonfun$InsertOfBatch$1, <function1>) at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40) at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47) at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84) at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:301) ... 21 more 17/07/06 16:35:26 INFO SparkContext: Invoking stop() from shutdown hook 17/07/06 16:35:26 INFO SparkUI: Stopped Spark web UI at http://192.168.12.243:4040 17/07/06 16:35:26 INFO DAGScheduler: Stopping DAGScheduler 17/07/06 16:35:27 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped! 17/07/06 16:35:27 INFO MemoryStore: MemoryStore cleared 17/07/06 16:35:27 INFO BlockManager: BlockManager stopped 17/07/06 16:35:27 INFO BlockManagerMaster: BlockManagerMaster stopped 17/07/06 16:35:27 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped! 17/07/06 16:35:27 INFO SparkContext: Successfully stopped SparkContext 17/07/06 16:35:27 INFO ShutdownHookManager: Shutdown hook called 17/07/06 16:35:27 INFO ShutdownHookManager: Deleting directory /tmp/spark-18cc3337-cdda-498f-a7f3-124f239a6bfe
windows 下启动hbase报错
D:\TSBrowserDownloads\db\hbase-2.1.0-bin\hbase-2.1.0\bin>start-hbase.cmd Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:178) at org.apache.hadoop.hbase.util.HBaseConfTool.main(HBaseConfTool.java:39) Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 2 more ERROR: Could not determine the startup mode. ---------------------------------------------------
hbase建表失败报错:ERROR: The procedure 22330 is still running
在一次停电后hbase 发生了block丢失的异常,检查后发现是WELs里的两个block处于分裂状态,经过多次修复命令无果后,使用 hdfs fsck -delete 将缺损的删除了,在删除后表ppr数据无法查询 , 正常删表失败后 , 选择了暴力删除 , 将hdfs上的datadir删除,meta表里的记录删除,zookeepercli里/hbase/table/ppr删除,然后进行了集群的重启,重启后在hbase shell里进行了list,没有ppr表了,再次创建出现了如下的报错: ``` hbase(main):001:0> create "ppr" ,"f" ERROR: The procedure 22330 is still running For usage try 'help "create"' Took 669.0389 seconds ``` 经过排查是Regions in Transition中有一个该表的region: 273f6a209c5e2bcad57db5e0e18b4028 ppr OFFLINE 19064 WAITING_TIMEOUT 但是这个region我该怎么删掉,在hdfs都已经把表删了 大神们,求解决方式
java连接单机hbase操作数据
16/03/04 17:09:56 INFO support.ClassPathXmlApplicationContext: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1f6ae4d: startup date [Fri Mar 04 17:09:56 CST 2016]; root of context hierarchy 16/03/04 17:09:56 INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from class path resource [applicationContest.xml] 16/03/04 17:09:57 INFO support.DefaultListableBeanFactory: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1d8fe20: defining beans [hbaseConfiguration,htemplate]; root of factory hierarchy 16/03/04 17:09:57 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16/03/04 17:09:57 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xc4dc7c connecting to ZooKeeper ensemble=192.168.1.202:2181 16/03/04 17:09:57 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.2.0--1, built on 10/11/2014 20:49 GMT 16/03/04 17:09:57 INFO zookeeper.ZooKeeper: Client environment:host.name=xiaoming 16/03/04 17:09:57 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_17 16/03/04 17:09:57 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 16/03/04 17:09:57 INFO zookeeper.ZooKeeper: Client environment:java.home=C:\Program Files (x86)\Java\jdk1.7.0_17\jre 16/03/04 17:09:57 INFO zookeeper.ZooKeeper: Client environment:java.class.path=F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\classes;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\commons-codec-1.7.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\commons-collections-3.2.1.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\commons-configuration-1.6.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\commons-lang-2.6.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\commons-logging-1.1.1.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\guava-12.0.1.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\hadoop-auth-2.5.0-cdh5.2.0.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\hadoop-common-2.5.0-cdh5.2.0.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\hadoop-core-2.5.0-mr1-cdh5.2.0.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\hbase-client-0.98.6-cdh5.2.0.jar;F:\mess\12Hadoop\workspace\myhbasetest\WebRoot\WEB-INF\lib\hbase-common-0.98.6-cdh5.2.0.jar;F:\mess\12Hadoop\workspace....... 16/03/04 17:09:57 INFO zookeeper.ClientCnxn: Opening socket connection to server 192.168.1.202/192.168.1.202:2181. Will not attempt to authenticate using SASL (unknown error) 16/03/04 17:09:57 INFO zookeeper.ClientCnxn: Socket connection established to 192.168.1.202/192.168.1.202:2181, initiating session 16/03/04 17:09:57 INFO zookeeper.ClientCnxn: Session establishment complete on server 192.168.1.202/192.168.1.202:2181, sessionid = 0x15341c9328d0012, negotiated timeout = 40000 16/03/04 17:17:50 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=31, exceptions: .......Fri Mar 04 17:17:50 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@b7b17a, java.net.UnknownHostException: unknown host: hbase ......at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:129) at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:714) at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:144) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1140) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1204) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1092) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1049) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:890) at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:72) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:780) at org.springframework.data.hadoop.hbase.HbaseTemplate$2.doInTable(HbaseTemplate.java:182) at org.springframework.data.hadoop.hbase.HbaseTemplate.execute(HbaseTemplate.java:58) at org.springframework.data.hadoop.hbase.HbaseTemplate.get(HbaseTemplate.java:168) at org.springframework.data.hadoop.hbase.HbaseTemplate.get(HbaseTemplate.java:158) at com.bw.test.Mytest.get(Mytest.java:46) at com.bw.test.Mytest.main(Mytest.java:40) Caused by: java.net.UnknownHostException: unknown host: hbase at org.apache.hadoop.hbase.ipc.RpcClient$Connection.<init>(RpcClient.java:385) at org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:351) at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1530) at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1442) at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1661) at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:29966) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1562) at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:710) at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:708) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114) ... 16 more
hive建hbase关联表报错
hadoop-2.5.2 ,hbase-1.0.1.1,hive-0.9.0 hive> CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz"); FAILED: Error in metadata: MetaException(message:java.io.IOException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240) at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:410) at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:403) at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:281) at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:202) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:398) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:538) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3305) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:242) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1118) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ... 25 more Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:218) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623) ... 30 more Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 36 more ) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
list的时候Hbase报错的问题?
2016-09-21 00:11:28,533 ERROR [main] client.ConnectionManager$HConnectionImplementation: Can't get connection to ZooKeeper: KeeperErrorCode = OperationTimeout ERROR: KeeperErrorCode = OperationTimeout Here is some help for this command: List all tables in hbase. Optional regular expression parameter could be used to filter the output. Examples: hbase> list hbase> list 'abc.*' hbase> list 'ns:abc.*' hbase> list 'ns:.*' hbase(main):002:0> [root@cluster12 bin]#
Hadoop+Hbase报错java.net.UnknownHostException:
问题:java.net.UnknownHostException: Invalid host name: local host is: (unknown); destination host is: "master":9000; java.net.UnknownHostException; log错误日志 2017-07-13 21:26:45,915 FATAL [master:16000.activeMasterManager] master.HMaster: Failed to become active master java.net.UnknownHostException: Invalid host name: local host is: (unknown); destination host is: "master":9000; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:744) at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:409) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1518) at org.apache.hadoop.ipc.Client.call(Client.java:1451) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy18.setSafeMode(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.setSafeMode(ClientNamenodeProtocolTranslatorPB.java:666) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy19.setSafeMode(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279) at com.sun.proxy.$Proxy20.setSafeMode(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.setSafeMode(DFSClient.java:2596) at org.apache.hadoop.hdfs.DistributedFileSystem.setSafeMode(DistributedFileSystem.java:1223) at org.apache.hadoop.hdfs.DistributedFileSystem.setSafeMode(DistributedFileSystem.java:1207) at org.apache.hadoop.hbase.util.FSUtils.isInSafeMode(FSUtils.java:525) at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:971) at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:429) at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:153) at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:128) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:693) at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:189) at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1803) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.UnknownHostException ... 32 more 2017-07-13 21:26:45,924 FATAL [master:16000.activeMasterManager] master.HMaster: Unhandled exception. Starting shutdown. java.net.UnknownHostException: Invalid host name: local host is: (unknown); destination host is: "master":9000; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:744) at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:409) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1518) at org.apache.hadoop.ipc.Client.call(Client.java:1451) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy18.setSafeMode(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.setSafeMode(ClientNamenodeProtocolTranslatorPB.java:666) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy19.setSafeMode(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279) at com.sun.proxy.$Proxy20.setSafeMode(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.setSafeMode(DFSClient.java:2596) at org.apache.hadoop.hdfs.DistributedFileSystem.setSafeMode(DistributedFileSystem.java:1223) at org.apache.hadoop.hdfs.DistributedFileSystem.setSafeMode(DistributedFileSystem.java:1207) at org.apache.hadoop.hbase.util.FSUtils.isInSafeMode(FSUtils.java:525) at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:971) at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:429) at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:153) at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:128) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:693) at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:189) at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1803) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.UnknownHostException ... 32 more 2017-07-13 21:26:45,925 INFO [master:16000.activeMasterManager] regionserver.HRegionServer: STOPPED: Unhandled exception. Starting shutdown. 问题补充 1、防火墙均已关闭、root最高权限 2、hadoop启动正常jps查看已启动,通过浏览器访问50070,8088无任何问题 3、Zookeeper启动正常jps查看已启动 4、已删除hbase/lib下所有关于hadoop的jar并将 hadoop/share所有关于Hadoop的jar拷贝到hbase/lib下,并添加aws-java-sdk-core-1.11.158.jar和aws-java-sdk-s3-1.11.155.jar 版本说明 1、hadoop 2.7.2 2、hbase 1.2.6 3、zookeeper 3.4.2
使用Java对hbase进行连接测试,连接不上,超时,
我在Linux上安装了hbase的单机版,hbase版本是1.2.6, zookeeper使用的是hbase自己的,Linux防火墙开放了16010和2181端口, 使用hbase shell 可以正常操作hbase,浏览器也可以正常访问hbase, 但是我使用Java对hbase进行连接测试的时候就是一直连接不上,然后当我关闭 Linux防火墙的时候java又可以正常的对hbase进行连接访问了,???,这是怎么回事,是因为Linux防火墙还需要开放其他端口吗
相见恨晚的超实用网站
搞学习 知乎:www.zhihu.com 简答题:http://www.jiandati.com/ 网易公开课:https://open.163.com/ted/ 网易云课堂:https://study.163.com/ 中国大学MOOC:www.icourse163.org 网易云课堂:study.163.com 哔哩哔哩弹幕网:www.bilibili.com 我要自学网:www.51zxw
爬虫福利二 之 妹子图网MM批量下载
爬虫福利一:27报网MM批量下载    点击 看了本文,相信大家对爬虫一定会产生强烈的兴趣,激励自己去学习爬虫,在这里提前祝:大家学有所成! 目标网站:妹子图网 环境:Python3.x 相关第三方模块:requests、beautifulsoup4 Re:各位在测试时只需要将代码里的变量 path 指定为你当前系统要保存的路径,使用 python xxx.py 或IDE运行即可。
字节跳动视频编解码面经
引言 本文主要是记录一下面试字节跳动的经历。 三四月份投了字节跳动的实习(图形图像岗位),然后hr打电话过来问了一下会不会opengl,c++,shador,当时只会一点c++,其他两个都不会,也就直接被拒了。 七月初内推了字节跳动的提前批,因为内推没有具体的岗位,hr又打电话问要不要考虑一下图形图像岗,我说实习投过这个岗位不合适,不会opengl和shador,然后hr就说秋招更看重基础。我当时
开源一个功能完整的SpringBoot项目框架
福利来了,给大家带来一个福利。 最近想了解一下有关Spring Boot的开源项目,看了很多开源的框架,大多是一些demo或者是一个未成形的项目,基本功能都不完整,尤其是用户权限和菜单方面几乎没有完整的。 想到我之前做的框架,里面通用模块有:用户模块,权限模块,菜单模块,功能模块也齐全了,每一个功能都是完整的。 打算把这个框架分享出来,供大家使用和学习。 为什么用框架? 框架可以学习整体
源码阅读(19):Java中主要的Map结构——HashMap容器(下1)
(接上文《源码阅读(18):Java中主要的Map结构——HashMap容器(中)》) 3.4.4、HashMap添加K-V键值对(红黑树方式) 上文我们介绍了在HashMap中table数组的某个索引位上,基于单向链表添加新的K-V键值对对象(HashMap.Node&lt;K, V&gt;类的实例),但是我们同时知道在某些的场景下,HashMap中table数据的某个索引位上,数据是按照红黑树
c++制作的植物大战僵尸,开源,一代二代结合游戏
    此游戏全部由本人自己制作完成。游戏大部分的素材来源于原版游戏素材,少部分搜集于网络,以及自己制作。 此游戏为同人游戏而且仅供学习交流使用,任何人未经授权,不得对本游戏进行更改、盗用等,否则后果自负。 目前有六种僵尸和六种植物,植物和僵尸的动画都是本人做的。qq:2117610943 开源代码下载 提取码:3vzm 点击下载--&gt; 11月28日 新增四种植物 统一植物画风,全部修
Java学习的正确打开方式
在博主认为,对于入门级学习java的最佳学习方法莫过于视频+博客+书籍+总结,前三者博主将淋漓尽致地挥毫于这篇博客文章中,至于总结在于个人,实际上越到后面你会发现学习的最好方式就是阅读参考官方文档其次就是国内的书籍,博客次之,这又是一个层次了,这里暂时不提后面再谈。博主将为各位入门java保驾护航,各位只管冲鸭!!!上天是公平的,只要不辜负时间,时间自然不会辜负你。 何谓学习?博主所理解的学习,它
程序员必须掌握的核心算法有哪些?
由于我之前一直强调数据结构以及算法学习的重要性,所以就有一些读者经常问我,数据结构与算法应该要学习到哪个程度呢?,说实话,这个问题我不知道要怎么回答你,主要取决于你想学习到哪些程度,不过针对这个问题,我稍微总结一下我学过的算法知识点,以及我觉得值得学习的算法。这些算法与数据结构的学习大多数是零散的,并没有一本把他们全部覆盖的书籍。下面是我觉得值得学习的一些算法以及数据结构,当然,我也会整理一些看过
Python——画一棵漂亮的樱花树(不同种樱花+玫瑰+圣诞树喔)
最近翻到一篇知乎,上面有不少用Python(大多是turtle库)绘制的树图,感觉很漂亮,我整理了一下,挑了一些我觉得不错的代码分享给大家(这些我都测试过,确实可以生成喔~) one 樱花树 动态生成樱花 效果图(这个是动态的): 实现代码 import turtle as T import random import time # 画樱花的躯干(60,t) def Tree(branch
linux系列之常用运维命令整理笔录
本博客记录工作中需要的linux运维命令,大学时候开始接触linux,会一些基本操作,可是都没有整理起来,加上是做开发,不做运维,有些命令忘记了,所以现在整理成博客,当然vi,文件操作等就不介绍了,慢慢积累一些其它拓展的命令,博客不定时更新 顺便拉下票,我在参加csdn博客之星竞选,欢迎投票支持,每个QQ或者微信每天都可以投5票,扫二维码即可,http://m234140.nofollow.ax.
Python 基础(一):入门必备知识
目录1 标识符2 关键字3 引号4 编码5 输入输出6 缩进7 多行8 注释9 数据类型10 运算符10.1 常用运算符10.2 运算符优先级 1 标识符 标识符是编程时使用的名字,用于给变量、函数、语句块等命名,Python 中标识符由字母、数字、下划线组成,不能以数字开头,区分大小写。 以下划线开头的标识符有特殊含义,单下划线开头的标识符,如:_xxx ,表示不能直接访问的类属性,需通过类提供
深度学习图像算法在内容安全领域的应用
互联网给人们生活带来便利的同时也隐含了大量不良信息,防范互联网平台有害内容传播引起了多方面的高度关注。本次演讲从技术层面分享网易易盾在内容安全领域的算法实践经验,包括深度学习图
程序员接私活怎样防止做完了不给钱?
首先跟大家说明一点,我们做 IT 类的外包开发,是非标品开发,所以很有可能在开发过程中会有这样那样的需求修改,而这种需求修改很容易造成扯皮,进而影响到费用支付,甚至出现做完了项目收不到钱的情况。 那么,怎么保证自己的薪酬安全呢? 我们在开工前,一定要做好一些证据方面的准备(也就是“讨薪”的理论依据),这其中最重要的就是需求文档和验收标准。一定要让需求方提供这两个文档资料作为开发的基础。之后开发
网页实现一个简单的音乐播放器(大佬别看。(⊙﹏⊙))
今天闲着无事,就想写点东西。然后听了下歌,就打算写个播放器。 于是乎用h5 audio的加上js简单的播放器完工了。 欢迎 改进 留言。 演示地点跳到演示地点 html代码如下`&lt;!DOCTYPE html&gt; &lt;html&gt; &lt;head&gt; &lt;title&gt;music&lt;/title&gt; &lt;meta charset="utf-8"&gt
Python十大装B语法
Python 是一种代表简单思想的语言,其语法相对简单,很容易上手。不过,如果就此小视 Python 语法的精妙和深邃,那就大错特错了。本文精心筛选了最能展现 Python 语法之精妙的十个知识点,并附上详细的实例代码。如能在实战中融会贯通、灵活使用,必将使代码更为精炼、高效,同时也会极大提升代码B格,使之看上去更老练,读起来更优雅。 1. for - else 什么?不是 if 和 else 才
数据库优化 - SQL优化
前面一篇文章从实例的角度进行数据库优化,通过配置一些参数让数据库性能达到最优。但是一些“不好”的SQL也会导致数据库查询变慢,影响业务流程。本文从SQL角度进行数据库优化,提升SQL运行效率。 判断问题SQL 判断SQL是否有问题时可以通过两个表象进行判断: 系统级别表象 CPU消耗严重 IO等待严重 页面响应时间过长
2019年11月中国大陆编程语言排行榜
2019年11月2日,我统计了某招聘网站,获得有效程序员招聘数据9万条。针对招聘信息,提取编程语言关键字,并统计如下: 编程语言比例 rank pl_ percentage 1 java 33.62% 2 c/c++ 16.42% 3 c_sharp 12.82% 4 javascript 12.31% 5 python 7.93% 6 go 7.25% 7
通俗易懂地给女朋友讲:线程池的内部原理
餐厅的约会 餐盘在灯光的照耀下格外晶莹洁白,女朋友拿起红酒杯轻轻地抿了一小口,对我说:“经常听你说线程池,到底线程池到底是个什么原理?”我楞了一下,心里想女朋友今天是怎么了,怎么突然问出这么专业的问题,但做为一个专业人士在女朋友面前也不能露怯啊,想了一下便说:“我先给你讲讲我前同事老王的故事吧!” 大龄程序员老王 老王是一个已经北漂十多年的程序员,岁数大了,加班加不动了,升迁也无望,于是拿着手里
经典算法(5)杨辉三角
杨辉三角 是经典算法,这篇博客对它的算法思想进行了讲解,并有完整的代码实现。
腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹?
昨天,有网友私信我,说去阿里面试,彻底的被打击到了。问了为什么网上大量使用ThreadLocal的源码都会加上private static?他被难住了,因为他从来都没有考虑过这个问题。无独有偶,今天笔者又发现有网友吐槽了一道腾讯的面试题,我们一起来看看。 腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹? 在互联网职场论坛,一名程序员发帖求助到。二面腾讯,其中一个算法题:64匹
面试官:你连RESTful都不知道我怎么敢要你?
面试官:了解RESTful吗? 我:听说过。 面试官:那什么是RESTful? 我:就是用起来很规范,挺好的 面试官:是RESTful挺好的,还是自我感觉挺好的 我:都挺好的。 面试官:… 把门关上。 我:… 要干嘛?先关上再说。 面试官:我说出去把门关上。 我:what ?,夺门而去 文章目录01 前言02 RESTful的来源03 RESTful6大原则1. C-S架构2. 无状态3.统一的接
为啥国人偏爱Mybatis,而老外喜欢Hibernate/JPA呢?
关于SQL和ORM的争论,永远都不会终止,我也一直在思考这个问题。昨天又跟群里的小伙伴进行了一番讨论,感触还是有一些,于是就有了今天这篇文。 声明:本文不会下关于Mybatis和JPA两个持久层框架哪个更好这样的结论。只是摆事实,讲道理,所以,请各位看官勿喷。 一、事件起因 关于Mybatis和JPA孰优孰劣的问题,争论已经很多年了。一直也没有结论,毕竟每个人的喜好和习惯是大不相同的。我也看
项目中的if else太多了,该怎么重构?
介绍 最近跟着公司的大佬开发了一款IM系统,类似QQ和微信哈,就是聊天软件。我们有一部分业务逻辑是这样的 if (msgType = "文本") { // dosomething } else if(msgType = "图片") { // doshomething } else if(msgType = "视频") { // doshomething } else { // doshom...
致 Python 初学者
欢迎来到“Python进阶”专栏!来到这里的每一位同学,应该大致上学习了很多 Python 的基础知识,正在努力成长的过程中。在此期间,一定遇到了很多的困惑,对未来的学习方向感到迷茫。我非常理解你们所面临的处境。我从2007年开始接触 python 这门编程语言,从2009年开始单一使用 python 应对所有的开发工作,直至今天。回顾自己的学习过程,也曾经遇到过无数的困难,也曾经迷茫过、困惑过。开办这个专栏,正是为了帮助像我当年一样困惑的 Python 初学者走出困境、快速成长。希望我的经验能真正帮到你
Python 编程实用技巧
Python是一门很灵活的语言,也有很多实用的方法,有时候实现一个功能可以用多种方法实现,我这里总结了一些常用的方法,并会持续更新。
“狗屁不通文章生成器”登顶GitHub热榜,分分钟写出万字形式主义大作
一、垃圾文字生成器介绍 最近在浏览GitHub的时候,发现了这样一个骨骼清奇的雷人项目,而且热度还特别高。 项目中文名:狗屁不通文章生成器 项目英文名:BullshitGenerator 根据作者的介绍,他是偶尔需要一些中文文字用于GUI开发时测试文本渲染,因此开发了这个废话生成器。但由于生成的废话实在是太过富于哲理,所以最近已经被小伙伴们给玩坏了。 他的文风可能是这样的: 你发现,...
程序员:我终于知道post和get的区别
是一个老生常谈的话题,然而随着不断的学习,对于以前的认识有很多误区,所以还是需要不断地总结的,学而时习之,不亦说乎
"狗屁不通文章生成器"登顶GitHub热榜,分分钟写出万字形式主义大作
GitHub 被誉为全球最大的同性交友网站,……,陪伴我们已经走过 10+ 年时间,它托管了大量的软件代码,同时也承载了程序员无尽的欢乐。 万字申请,废话报告,魔幻形式主义大作怎么写?兄dei,狗屁不通文章生成器了解一下。这个富有灵魂的项目名吸引了众人的目光。项目仅仅诞生一周,便冲上了GitHub趋势榜榜首(Js中文网 -前端进阶资源教程)、是榜首哦
推荐几款比较实用的工具,网站
1.盘百度PanDownload 这个云盘工具是免费的,可以进行资源搜索,提速(偶尔会抽风????) 不要去某站买付费的???? PanDownload下载地址 2.BeJSON 这是一款拥有各种在线工具的网站,推荐它的主要原因是网站简洁,功能齐全,广告相比其他广告好太多了 bejson网站 3.二维码美化 这个网站的二维码美化很好看,网站界面也很...
《程序人生》系列-这个程序员只用了20行代码就拿了冠军
你知道的越多,你不知道的越多 点赞再看,养成习惯GitHub上已经开源https://github.com/JavaFamily,有一线大厂面试点脑图,欢迎Star和完善 前言 这一期不算《吊打面试官》系列的,所有没前言我直接开始。 絮叨 本来应该是没有这期的,看过我上期的小伙伴应该是知道的嘛,双十一比较忙嘛,要值班又要去帮忙拍摄年会的视频素材,还得搞个程序员一天的Vlog,还要写BU...
相关热词 如何提升c#开发能力 矩阵乘法c# c#调用谷歌浏览器 c# 去空格去转义符 c#用户登录窗体代码 c# 流 c# linux 可视化 c# mvc 返回图片 c# 像素空间 c# 日期 最后一天
立即提问