spark集群运行错误:15

这是一个仿照网上例子,自己学习测试的。用scala编写写了一个wordCount的例子,在myEclipse上是可以运行的,并可以得出结果。现在将例子导出jar包,然后放到hadoop集群上运行,出现如下错误:Stack trace: ExitCodeException exitCode=15跪求各路大神帮忙, 这个问题已经困扰我一个星期了,网上也找了很久,没找到解决办法。没有多少分了 。。。非常感谢!!!环境: hadoop2.6.2 spark2.2 jdk1.8 scala2.2hadoop集群应该是没有问题的,浏览器可以打开50070的页面下面是spark on yarn的环境:export JAVA_HOME=/usr/local/src/jdk1.8.0_144export SPARK_MASTER_IP=node1export SPARK_MASTER_PORT=7077export SPARK_WORKER_CORES=1export SPARK_WORKER_INSTANCES=1export SPARK_WORKER_MEMORY=1gexport SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=node1:2181,node2:2181,node3:2181"export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoopexport YARN_CONF_DIR=$HADOOP_HOME/etc/hadoopexport SPARK_HOME=/usr/local/src/spark-2.2.0-bin-hadoop2.6export SPARK_JAR=/usr/local/src/spark-2.2.0-bin-hadoop2.6/jars/*.jarexport PATH=$SPARK_HOME/bin:$PATHWordCount例子:object wc { def main(args: Array[String]): Unit = { val conf = new SparkConf().setAppName("wc") val sc = new SparkContext(conf) val text = sc.textFile("test.txt") val words = text.flatMap(line => line.split(" ")) val pairs = words.map(word => (word, 1)) val results = pairs.reduceByKey(_+_).map(tuple => (tuple._2 , tuple._1 )) val sorted = results.sortByKey(false).map(tuple => (tuple._2 , tuple._1 )) sorted.foreach(x => println(x)) sc.stop() }}错误信息:Application Attempt State: FAILEDAM Container: container_1507729080248_0001_01_000001Node: N/ATracking URL: HistoryDiagnostics Info: AM Container for appattempt_1507729080248_0001_000001 exited with exitCode: 15For more detailed output, check application tracking page:http://node1:8088/proxy/application_1507729080248_0001/Then, click on links to logs of each attempt.Diagnostics: Exception from container-launch.Container id: container_1507729080248_0001_01_000001Exit code: 15Stack trace: ExitCodeException exitCode=15:at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)at org.apache.hadoop.util.Shell.run(Shell.java:455)at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)Container exited with a non-zero exit code 15Failing this attempt图片图片图片

1个回答

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
Jar在spark-shell上运行报错:主类找不到

scala IntelliJ的项目,sbt打好包在spark-shell上运行后报错:主类找不到;使用了两个中文分词包(ansj_seg-2.0.8.jar,nlp-lang-0.3.jar),但是已经加入到 External libraries里去了;打包没问题,运行报错 ![![图片说明](https://img-ask.csdn.net/upload/201601/26/1453780626_723163.jpg)![图片说明](https://img-ask.csdn.net/upload/201601/26/1453780648_659305.jpg) spark-shell 提交命令: [gaohui@hadoop-1-2 test]$ spark-submit --master yarn --driver-memory 5G --num-executors 20 --executor-cores 16 --executor-memory 10G --conf spark.serializer=org.apache.spark.serializer.KryoSerializer --class NLP_V6.Nlp_test --jars /home/gaohui/test/NLP_v6_test.jar /home/gaohui/test/NLP_v6_test.jar 报错图片: ![图片说明](https://img-ask.csdn.net/upload/201601/26/1453780776_603750.jpg)

spark-submit命令运行jar包报空指针,Java -jar命令可以运行。

local[*]模式下的spark程序,在idea上运行没问题,用maven打出来的jar包在windows的cmd下运行java -jar XX.jar 也可以运行成功。将jar包放到集群中运用spark-submit运行可以打印输出控制台打印结果,但是最后报空指针异常,并且结果不能保存到mysql。可是在集群中也运用java -jar xxx.jar一样可以运行不会报错。这是什么原因?我怎么才能用spark-submit命令运行成功?我想要通过这个命令去具体制定某一个类,利用Java -jar命令就不能具体制定哪一个类 。求帮忙 Exception in thread "main" java.lang.NullPointerException at java.util.Properties$LineReader.readLine(Properties.java:434) at java.util.Properties.load0(Properties.java:353) at java.util.Properties.load(Properties.java:341) at com.bigdata.bcht.spes.SparkEsLawUndel$.main(SparkEsLawUndel.scala:90) at com.bigdata.bcht.spes.SparkEsLawUndel.main(SparkEsLawUndel.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:745) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

六台机器集群,40M数据就报错,spark streaming运行例子程序wordcount

请大神帮忙解决一下:六台机器,SparkStreaming的例子程序,运行在yarn上四个计算节点(nodemanager),每台8G内存,i7处理器,想测测性能。 自己写了socket一直向一个端口发送数据,spark 接收并处理 运行十几分钟汇报错:WARN scheduler TaskSetManagerost task 0.1 in stage 265.0 :java.lang.Exception:Could not compute split ,block input-0-145887651600 not found![图片说明](https://img-ask.csdn.net/upload/201603/29/1459223107_940575.png)

java 远程连接spark 出现错误

我使用的是sequenceiq/spark 搭建的docker集群,但是本机上能正常的运行,但是通过java远程连接访问的时候出现错误 代码为: ``` SparkConf sparkConf = new SparkConf().setAppName("JavaTopGroup").setMaster("spark://10.73.21.221:7077"); JavaSparkContext ctx = new JavaSparkContext(sparkConf); ``` 出现的错误为: ``` 17/12/07 19:17:47 ERROR StandaloneSchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up. 17/12/07 19:17:47 WARN StandaloneSchedulerBackend: Application ID is not initialized yet. 17/12/07 19:17:47 INFO SparkUI: Stopped Spark web UI at http://10.73.7.25:4040 17/12/07 19:17:47 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 8163. 17/12/07 19:17:47 INFO StandaloneSchedulerBackend: Shutting down all executors 17/12/07 19:17:47 INFO NettyBlockTransferService: Server created on 10.73.7.25:8163 17/12/07 19:17:47 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 17/12/07 19:17:47 INFO CoarseGrainedSchedulerBackend$DriverEndpoint: Asking each executor to shut down 17/12/07 19:17:47 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.73.7.25, 8163, None) 17/12/07 19:17:47 INFO BlockManagerMasterEndpoint: Registering block manager 10.73.7.25:8163 with 900.6 MB RAM, BlockManagerId(driver, 10.73.7.25, 8163, None) 17/12/07 19:17:47 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.73.7.25, 8163, None) 17/12/07 19:17:47 WARN StandaloneAppClient$ClientEndpoint: Drop UnregisterApplication(null) because has not yet connected to master 17/12/07 19:17:47 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.73.7.25, 8163, None) 17/12/07 19:17:47 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped! 17/12/07 19:17:47 INFO MemoryStore: MemoryStore cleared 17/12/07 19:17:47 INFO BlockManager: BlockManager stopped 17/12/07 19:17:47 INFO BlockManagerMaster: BlockManagerMaster stopped 17/12/07 19:17:47 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped! 17/12/07 19:17:47 ERROR TransportResponseHandler: Still have 3 requests outstanding when connection from /10.73.21.21:7077 is closed 17/12/07 19:17:47 INFO SparkContext: Successfully stopped SparkContext 17/12/07 19:17:47 ERROR SparkContext: Error initializing SparkContext. java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a running MetricsSystem at scala.Predef$.require(Predef.scala:224) at org.apache.spark.metrics.MetricsSystem.getServletHandlers(MetricsSystem.scala:91) at org.apache.spark.SparkContext.<init>(SparkContext.scala:524) at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) at org.com.will.sparkl.App.main(App.java:24) 17/12/07 19:17:48 INFO SparkContext: SparkContext already stopped. Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a running MetricsSystem at scala.Predef$.require(Predef.scala:224) at org.apache.spark.metrics.MetricsSystem.getServletHandlers(MetricsSystem.scala:91) at org.apache.spark.SparkContext.<init>(SparkContext.scala:524) at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) at org.com.will.sparkl.App.main(App.java:24) 17/12/07 19:17:48 INFO ShutdownHookManager: Shutdown hook called 17/12/07 19:17:48 INFO ShutdownHookManager: Deleting directory C:\Users\will\AppData\Local\Temp\spark-c60f05a8-5476-469b-8c43-d8476796a1dd ```

spark 如何 多个application同时运行不报错

如题,在代码层面,已经设置了config的 set("spark.driver.allowMultipleContexts","true") 但是同时两个任务提交的时候还是会出现两个任务都阻塞掉的问题 求教下,这个如何友好的优化,谢谢啦

关于spark 的执行有问题求教。

前话有点多。。现有一java项目涉及到了hive部分功能,但是由于hive的查询速度很慢,想把底层hive部分的代码替换为spark,了解到CDH5可以直接将hive引擎更换为spark,更换后想测试一下速度差异,但是在hive命令行输入完sql总是会卡住一段时间,找不到原因,然后就尝试用sparksql代码尝试操作hive,因为刚接触spark,在用代码操作hive的过程中也经历很多错误,现在终于能用javaspark连到我集群的hive了,我看网上spark大多都是以jar包的形式提交到服务器执行,而我只是想把spark的部分穿插进java项目里像java代码一样调用,这样可以吗?跟jar包的形式执行有什么差异吗 另外求一些spark hive 的经验。。。。

Scala中动态加载jar包并调用方法,有个匿名类出现ClassNotFoundException

测试环境为:scala 2.11.8和spark2.1.0 单独jar包中TextProcess.scala中的代码: ``` object TextProcess { def process(col: String, df: Dataset[Row]) = { val rdd = df.select(col).rdd.map { x => val word = x.getAs[String](0).split(".")(0).toString() (word, 1) } val spark = SparkSession.builder().getOrCreate() val words = Array("one", "two", "two", "three", "three", "three") val wordPairsRDD = spark.sparkContext.parallelize(words).map(word => (word, 1)) val wordCountsWithReduce = wordPairsRDD.reduceByKey(_ + _) wordCountsWithReduce.foreach { x => println(x._1 , x._2) } val rd = rdd.reduceByKey(_ + _).map { x => Row.fromSeq(Array(x._1, x._2)) } val schema = StructType(Array(StructField("word", StringType, false), StructField("num", IntegerType, false))) spark.createDataFrame(rd, schema) } } ``` 测试调用代码: ``` object ProcessTest { def main(args: Array[String]) = { val spark = SparkSession.builder().master("local").getOrCreate() val data = Array("a.", "b.", "b.", "c.", "b.", "d.", "b.", "d.", "b.aa") val rdd = spark.sparkContext.parallelize(data, 1).map { x => Row.fromSeq(Seq(x)) } val schema = StructType(Array(StructField("text", StringType, false))) val df = spark.createDataFrame(rdd, schema) df.show() invoke(df) } def invoke(df: Dataset[Row]):Unit = { val f = new File("F:\\360CloudUI\\textprocess.jar") val loader = new URLClassLoader(Array(f.toURI().toURL()), getClass.getClassLoader) val clazz = loader.loadClass("cc.eabour.spark.TextProcess") val process = clazz.getDeclaredMethod("process", classOf[String], classOf[Dataset[_]]) process.invoke(null, "text", df) } } ``` 错误信息堆栈: ``` 7/09/24 18:43:08 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, localhost, executor driver, partition 0, PROCESS_LOCAL, 5981 bytes) 17/09/24 18:43:08 INFO Executor: Running task 0.0 in stage 1.0 (TID 1) 17/09/24 18:43:08 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1) java.lang.ClassNotFoundException: cc.eabour.spark.TextProcess$$anonfun$4 at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:67) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1593) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369) at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75) at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:85) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 17/09/24 18:43:08 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1, localhost, executor driver): java.lang.ClassNotFoundException: cc.eabour.spark.TextProcess$$anonfun$4 at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:67) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1593) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369) at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75) at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:85) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 17/09/24 18:43:08 ERROR TaskSetManager: Task 0 in stage 1.0 failed 1 times; aborting job 17/09/24 18:43:08 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 17/09/24 18:43:08 INFO TaskSchedulerImpl: Cancelling stage 1 ``` 不知道是spark的使用问题还是scala的调用问题。求指导

从hdfs中读取数据并用spark操作时出现问题

我从集群环境的hdfs中读取数据,然后处理数据时出现问题,在循环里面添加的对象在循环外就没有了,初学spark和scala,请大佬指点. object Test { case class Passenger(name: String, txn_date: String, txn_time: String, txn_station: String, ticket_type: String, trans_code: String, sub: String, txn_station_id: String) def main(args: Array[String]): Unit = { val inputFile = "hdfs://Master:9000/user/hadoop/input/tmp.txt" val conf = new SparkConf().setAppName("WordCount") val sc = new SparkContext(conf) val text = sc.textFile(inputFile) //25 lines like "000025643 " "20141201" "060912" "0328" "88" "22" "" from hdfs val Passengers = new ArrayBuffer[Passenger]() for (line <- text) { val points = for (i <- 0 until (line.length) if (line.charAt(i) == '"')) yield { i } val items = for (i <- 0 until (points.length) if (i % 2 == 0)) yield { if (!line.slice(points(i).toString.toInt + 1, points(i + 1).toString.toInt).equals("")) { line.slice(points(i).toString.toInt + 1, points(i + 1).toString.toInt).trim } else "null" } val tmp:Passenger=new Passenger(items(0).trim, items(1), items(2), items(3), items(4), items(5), "null", items(6)) println(tmp) //it is Passenger(000026853,20141201,060921,0325,88,21,null,null) [no problem] Passengers.append(tmp) println(Passengers.length) //1,2,3.....25 [no problem] } println("----------------------------" + Passengers.length) //it is 0!!!! why? val passengersArray = Passengers.toArray val customersRDD = sc.parallelize(passengersArray) val customersDF = customersRDD.toDF() } } ``` ```

hadoop HA高可用集群put文件的问题

HA集群上传文件时报错: ![图片说明](https://img-ask.csdn.net/upload/201608/29/1472439755_984382.png) could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation. namespaceID主从节点一致,并且之前没有格式化过····做的有可能导致这个问题的操作仅仅是修改了spark的配置文件(把hadoop的两个配置文件移到了spark下,并且修改了spark配置文件) http://my.oschina.net/cjun/blog/668990?p=1 之后向hadoop集群上传文件,就出现了这个错误。搜索枯肠试了很多种方法,无奈之下格式化了hadoop集群。还是这个问题·····又试图调低了系统安全···还是不行·············· 现在put空文件可以上去,有内容的就会报错········· 求各位大神指点!! jps指令运行效果,hadoop版本2.5.2,数据已经全部删除,本地路径下的也都删除了 ![图片说明](https://img-ask.csdn.net/upload/201608/30/1472518284_47051.png)

hadoop集群启动后namenode自动关闭

2017-09-05 10:14:17,973 INFO org.apache.hadoop.hdfs.StateChange: STATE* Safe mode ON, in safe mode extension. The reported blocks 189 has reached the threshold 0.9990 of total blocks 189. The number of live datanodes 2 has reached the minimum number 0. In safe mode extension. Safe mode will be turned off automatically in 9 seconds. 2017-09-05 10:14:23,736 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.updateBlockForPipeline from 172.28.14.61:41497 Call#164039 Retry#12 org.apache.hadoop.ipc.RetriableException: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot get a new generation stamp and an access token for block BP-1552766309-172.28.41.193-1503397713205:blk_1073742745_1926. Name node is in safe mode. The reported blocks 189 has reached the threshold 0.9990 of total blocks 189. The number of live datanodes 2 has reached the minimum number 0. In safe mode extension. Safe mode will be turned off automatically in 4 seconds. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1331) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkUCBlock(FSNamesystem.java:6234) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.updateBlockForPipeline(FSNamesystem.java:6309) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.updateBlockForPipeline(NameNodeRpcServer.java:806) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.updateBlockForPipeline(ClientNamenodeProtocolServerSideTranslatorPB.java:955) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) Caused by: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot get a new generation stamp and an access token for block BP-1552766309-172.28.41.193-1503397713205:blk_1073742745_1926. Name node is in safe mode. The reported blocks 189 has reached the threshold 0.9990 of total blocks 189. The number of live datanodes 2 has reached the minimum number 0. In safe mode extension. Safe mode will be turned off automatically in 4 seconds. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1327) ... 13 more 2017-09-05 10:14:27,976 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: initializing replication queues 2017-09-05 10:14:27,977 INFO org.apache.hadoop.hdfs.StateChange: STATE* Leaving safe mode after 55 secs 2017-09-05 10:14:27,977 INFO org.apache.hadoop.hdfs.StateChange: STATE* Safe mode is OFF 2017-09-05 10:14:27,977 INFO org.apache.hadoop.hdfs.StateChange: STATE* Network topology has 1 racks and 2 datanodes 2017-09-05 10:14:27,977 INFO org.apache.hadoop.hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks 2017-09-05 10:14:28,013 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Total number of blocks = 190 2017-09-05 10:14:28,013 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of invalid blocks = 0 2017-09-05 10:14:28,013 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of under-replicated blocks = 3 2017-09-05 10:14:28,013 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of over-replicated blocks = 0 2017-09-05 10:14:28,013 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of blocks being written = 1 2017-09-05 10:14:28,013 INFO org.apache.hadoop.hdfs.StateChange: STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 29 msec 2017-09-05 10:14:59,141 INFO org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager: topN size for command listStatus is: 0 2017-09-05 10:14:59,141 INFO org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager: topN size for command * is: 0 2017-09-05 10:14:59,143 INFO org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager: topN size for command listStatus is: 1 2017-09-05 10:14:59,145 INFO org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager: topN size for command * is: 1 2017-09-05 10:14:59,185 INFO org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager: topN size for command listStatus is: 1 2017-09-05 10:14:59,186 INFO org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager: topN size for command * is: 1 2017-09-05 10:16:50,848 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 172.28.41.196 2017-09-05 10:16:50,849 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs 2017-09-05 10:16:50,849 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 15839 2017-09-05 10:16:50,849 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 3 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 88 18 2017-09-05 10:16:50,883 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 3 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 120 20 2017-09-05 10:16:50,910 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoop/hadoop_name/current/edits_inprogress_0000000000000015839 -> /home/hadoop/hadoop_name/current/edits_0000000000000015839-0000000000000015841 2017-09-05 10:16:50,915 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 15842 2017-09-05 10:18:51,193 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 172.28.41.196 2017-09-05 10:18:51,193 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs 2017-09-05 10:18:51,193 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 15842 2017-09-05 10:18:51,194 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 1 SyncTimes(ms): 19 8 2017-09-05 10:18:51,372 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 129 76 2017-09-05 10:18:51,405 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoop/hadoop_name/current/edits_inprogress_0000000000000015842 -> /home/hadoop/hadoop_name/current/edits_0000000000000015842-0000000000000015843 2017-09-05 10:18:51,406 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 15844 2017-09-05 10:20:52,122 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 172.28.41.196 2017-09-05 10:20:52,122 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs 2017-09-05 10:20:52,122 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 15844 2017-09-05 10:20:52,122 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 1 SyncTimes(ms): 39 341 2017-09-05 10:20:52,258 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 103 413 2017-09-05 10:20:52,284 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoop/hadoop_name/current/edits_inprogress_0000000000000015844 -> /home/hadoop/hadoop_name/current/edits_0000000000000015844-0000000000000015845 2017-09-05 10:20:52,284 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 15846 报这样的错误是不是有问题

Spark部署不知为何出错

从网上找的linearRegression案例 ![图片说明](https://img-ask.csdn.net/upload/201712/11/1512922142_727946.png) 本地是可以跑通的 但是打成jar包之后,运行到集群上就出错(集群配置应该没有问题,SimpleTest是可以成功运行的) ![图片说明](https://img-ask.csdn.net/upload/201712/11/1512922241_352591.png) 求大神指点 代码: package spark import org.apache.spark.{SparkConf,SparkContext} import org.apache.spark.mllib.regression.LinearRegressionWithSGD import org.apache.spark.mllib.regression.LabeledPoint import org.apache.spark.mllib.linalg.{Vectors,Vector } import org.apache.log4j.{Level, Logger} object LR extends App{ Logger.getLogger("org.apache.spark").setLevel(Level.WARN) Logger.getLogger("org.eclipse.jetty.server").setLevel(Level.OFF) val conf = new SparkConf().setAppName("LinearRegressionTest Yunjisuan") .setMaster("spark://192.168.2.101:7077") // .setMaster("local[2]") val sc = new SparkContext(conf) // 加载和解析数据文件 val data = sc.textFile("hdfs://cloud01:9000/user/hduser/input/lpsa.data") val parsedData = data.map { line => val parts = line.split(",") LabeledPoint(parts(0).toDouble, Vectors.dense(parts(1).split(" ").map(x => x.toDouble))) } //设置迭代次数并进行训练 val numIterations = 100 val model = LinearRegressionWithSGD.train(parsedData, numIterations) //统计回归错误的样本比例 val valuesAndPreds = parsedData.map { point => val prediction = model.predict(point.features) (point.label, prediction) } val MSE = valuesAndPreds.map{ case(v, p) => math.pow((v - p), 2)}.reduce(_ + _)/valuesAndPreds.count println("training Mean Squared Error = " + MSE) val d= Vectors.dense(1.0, 0.0, 0.50,0.32,0.565,0.5,0.7,0.121) val rdd=sc.makeRDD(Array(Vectors.dense(1.0, 0.0, 0.50,0.32,0.565,0.5,0.7,0.121),Vectors.dense(2.0, 1.0, 1.50,1.32,0.565,4.5,0.7,0.1))) val prediction = model.predict(d) val predictions=model.predict(rdd) println(prediction) predictions.foreach(println) }

CentOS7安装R语言后无法下载R包,错误如下:

![图片说明](https://img-ask.csdn.net/upload/201610/07/1475828474_196009.png) 我准备在spark集群上运行R脚本,因为需要sparkR包,所以在centos虚拟机上装了R语言,可以联网,但是就是无法下载安装包。。。

初入Hadoop,start-all.sh问题

Hadoop 版本:hadoop-2.6.5 环境: ![图片说明](https://img-ask.csdn.net/upload/201711/08/1510104898_307684.png) HODOOP_HOME 目录也不同:都在各自用户的目录下 e.g: /home/yann/hadoop /home/ubuntu01/hadoop /home/ubuntu02/hadoop - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 当我在启动的时候start-all.sh 提示我如下错误: yann@yann-laptop:~/develop/tool/hadoop-2.6.5/sbin$ ./start-all.sh This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh Starting namenodes on [yann-laptop] yann-laptop: namenode running as process 7041. Stop it first. yann@ubuntu01-virtual-machine's password: The authenticity of host 'ubuntu02-virtual-machine (192.168.2.182)' can't be established. ECDSA key fingerprint is 57:ee:92:a8:85:85:ef:16:26:a3:b7:1d:54:77:19:18. Are you sure you want to continue connecting (yes/no)? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - hadoop/etc/hadoop/slaves: ubuntu01-virtual-machine ubuntu02-virtual-machine - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - SSH公钥已经配置好: ssh-rsa XXXXX......XXXXXX yann@yann-laptop ssh-rsa XXXXX......XXXXXX ubuntu01@ubuntu01-virtual-machine ssh-rsa XXXXX......XXXXXX ubuntu02@ubuntu02-virtual-machine - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 从提示来看它使用了yann去连接ubuntu01-virtual-machine。 请问这种情况如何配置(主从机器不同的用户名)?

hadoop + hive + tez 整合问题

我使用的hadoop以及hive还有tez的版本是: hadoop2.6.0-cdh5.13.0 + hive1.1.0-cdh5.13.0 + tez0.8.5 在hadoop集群上面,mr能够换成tez能够运行,但是当整合了hive以后,就会报 一个错误: Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1544497169142_0018_1_00, diagnostics=[Vertex vertex_1544497169142_0018_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: student initializer failed, vertex=vertex_1544497169142_0018_1_00 [Map 1], java.lang.NoClassDefFoundError: com/esotericsoftware/kryo/Serializer at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:107) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: com.esotericsoftware.kryo.Serializer at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 12 more ] Vertex killed, vertexName=Reducer 2, vertexId=vertex_1544497169142_0018_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1544497169142_0018_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE] DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask ``` ``` 然后缺少的这个jar包在tez以及hadoop里面都没有,hive的序列化也更改不了,之前查看有人换成javaXML,但是行不通,想请问大佬们,有知道这个问题怎么解决的吗?跪谢大佬。。。

hadoop Master节点namenode进程没有启动

刚开始单节点的时候全部都能启动,但是部署完子节点重新启动后,就无法启动主节点Master中的namenode进程了,而且通过查看日志也没有发现错误,求大神指点!! 这是日志中的内容 ![图片说明](https://img-ask.csdn.net/upload/201709/07/1504787698_513193.png) 启动的时候 ![图片说明](https://img-ask.csdn.net/upload/201709/07/1504787735_303226.png)

大学四年自学走来,这些私藏的实用工具/学习网站我贡献出来了

大学四年,看课本是不可能一直看课本的了,对于学习,特别是自学,善于搜索网上的一些资源来辅助,还是非常有必要的,下面我就把这几年私藏的各种资源,网站贡献出来给你们。主要有:电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意:文中提到的所有资源,文末我都给你整理好了,你们只管拿去,如果觉得不错,转发、分享就是最大的支持了。 一、电子书搜索 对于大部分程序员...

在中国程序员是青春饭吗?

今年,我也32了 ,为了不给大家误导,咨询了猎头、圈内好友,以及年过35岁的几位老程序员……舍了老脸去揭人家伤疤……希望能给大家以帮助,记得帮我点赞哦。 目录: 你以为的人生 一次又一次的伤害 猎头界的真相 如何应对互联网行业的「中年危机」 一、你以为的人生 刚入行时,拿着傲人的工资,想着好好干,以为我们的人生是这样的: 等真到了那一天,你会发现,你的人生很可能是这样的: ...

程序员请照顾好自己,周末病魔差点一套带走我。

程序员在一个周末的时间,得了重病,差点当场去世,还好及时挽救回来了。

技术大佬:我去,你写的 switch 语句也太老土了吧

昨天早上通过远程的方式 review 了两名新来同事的代码,大部分代码都写得很漂亮,严谨的同时注释也很到位,这令我非常满意。但当我看到他们当中有一个人写的 switch 语句时,还是忍不住破口大骂:“我擦,小王,你丫写的 switch 语句也太老土了吧!” 来看看小王写的代码吧,看完不要骂我装逼啊。 private static String createPlayer(PlayerTypes p...

你以为这样写Java代码很6,但我看不懂

为了提高 Java 编程的技艺,我最近在 GitHub 上学习一些高手编写的代码。下面这一行代码(出自大牛之手)据说可以征服你的朋友,让他们觉得你写的代码很 6,来欣赏一下吧。 IntStream.range(1, 5).boxed().map(i -&gt; { System.out.print("Happy Birthday "); if (i == 3) return "dear NAME"...

上班一个月,后悔当初着急入职的选择了

最近有个老铁,告诉我说,上班一个月,后悔当初着急入职现在公司了。他之前在美图做手机研发,今年美图那边今年也有一波组织优化调整,他是其中一个,在协商离职后,当时捉急找工作上班,因为有房贷供着,不能没有收入来源。所以匆忙选了一家公司,实际上是一个大型外包公司,主要派遣给其他手机厂商做外包项目。**当时承诺待遇还不错,所以就立马入职去上班了。但是后面入职后,发现薪酬待遇这块并不是HR所说那样,那个HR自...

女程序员,为什么比男程序员少???

昨天看到一档综艺节目,讨论了两个话题:(1)中国学生的数学成绩,平均下来看,会比国外好?为什么?(2)男生的数学成绩,平均下来看,会比女生好?为什么?同时,我又联想到了一个技术圈经常讨...

副业收入是我做程序媛的3倍,工作外的B面人生是怎样的?

提到“程序员”,多数人脑海里首先想到的大约是:为人木讷、薪水超高、工作枯燥…… 然而,当离开工作岗位,撕去层层标签,脱下“程序员”这身外套,有的人生动又有趣,马上展现出了完全不同的A/B面人生! 不论是简单的爱好,还是正经的副业,他们都干得同样出色。偶尔,还能和程序员的特质结合,产生奇妙的“化学反应”。 @Charlotte:平日素颜示人,周末美妆博主 大家都以为程序媛也个个不修边幅,但我们也许...

MySQL数据库面试题(2020最新版)

文章目录数据库基础知识为什么要使用数据库什么是SQL?什么是MySQL?数据库三大范式是什么mysql有关权限的表都有哪几个MySQL的binlog有有几种录入格式?分别有什么区别?数据类型mysql有哪些数据类型引擎MySQL存储引擎MyISAM与InnoDB区别MyISAM索引与InnoDB索引的区别?InnoDB引擎的4大特性存储引擎选择索引什么是索引?索引有哪些优缺点?索引使用场景(重点)...

如果你是老板,你会不会踢了这样的员工?

有个好朋友ZS,是技术总监,昨天问我:“有一个老下属,跟了我很多年,做事勤勤恳恳,主动性也很好。但随着公司的发展,他的进步速度,跟不上团队的步伐了,有点...

我入职阿里后,才知道原来简历这么写

私下里,有不少读者问我:“二哥,如何才能写出一份专业的技术简历呢?我总感觉自己写的简历太烂了,所以投了无数份,都石沉大海了。”说实话,我自己好多年没有写过简历了,但我认识的一个同行,他在阿里,给我说了一些他当年写简历的方法论,我感觉太牛逼了,实在是忍不住,就分享了出来,希望能够帮助到你。 01、简历的本质 作为简历的撰写者,你必须要搞清楚一点,简历的本质是什么,它就是为了来销售你的价值主张的。往深...

程序员写出这样的代码,能不挨骂吗?

当你换槽填坑时,面对一个新的环境。能够快速熟练,上手实现业务需求是关键。但是,哪些因素会影响你快速上手呢?是原有代码写的不够好?还是注释写的不够好?昨夜...

带了6个月的徒弟当了面试官,而身为高级工程师的我天天修Bug......

即将毕业的应届毕业生一枚,现在只拿到了两家offer,但最近听到一些消息,其中一个offer,我这个组据说客户很少,很有可能整组被裁掉。 想问大家: 如果我刚入职这个组就被裁了怎么办呢? 大家都是什么时候知道自己要被裁了的? 面试软技能指导: BQ/Project/Resume 试听内容: 除了刷题,还有哪些技能是拿到offer不可或缺的要素 如何提升面试软实力:简历, 行为面试,沟通能...

优雅的替换if-else语句

场景 日常开发,if-else语句写的不少吧??当逻辑分支非常多的时候,if-else套了一层又一层,虽然业务功能倒是实现了,但是看起来是真的很不优雅,尤其是对于我这种有强迫症的程序"猿",看到这么多if-else,脑袋瓜子就嗡嗡的,总想着解锁新姿势:干掉过多的if-else!!!本文将介绍三板斧手段: 优先判断条件,条件不满足的,逻辑及时中断返回; 采用策略模式+工厂模式; 结合注解,锦...

离职半年了,老东家又发 offer,回不回?

有小伙伴问松哥这个问题,他在上海某公司,在离职了几个月后,前公司的领导联系到他,希望他能够返聘回去,他很纠结要不要回去? 俗话说好马不吃回头草,但是这个小伙伴既然感到纠结了,我觉得至少说明了两个问题:1.曾经的公司还不错;2.现在的日子也不是很如意。否则应该就不会纠结了。 老实说,松哥之前也有过类似的经历,今天就来和小伙伴们聊聊回头草到底吃不吃。 首先一个基本观点,就是离职了也没必要和老东家弄的苦...

2020阿里全球数学大赛:3万名高手、4道题、2天2夜未交卷

阿里巴巴全球数学竞赛( Alibaba Global Mathematics Competition)由马云发起,由中国科学技术协会、阿里巴巴基金会、阿里巴巴达摩院共同举办。大赛不设报名门槛,全世界爱好数学的人都可参与,不论是否出身数学专业、是否投身数学研究。 2020年阿里巴巴达摩院邀请北京大学、剑桥大学、浙江大学等高校的顶尖数学教师组建了出题组。中科院院士、美国艺术与科学院院士、北京国际数学...

为什么你不想学习?只想玩?人是如何一步一步废掉的

不知道是不是只有我这样子,还是你们也有过类似的经历。 上学的时候总有很多光辉历史,学年名列前茅,或者单科目大佬,但是虽然慢慢地长大了,你开始懈怠了,开始废掉了。。。 什么?你说不知道具体的情况是怎么样的? 我来告诉你: 你常常潜意识里或者心理觉得,自己真正的生活或者奋斗还没有开始。总是幻想着自己还拥有大把时间,还有无限的可能,自己还能逆风翻盘,只不是自己还没开始罢了,自己以后肯定会变得特别厉害...

男生更看重女生的身材脸蛋,还是思想?

往往,我们看不进去大段大段的逻辑。深刻的哲理,往往短而精悍,一阵见血。问:产品经理挺漂亮的,有点心动,但不知道合不合得来。男生更看重女生的身材脸蛋,还是...

程序员为什么千万不要瞎努力?

本文作者用对比非常鲜明的两个开发团队的故事,讲解了敏捷开发之道 —— 如果你的团队缺乏统一标准的环境,那么即使勤劳努力,不仅会极其耗时而且成果甚微,使用...

为什么程序员做外包会被瞧不起?

二哥,有个事想询问下您的意见,您觉得应届生值得去外包吗?公司虽然挺大的,中xx,但待遇感觉挺低,马上要报到,挺纠结的。

当HR压你价,说你只值7K,你该怎么回答?

当HR压你价,说你只值7K时,你可以流畅地回答,记住,是流畅,不能犹豫。 礼貌地说:“7K是吗?了解了。嗯~其实我对贵司的面试官印象很好。只不过,现在我的手头上已经有一份11K的offer。来面试,主要也是自己对贵司挺有兴趣的,所以过来看看……”(未完) 这段话主要是陪HR互诈的同时,从公司兴趣,公司职员印象上,都给予对方正面的肯定,既能提升HR的好感度,又能让谈判气氛融洽,为后面的发挥留足空间。...

面试阿里p7,被按在地上摩擦,鬼知道我经历了什么?

面试阿里p7被问到的问题(当时我只知道第一个):@Conditional是做什么的?@Conditional多个条件是什么逻辑关系?条件判断在什么时候执...

Python爬虫,高清美图我全都要(彼岸桌面壁纸)

爬取彼岸桌面网站较为简单,用到了requests、lxml、Beautiful Soup4

无代码时代来临,程序员如何保住饭碗?

编程语言层出不穷,从最初的机器语言到如今2500种以上的高级语言,程序员们大呼“学到头秃”。程序员一边面临编程语言不断推陈出新,一边面临由于许多代码已存在,程序员编写新应用程序时存在重复“搬砖”的现象。 无代码/低代码编程应运而生。无代码/低代码是一种创建应用的方法,它可以让开发者使用最少的编码知识来快速开发应用程序。开发者通过图形界面中,可视化建模来组装和配置应用程序。这样一来,开发者直...

面试了一个 31 岁程序员,让我有所触动,30岁以上的程序员该何去何从?

最近面试了一个31岁8年经验的程序猿,让我有点感慨,大龄程序猿该何去何从。

大三实习生,字节跳动面经分享,已拿Offer

说实话,自己的算法,我一个不会,太难了吧

程序员垃圾简历长什么样?

已经连续五年参加大厂校招、社招的技术面试工作,简历看的不下于万份 这篇文章会用实例告诉你,什么是差的程序员简历! 疫情快要结束了,各个公司也都开始春招了,作为即将红遍大江南北的新晋UP主,那当然要为小伙伴们做点事(手动狗头)。 就在公众号里公开征简历,义务帮大家看,并一一点评。《启舰:春招在即,义务帮大家看看简历吧》 一石激起千层浪,三天收到两百多封简历。 花光了两个星期的所有空闲时...

Java岗开发3年,公司临时抽查算法,离职后这几题我记一辈子

前几天我们公司做了一件蠢事,非常非常愚蠢的事情。我原以为从学校出来之后,除了找工作有测试外,不会有任何与考试有关的事儿。 但是,天有不测风云,公司技术总监、人事总监两位大佬突然降临到我们事业线,叫上我老大,给我们组织了一场别开生面的“考试”。 那是一个风和日丽的下午,我翘着二郎腿,左手端着一杯卡布奇诺,右手抓着我的罗技鼠标,滚动着轮轴,穿梭在头条热点之间。 “淡黄的长裙~蓬松的头发...

大牛都会用的IDEA调试技巧!!!

导读 前天面试了一个985高校的实习生,问了他平时用什么开发工具,他想也没想的说IDEA,于是我抛砖引玉的问了一下IDEA的调试用过吧,你说说怎么设置断点...

面试官:你连SSO都不懂,就别来面试了

大厂竟然要考我SSO,卧槽。

程序员是做全栈工程师好?还是专注一个领域好?

昨天,有位大一的同学私信我,说他要做全栈工程师。 我一听,这不害了孩子么,必须制止啊。 谁知,讲到最后,更确定了他做全栈程序员的梦想。 但凡做全栈工程师的,要么很惨,要么很牛! 但凡很牛的,绝不是一开始就是做全栈的! 全栈工程师听起来好听,但绝没有你想象的那么简单。 今天听我来给你唠,记得帮我点赞哦。 一、全栈工程师的职责 如果你学习编程的目的只是玩玩,那随意,想怎么学怎么学。...

不要再到处使用 === 了

我们知道现在的开发人员都使用 === 来代替 ==,为什么呢?我在网上看到的大多数教程都认为,要预测 JavaScript 强制转换是如何工作这太复杂了,因此建议总是使用===。这些都...

终于,月薪过5万了!

来看几个问题想不想月薪超过5万?想不想进入公司架构组?想不想成为项目组的负责人?想不想成为spring的高手,超越99%的对手?那么本文内容是你必须要掌握的。本文主要详解bean的生命...

MySQL性能优化(五):为什么查询速度这么慢

前期回顾: MySQL性能优化(一):MySQL架构与核心问题 MySQL性能优化(二):选择优化的数据类型 MySQL性能优化(三):深入理解索引的这点事 MySQL性能优化(四):如何高效正确的使用索引 前面章节我们介绍了如何选择优化的数据类型、如何高效的使用索引,这些对于高性能的MySQL来说是必不可少的。但这些还完全不够,还需要合理的设计查询。如果查询写的很糟糕,即使表结构再合理、索引再...

立即提问
相关内容推荐