如何分析Spark集群的启动日志


这篇文章给大家介绍如何分析Spark集群的启动日志,内容非常详细,感兴趣的小伙伴们可以参考借鉴,希望对大家能有所帮助。added by Jerry:…
/root/devExpert/spark-1.4.1/sbin/…/conf – ( Jerry: I haven’t copied out the template from my own confiugration file yet )
starting org.apache.spark.deploy.master.Master, logging to /root/devExpert/spark-1.4.1/sbin/…/logs/spark-root-org.apache.spark.deploy.master.Master-1-NKGV50849583FV1.out
Jerry- location of log file:NKGV50849583FV1:~/devExpert/spark-1.4.1/logs # vi spark-root-org.apache.spark.deploy.master.Master-1-NKGV50849583FV1.out1 added by Jerry: loading load-spark-env.sh !!!1
2 added by Jerry, number of Jars: 1
3 added by Jerry, launch_classpath: /root/devExpert/spark-1.4.1/assembly/target/scala-2.10/spark-assembly-1.4.1-hadoop2.4.0.jar
4 added by Jerry,RUNNER:/usr/jdk1.7.0_79/bin/java
5 added by Jerry, printf argument list: org.apache.spark.deploy.master.Master –ip NKGV50849583FV1 –port 7077 –webui-port 8080
Jerry: this is default value6 Spark Command: /usr/jdk1.7.0_79/bin/java -cp /root/devExpert/spark-1.4.1/sbin/…/conf/:/root/devExpert/spark-1.4.1/assembly/target/scala-2.10/spark-assem bly-1.4.1-hadoop2.4.0.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-rdbms-3.2.9.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucle us-core-3.2.10.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-api-jdo-3.2.6.jar -Xms512m -Xmx512m -XX:MaxPermSize=256m org.apache.spark.dep loy.master.Master –ip NKGV50849583FV1 –port 7077 –webui-port 8080
7 ========================================
8 added by Jerry, I am in if-else branch: /usr/jdk1.7.0_79/bin/java -cp /root/devExpert/spark-1.4.1/sbin/…/conf/:/root/devExpert/spark-1.4.1/assembly/targ et/scala-2.10/spark-assembly-1.4.1-hadoop2.4.0.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-rdbms-3.2.9.jar:/root/devExpert/spark-1.4.1/l ib_managed/jars/datanucleus-core-3.2.10.jar:/root/devExpert/spark-1.4.1/lib_managed/jars/datanucleus-api-jdo-3.2.6.jar -Xms512m -Xmx512m -XX:MaxPermSize= 256m org.apache.spark.deploy.master.Master –ip NKGV50849583FV1 –port 7077 –webui-port 8080
9 Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties
10 15/08/16 12:45:17 INFO Master: Registered signal handlers for [TERM, HUP, INT]
11 15/08/16 12:45:17 WARN Utils: Your hostname, NKGV50849583FV1 resolves to a loopback address: 127.0.0.1; using 10.128.184.131 instead (on interface eth0)
12 15/08/16 12:45:17 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
Jerry: useful hint
13 15/08/16 12:45:18 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
14 15/08/16 12:45:18 INFO SecurityManager: Changing view acls to: root
15 15/08/16 12:45:18 INFO SecurityManager: Changing modify acls to: root
16 15/08/16 12:45:18 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with mo dify permissions: Set(root)
17 15/08/16 12:45:19 INFO Slf4jLogger: Slf4jLogger started
18 15/08/16 12:45:19 INFO Remoting: Starting remoting
19 15/08/16 12:45:19 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkMaster@NKGV50849583FV1:7077]
20 15/08/16 12:45:19 INFO Utils: Successfully started service ‘sparkMaster’ on port 7077.21 15/08/16 12:45:19 INFO Utils: Successfully started service on port 6066.
22 15/08/16 12:45:19 INFO StandaloneRestServer: Started REST server for submitting applications on port 6066
23 15/08/16 12:45:19 INFO Master: Starting Spark master at spark://NKGV50849583FV1:7077
24 15/08 香港云主机/16 12:45:19 INFO Master: Running Spark version 1.4.1
25 15/08/16 12:45:19 INFO Utils: Successfully started service ‘MasterUI’ on port 8080.26 15/08/16 12:45:19 INFO MasterWebUI: Started MasterWebUI at http://10.128.184.131:8080
27 15/08/16 12:45:20 INFO Master: I have been elected leader! New state: ALIVE – cool!关于如何分析Spark集群的启动日志就分享到这里了,希望以上内容可以对大家有一定的帮助,可以学到更多知识。如果觉得文章不错,可以把它分享出去让更多的人看到。

相关推荐: Python中repr() 函数的作用是什么

今天就跟大家聊聊有关Python中repr() 函数的作用是什么,可能很多人都不太了解,为了让大家更加了解,小编给大家总结了以下内容,希望大家根据这篇文章可以有所收获。repr() 函数将对象转化为供解释器读取的形式。以下是 repr() 方法的语法:obje…

免责声明:本站发布的图片视频文字,以转载和分享为主,文章观点不代表本站立场,本站不承担相关法律责任;如果涉及侵权请联系邮箱:360163164@qq.com举报,并提供相关证据,经查实将立刻删除涉嫌侵权内容。

(0)
打赏 微信扫一扫 微信扫一扫
上一篇 09/22 22:06
下一篇 09/22 22:06

相关推荐