cloudera manager 离线安装安装agent时,向主节点下载资源超时 20C

错误日志:
[19/Nov/2018 16:16:04 +0000] 2789 MainThread stacks_collection_manager INFO Using max_uncompressed_file_size_bytes: 5242880
[19/Nov/2018 16:16:04 +0000] 2789 MainThread init INFO Importing metric schema from file /opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/monitor/schema.json
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/opt/cloudera-manager/cm-5.10.2/lib64/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/opt/cloudera-manager/cm-5.10.2/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUSTEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/opt/cloudera-manager/cm-5.10.2/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'ROOT': '/opt/cloudera-manager/cm-5.10.2/lib64/cmf', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Created /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/process
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Chmod'ing /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/process to 0751
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Created /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/supervisor
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Chmod'ing /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/supervisor to 0751
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Created /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/flood
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Chowning /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/flood to cloudera-scm (498) cloudera-scm (498)
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Chmod'ing /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/flood to 0751
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Created /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/supervisor/include
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent INFO Chmod'ing /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/supervisor/include to 0751
[19/Nov/2018 16:16:04 +0000] 2789 MainThread agent ERROR Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/agent.py", line 2073, in find_or_start_supervisor
self.configure_supervisor_clients()
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/agent.py", line 2254, in configure_supervisor_clients
supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")])
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 1599, in realize
Options.realize(self, *arg, **kw)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 333, in realize
self.process_config()
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 341, in process_config
self.process_config_file(do_usage)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 376, in process_config_file
self.usage(str(msg))
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 164, in usage
self.exit(2)
SystemExit: 2
[19/Nov/2018 16:16:04 +0000] 2789 MainThread tmpfs INFO Successfully mounted tmpfs at /opt/cloudera-manager/cm-5.10.2/run/cloudera-scm-agent/process
[19/Nov/2018 16:16:05 +0000] 2789 MainThread agent INFO Trying to connect to newly launched supervisor (Attempt 1)
[19/Nov/2018 16:16:05 +0000] 2789 MainThread agent INFO Supervisor version: 3.0, pid: 2821
[19/Nov/2018 16:16:05 +0000] 2789 MainThread agent INFO Successfully connected to supervisor
[19/Nov/2018 16:16:05 +0000] 2789 MainThread status_server INFO Using maximum impala profile bundle size of 1073741824 bytes.
[19/Nov/2018 16:16:05 +0000] 2789 MainThread status_server INFO Using maximum stacks log bundle size of 1073741824 bytes.
[19/Nov/2018 16:16:05 +0000] 2789 MainThread cplogging INFO [19/Nov/2018:16:16:05] ENGINE Bus STARTING
[19/Nov/2018 16:16:05 +0000] 2789 MainThread _cplogging INFO [19/Nov/2018:16:16:05] ENGINE Started monitor thread '_TimeoutMonitor'.
[19/Nov/2018 16:16:06 +0000] 2789 MainThread _cplogging INFO [19/Nov/2018:16:16:06] ENGINE Serving on yingzhi01.com:9000
[19/Nov/2018 16:16:06 +0000] 2789 MainThread _cplogging INFO [19/Nov/2018:16:16:06] ENGINE Bus STARTED
[19/Nov/2018 16:16:06 +0000] 2789 MainThread __init
_ INFO New monitor: (,)
[19/Nov/2018 16:16:06 +0000] 2789 MonitorDaemon-Scheduler init INFO Monitor ready to report: ('HostMonitor',)
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Setting default socket timeout to 30
[19/Nov/2018 16:16:06 +0000] 2789 Monitor-HostMonitor network_interfaces INFO NIC iface eth0 doesn't support ETHTOOL (95)
[19/Nov/2018 16:16:06 +0000] 2789 Monitor-HostMonitor throttling_logger ERROR Error getting directory attributes for /opt/cloudera-manager/cm-5.10.2/log/cloudera-scm-agent
Traceback (most recent call last):
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/monitor/dir_monitor.py", line 90, in get_directory_attributes
name = pwd.getpwuid(uid)[0]
KeyError: 'getpwuid(): uid not found: 1106'
[19/Nov/2018 16:16:06 +0000] 2789 MainThread heartbeat_tracker INFO HB stats (seconds): num:1 LIFE_MIN:0.22 min:0.22 mean:0.22 max:0.22 LIFE_MAX:0.22
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO CM server guid: dceeafae-a884-42f1-ba7b-4ee187ef3bef
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent WARNING Expected user root for /opt/cloudera/parcels but was cloudera-scm
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent WARNING Expected group root for /opt/cloudera/parcels but was cloudera-scm
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Created /opt/cloudera/parcel-cache
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Chowning /opt/cloudera/parcel-cache to root (0) root (0)
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Chmod'ing /opt/cloudera/parcel-cache to 0755
[19/Nov/2018 16:16:06 +0000] 2789 MainThread parcel INFO Agent does create users/groups and apply file permissions
[19/Nov/2018 16:16:06 +0000] 2789 MainThread downloader INFO Downloader path: /opt/cloudera/parcel-cache
[19/Nov/2018 16:16:06 +0000] 2789 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Flood daemon (re)start attempt
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Created /opt/cloudera/parcels/.flood
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Chowning /opt/cloudera/parcels/.flood to cloudera-scm (498) cloudera-scm (498)
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Chmod'ing /opt/cloudera/parcels/.flood to 0755
[19/Nov/2018 16:16:06 +0000] 2789 MainThread agent INFO Triggering supervisord update.
[19/Nov/2018 16:16:36 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:16:36 +0000] 2789 MainThread agent INFO Active parcel list updated; recalculating component info.
[19/Nov/2018 16:16:36 +0000] 2789 MainThread throttling_logger WARNING CMF_AGENT_JAVA_HOME environment variable host override will be deprecated in future. JAVA_HOME setting configured from CM server takes precedence over host agent override. Configure JAVA_HOME setting from CM server.
[19/Nov/2018 16:16:36 +0000] 2789 MainThread throttling_logger INFO Identified java component java8 with full version JAVA_HOME=/opt/modules/jdk1.8.0_144 java version "1.8.0_144" Java(TM) SE Runtime Environment (build 1.8.0_144-b01) Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode) for requested version .
[19/Nov/2018 16:16:36 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.6659779549
[19/Nov/2018 16:16:36 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:16:44 +0000] 2789 Monitor-HostMonitor throttling_logger ERROR Timeout with args ['ntpdc', '-np']
None
[19/Nov/2018 16:16:44 +0000] 2789 Monitor-HostMonitor throttling_logger ERROR Failed to collect NTP metrics
Traceback (most recent call last):
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/monitor/host/ntp_monitor.py", line 48, in collect
self.collect_ntpd()
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/monitor/host/ntp_monitor.py", line 66, in collect_ntpd
result, stdout, stderr = self._subprocess_with_timeout(args, self._timeout)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/monitor/host/ntp_monitor.py", line 38, in _subprocess_with_timeout
return subprocess_with_timeout(args, timeout)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/subprocess_timeout.py", line 94, in subprocess_with_timeout
raise Exception("timeout with args %s" % args)
Exception: timeout with args ['ntpdc', '-np']
[19/Nov/2018 16:17:06 +0000] 2789 DnsResolutionMonitor throttling_logger INFO Using java location: '/opt/modules/jdk1.8.0_144/bin/java'.
[19/Nov/2018 16:17:06 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:17:06 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1082139015
[19/Nov/2018 16:17:06 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:17:36 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:17:36 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1235852242
[19/Nov/2018 16:17:36 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:18:07 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:18:07 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1040799618
[19/Nov/2018 16:18:07 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:18:37 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:18:37 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1849529743
[19/Nov/2018 16:18:37 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:19:07 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:19:07 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1211960316
[19/Nov/2018 16:19:07 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:19:37 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:19:37 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1215620041
[19/Nov/2018 16:19:37 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:20:01 +0000] 2789 CP Server Thread-4 _cplogging INFO 192.168.164.35 - - [19/Nov/2018:16:20:01] "GET /heartbeat HTTP/1.1" 200 2 "" "NING/1.0"
[19/Nov/2018 16:20:04 +0000] 2789 CP Server Thread-5 _cplogging INFO 192.168.164.35 - - [19/Nov/2018:16:20:04] "GET /heartbeat HTTP/1.1" 200 2 "" "NING/1.0"
[19/Nov/2018 16:20:07 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:20:07 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1212861538
[19/Nov/2018 16:20:07 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:20:37 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:20:37 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1753029823
[19/Nov/2018 16:20:37 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:20:37 +0000] 2789 Thread-13 downloader INFO Fetching torrent: http://yingzhi01.com:7180/cmf/parcel/download/CDH-5.10.2-1.cdh5.10.2.p0.5-el6.parcel.torrent
[19/Nov/2018 16:20:37 +0000] 2789 Thread-13 downloader INFO Starting download of: http://yingzhi01.com:7180/cmf/parcel/download/CDH-5.10.2-1.cdh5.10.2.p0.5-el6.parcel
[19/Nov/2018 16:21:07 +0000] 2789 Thread-13 downloader ERROR Unexpected exception during download
Traceback (most recent call last):
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/cmf/downloader.py", line 279, in download
self.client.AddTorrent(torrent_url)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/flood/util/cmd.py", line 159, in __call
_
return self.fn.__get__(self.binding)(*args, **kwargs)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/flood/util/rpc.py", line 68, in
return lambda *pargs, **kwargs: self._invoke(*pargs, **kwargs)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/flood/util/rpc.py", line 77, in invoke
return rpcClient.requestor.request(self.schema.name, msg)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/flood/util/rpc.py", line 129, in requestor
return avro.ipc.Requestor(self.SCHEMA, self.transceiver)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.2-py2.6.egg/flood/util/rpc.py", line 125, in transceiver
return avro.ipc.HTTPTransceiver(self.server.host, self.server.port)
File "/opt/cloudera-manager/cm-5.10.2/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init
_
self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
raise error, msg
timeout: timed out
[19/Nov/2018 16:21:07 +0000] 2789 Thread-13 downloader INFO Finished download [ url: http://yingzhi01.com:7180/cmf/parcel/download/CDH-5.10.2-1.cdh5.10.2.p0.5-el6.parcel, state: exception, total_bytes: 0, downloaded_bytes: 0, start_time: 2018-11-19 16:20:37, download_end_time: , end_time: 2018-11-19 16:21:07, code: 600, exception_msg: timed out, path: None ]
[19/Nov/2018 16:21:07 +0000] 2789 MainThread downloader ERROR Failed rack peer update: timed out
[19/Nov/2018 16:21:07 +0000] 2789 MainThread agent WARNING Long HB processing time: 30.1247620583
[19/Nov/2018 16:21:07 +0000] 2789 MainThread agent WARNING Delayed HB: 15s since last
[19/Nov/2018 16:21:07 +0000] 2789 Thread-13 downloader INFO Fetching torrent: http://yingzhi01.com:7180/cmf/parcel/download/CDH-5.10.2-1.cdh5.10.2.p0.5-el6.parcel.torrent
[19/Nov/2018 16:21:08 +0000] 2789 Thread-13 downloader INFO Starting download of: http://yingzhi01.com:7180/cmf/parcel/download/CDH-5.10.2-1.cdh5.10.2.p0.5-el6.parcel
[19/Nov/2018 16:21:38 +0000] 2789 Thread-13 downloader ERROR Unexpected exception during download

然后就是不断重复超时错误求大神指点。。。

0

1个回答

0
yhntgbrfv1
yhntgbrfv1 并不是这个问题!而是选择完机器之后的安装页面一直卡在那!log报错连接超时
7 个月之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
离线安装Cloudera Manager 5和CDH5及与遇到的问题总结
今天介绍一下:在linux系统下,如何搭建CDH集群. 1.1Cloudera Manager简介 Cloudera Manager是一个拥有集群自动化安装、中心化管理、集群监控、报警功能的一个工具(软件),使得安装集群从几天的时间缩短在几个小时内, 运维人员从数十人降低到几人以内,极大的提高集群管理的效率。 Cloudera Manager有四大功能:    (1)管理:...
CDH、Cloudera Manager简介及离线安装步骤
CDH是Hadoop众多分支中的一种,由Cloudera维护,基于稳定版本的Apache Hadoop构建,提供了Hadoop的核心, 可扩展存储,分布式计算,基于Web的用户界面Cloudera Manager是一个管理CDH的端到端的应用。它的作用是对CDH进行管理、监控、诊断和集成安装前的准备:①网络配置②SSH免密钥登录③防火墙关闭④SELINUX关闭(setenforce 0  vi /...
Cloudera CDH新增节点到集群参考步骤
1、新加机器环境配置 首先得在新的主机环境中安装JDK,关闭防火墙、修改selinux、NTP时钟与主机同步、修改hosts、与主机配置ssh免密码登录、保证安装好了perl和python。 2、上传cloudera-manager文件到/opt目录,修改agent配置文件: vi /opt/cm-5.7.0/etc/cloudera-scm-agent/config.ini server
Cloudera Manager安装主机节点选不中
1, 安装集群,部分节点没办法选中,显示节点不受管。 解决方法: (1) 打开Cloudera server节点/etc/cloudera-scm-server/db.properties的文件,可以看到Cloudera manager的配置文件。           数据库名为scm,用户名为scm。     (2)用如下命令登录Cloudera server节点的postgresql数
Cloudera Manager离线安装与卸载【按照此文章可以卸载干净,然后执行cm5的bin文件重新安装,不推荐此安装方法】
文章来源:http://www.chinahadoop.cn/group/5/thread/19 最近一直很苦恼,因为一直不知道Clouder Manager的离线安装与卸载,在生产环境中无法直接通过cm安装cdh。终于,皇天不负有心人,让我找到了一份有关搭建yum源来安装的步骤。 一、安装步骤     说明:yum源最好安装到一台有往外的服务器上,为内网服务器提供一个yum源;如果
Centos6离线安装CDH5.14.2最全详细教程-7浏览器安装ClouderaManager
概述 文章写得很详细,很长,所以这里分成了7个部分 这里是第四部分 可以先看下第一部分的概述 Centos6离线安装CDH5.14.2最全详细教程-1前言 这里再强调一下注意事项 1. 所有步骤如无特别说明都是要在所有服务器上都执行一遍 2. 所有的准备工作都要做,不要漏了或者错了,不然后面出现各种问题再改都是轻的,由此引发的其他的问题就严重多了 3. 所有操作均在root用户下进...
安装Cloudera Manager Server和Agent过程中遇到的问题
在安装hadoop 的cloudera manager 和 agent的过程中  版本 5.5.1 /opt/cm-5.5.1/etc/init.d/cloudera-scm-server start 是能正常启动的 /opt/cm-5.5.1/etc/init.d/cloudera-scm-agent 则无法正常启动 去查看了一下日志文件,也没有看出是什么错误,然后重装了好几次,还是这
使用cloudera manager重启集群后部分节点无法启动问题解决
    博主遇到过使用cloudera manager重启集群后出现多个节点所有功能均无法连接,对该主机功能进行单点启动时并无报错信息,因为查询不到报错信息,就无法针对性的进行修复,所以进行多次测试定位到问题原因:    常见单节点无法启动导致原因为cm连接异常,命令传输超时,并且日志中无报错,此时应重启该节点cm agent应用    cm重启方法:找到cm存放路径(如安装路径不同和安装版本不同...
离线安装Cloudera Manager 5和CDH5(最新版5.12.0) 完全教程
关于CDH和Cloudera Manager CDH (Cloudera's Distribution, including Apache Hadoop),是Hadoop众多分支中的一种,由Cloudera维护,基于稳定版本的Apache Hadoop构建,并集成了很多补丁,可直接用于生产环境。 Cloudera Manager则是为了便于在集群中进行Hadoop等大数据处理相关的服务
Cloudera集群初始搭建
2017 08 04 版本 初始1.0 (受用之后的一赞,将是鄙人持久的动力) 目录 一,集群规划 二,linux系统环境准备 1,安装linux centos6.5 2,配置网络 3,增加普通用户 4,配置普通用户的sudo权限 5,关闭防火墙 6,禁用Selinux 7, 设置文件打开数量和用户最大进程数 8,yum源跟换和添加 9,卸载Centos自带的JD...
Cloudera 向集群添加主机
本文将概述使用Cloudera Manager向集群中添加主机的操作。首先,我们这里是采用的离线安装模式安装的,所以对于你新的主机你必须配置好如下几个内容: 安装好JDK 安装好Cloudera Manager Agent 配置Agent指向Server 操作流程Created with Raphaël 2.1.0添加节点在CM界面选择主机向集群添加新主机选择管理的主机(确保你已经配好了Agent服
cloudera manager+CDH 完全离线安装文档 版本5.7
       写完全离线的目的有几点:1、数据安全考虑,服务器不允许与外部连通。2、对于下载速度很慢的时候,可以考虑。3、集群规模大,人手不够,又不会写shell脚本一键安装。cloudera manager 下载地址:http://archive.cloudera.com/cm5/CDH 下载地址:http://archive.cloudera.com/cdh5/说明:测试中的IP为伪造IP,本...
集群安装(一)Cloudera Manager离线部署CDH
Cloudera Manager离线部署CDH文档 一、说明 二、系统环境搭建 1、网络配置(所有节点) 2、SSH免密码登录 3、关闭防火墙 4、关闭SELINUX 5、安装JDK 6、设置NTP 7、安装配置MySql 8、下载依赖包 三、Cloudera Manager Server&Agent安装 1、安装Cloudera Manager Server&Agent...
Centos6离线安装CDH5.14.2最全详细教程-5安装Cloudera Manager
概述 文章写得很详细,很长,所以这里分成了7个部分 这里是第二部分, 第一部分:前言 第二部分:环境准备1 第三部分:环境准备2 第四部分:安装Mysql数据库 第五部分:安装Cloudera Manager主服务 第六部分:配置Mysql数据库 第七部分:所有节点安装Cloudera Manager从服务及大数据组件 记得先看下第一部分的概述 这里再强调一下注意事项 1....
Cloudera Manager uuid 文件导致无法添加主机问题
初接触CM不久,公司要求部署一个三个节点的测试集群(虚拟机)。 dn01,dn02,dn03 整个集群安装好之后才发现dn03的硬盘分的太小了,导致CM集群一些项以红色警告方式提醒,先尝试重新挂在一块硬盘,由于linux水平有限,没有成功,只好重新安装dn03。 删除CM集群上的Cluster 1 集群,然后将移除dn03,然后重装虚拟机,安装好之后,jdk、ssh等配置完毕。 我直接将
安装CDH5.15.0过程详细记录¥坑与解决办法(20180724)
参考了网上很多教程,踩过很多坑,总结一篇,仅供参考: node1 , node2, node3 ;其中node1 为主节点,2 3为从节点。 机子内存太小不建议安装测试学习,否则过程很难受 。。。说实话8g不建议考虑。16g还勉强。。。 最好是32g+的服务器。   下载相关软件包(推荐使用迅雷下载,国外源太慢,迅雷下载比较快原理自己了解) 名称 下载地址 1. JDK1.7...
cloudera manager5删除uuid,主机找不到集群
cloudera manager给开发人员节省了维护成本,但是一旦cm节点不能启动,将导致集群组件无法管理。 问题描述: 服务器重启后cm节点未能正常启动,在排查问题过程中误删除uuid文件。该文件在cm-5.3.6/lib/cloudera-scm-agent目录下。 解决步骤: agent无法启动,由于对7180、7182混淆,以为端口配置错误,修改后出现闪退,原因是修改配置文件写错,cm
Cloudera CDH增加节点到集群
新增一台机器步骤: 设置所有集群的hosts一样,加入新增的机器 vim /etc/sysconfig/network 修改hostnmae为aaa hostname aaa cp  cloudera-manager-el5-cm4.6.2_x86_64.tar.gz 到新增的机器上 安装jdk环境,最好jdk版本统一,参考此处
Cloudera Impala 常见问题(翻译)
Cloudera Impala 常见问题 下面是 Clouder Impala 产品常见问题的目录。 继续阅读: Trying Impala Impala System Requirements Supported and Unsupported Functionality In Impala How do I? Impala Performance Impala Use Case...
Cloudera Manager 5.12.0图文详解安装过程
这里介绍的是cdh5的离线安装方式,需要的文件提前准备好,安装过程会快一些。 安装前提:机器配置内存一定要高,我这里安装的虚拟机均是redhat7:内存分别是6G,4G,4G。 准备的文件: mysql-connector-java-5.1.38.jar cloudera-manager-centos7-cm5.12.0_x86_64.tar.gz manifest.json CDH-5.1...
【CDH】centos6 下离线安装CM5.16和CDH5.16完全教程
centos6 下离线安装CM5.16和CDH5.16完全教程关于CDH和Cloudera Manager系统环境安装包准备网络配置(所有节点)打通ssh,设置ssh无秘登录安装jdk安装mysql 关于CDH和Cloudera Manager CDH (Cloudera’s Distribution, including Apache Hadoop),是Hadoop众多分支中的一种,由Cloud...
离线安装CDH5.12.1
1.准备工作 准备三台虚拟机,内存和硬盘空间尽量给足,配置host映射。(我的是三台节点配置相同:6G内存,50G硬盘,名称分别是cdh1、cdh2、cdh3) 下载安装包 Cloudera Manager:下载地址 http://archive.cloudera.com/cm5/cm/5/ 选择 cloudera-manager-centos7-cm5.12.1_x86_64.tar.gz (7
CDH6.1.x离线安装
1. 安装文件下载 cloudera manager文件下载,红框中的不需要下载。 下载allkeys.asc文件,如下: cdh6文件下载,下载对应的系统文件: 2. 系统配置 2.1 环境介绍 CentOS7.5.1804 3.10.0-862.el7.x86_64 2.2 网络环境配置 所有节点配置/etc/hostname、/etc/hosts文件 2.3 ...
升级Cloudera Manager 5 到最新版本(官方文档翻译)
将cloudera manager 5 升级到最新版本需要最低角色为:管理员这个手册适合所有版本的Cloudera manager 5 升级在绝大多数情况下,可以不必关闭cdh服务来完成升级过程,但需要关闭几个依赖的服务。当Cloudera manager升级的时候,CDH实例会继续运行不受影响。升级过程并不会影响CDH的安装,升级完cloudera manager后最好升级cdh4集群到cdh5...
cloudera manager的agent安装时候 出现的问题
/tmp/scm_prepare_node.K0tOAFSy  using SSH_CLIENT to get the SCM hostname: 10.221.5.140 33607 22  opening logging file descriptor  正在启动安装脚本... 正在获取安装锁... BEGIN flock 4 END (0)  正在检测根权限... effectiv...
[解决] Cloudera Manager安装锁的问题
[解决] Cloudera Manager安装锁的问题
CDH5实践(四)Cloudera Manager 5安装中碰到的一些问题和解决办法
问题一:安装过程中会出现失败,查看detail信息中,提示 host 无法连接 解决:前面的文章中漏了对主机host的一个修改, 除了修改 /etc/hosts 文件外,我们还需要修改修改  /etc/sysconfig/network 文件,并且需要重启才能生效。 问题二:CDH5 安装中,hive和monitor host 都需要数据库,除了postgresql外,是否可以
CDH cm节点高可用方案
近期因工作需要,对CDHcm管理节点的高可用做了些相关的测试,测试大概分2部分。具体如下,希望有兴趣的同学可以多交流学习。   1. cm节点关于postgres数据库的备用节点高可用。   2. cm节点关于mysql数据库的备用节点高可用。 测试需求:因线上cm管理节点是单点,一旦cm管理节点的服务器宕机,那么整个CDH集群的管理就受到影响。主要是不方便管理。如果熟悉相关的命令,
CDH在Centos 7离线安装
由于本人原来都是采用在线安装方式,使用公司港台服务器代理,速度还是很可观的。不过最近要求在Centos 7上离线安装,有点坑,网上的很多都是抄来抄去,安装到一半可能就走不下去,这里是本人亲测,遇到的问题都一一说明并且解决掉。这里趟了一遍,希望对读者有帮助。不说废话,进入主题: Cloudera官网给出三种安装方式,我们采用PathC,使用下载好的包进行安装。我这里只使用本地虚拟机来进行安装说明:
手动安装cloudera manager 5.x(tar包方式)详解
官方共给出了3中安装方式:第一种方法必须要求所有机器都能连网,由于最近各种国外的网站被墙的厉害,我尝试了几次各种超时错误,巨耽误时间不说,一旦失败,重装非常痛苦。第二种方法下载很多包。第三种方法对系统侵入性最小,最大优点可实现全离线安装,而且重装什么的都非常方便。后期的集群统一包升级也非常好。这也是我之所以选择离线安装的原因。 包括MySQL数据库我均采用二进制安装包方式配置,且大部分步骤仅
【干货】离线安装Cloudera Manager 5和CDH5(最新版5.13.0)详细过程
—— 原文发布于本人的微信公众号“大数据与人工智能Lab”(BigdataAILab),欢迎关注。   Cloudera是Hadoop生态系统中,规模最大、知名度最高的公司。Cloudera对Hadoop进行了封装,简化安装过程,使用Cloudera可以快速、简...
Cloudera Manager 中如何正确的下线节点
最低要求的角色:超级管理员(所谓超级管理员就是 admin)可以通过两种方式从集群中删除主机:使用 Cloudera Manager 删除;从集群 A 中删除主机,将其提供给由 Cloudera Manager 管理的其他集群;两种方法都会涉及退役主机(decommission)、删除角色(role)、删除托管服务软件(service),但最终都会保留数据目录。使用 Cloudera Manage...
修改cloudera manager管理主机名(转)
I installed the Cloudera Manager 4 on a 1GbE default setup on eth0/bond0 and now I need to use 10GbE or Infiniband IB or 40GbE without the reinstall. How I did it!!  The assumptions here are that y...
在clouder manager中使用Flume处理数据链
目标: 在Cloudera Manager中创建两个Flume的Agent,Agent1从local file中获取内容,写入到kafka的队列中。Agent2以Agent1的sink作为source,将数据从kafka中读取出来,写入到HDFS中。 注意 我的Cloudera Manager使用的是CDH5.8的Parcel,在这个版本中,Kafka使用的是0.9.
Cloudera在Ubuntu的安装部署
1. 安装前的准备  准备好Ubuntu离线source,配置/etc/apt/source.list  准备好Cloudera Hadoop离线source,配置/etc/apt/source.list.d/cloudera.list  同步集群的/etc/hosts  ssh免密码登录,cloudera-scm-server所在机器能够免密码登录其他cloudera-scm
Cloudera Manager安装(只需要在NODE1主节点执行上述操作便会自动分配部署到各节点)
总帖:CDH 6系列(CDH 6.0、CHD6.1等)安装和使用 Cloudera Manager安装(只需要在NODE1做) 1.配置本地repo源 1.下载CM6.0的安装包,地址为: https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/cloudera-manager...
cloudera manager安装问题记录二
问题一 IOException thrown while collecting data from host: No route to host  因为agent机器开启了防火墙 解决: service iptables stop     问题二 在以下主机上,组 'hadoop' 不包括用户 'hdfs': 解决: usermod -a -G hadoop hdfs ...
Centos7离线搭建安装配置Cloudera-Manager5.9.3-说明书.docx
Centos7离线搭建安装配置Cloudera-Manager5.9.3-说明书
手把手部署CDH(5.12.1)完全离线模式安装超级详细攻略
前言 搭建小型BI大数据计算集群,由于资源有限(你懂的)部署的模式为CDH官方建议的最低配版本,没有配备HA,(我们还是相信惠普的服务器 此处手动滑稽)。 CDH背景 CDH (Cloudera's Distribution, including Apache Hadoop),是Hadoop众多分支中的一种,由Cloudera维护,基于稳定版本的Apache Hadoop构建,并集成了很多补...
cloudera manager5.11.0离线安装详细步骤
第一步: (下载安装包) cm5.11.0 bin包 http://archive.cloudera.com/cm5/installer/5.11.0/ cm5.11.0 依赖的rpm包 http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.11/RPMS/x86_64/ cm5.11.0 依赖的parcels包,也就是cdh包...
文章热词 机器学习教程 Objective-C培训 交互设计视频教程 颜色模型 设计制作学习
相关热词 mysql关联查询两次本表 native底部 react extjs glyph 图标 java学习资源 下载 cloudera大数据培训