Cloudera CDH

CDH安装

免密登录

[root@node123 ~]# ssh-keygen
[root@node123 ~]# ssh-copy-id node1
[root@node123 ~]# ssh-copy-id node2
[root@node123 ~]# ssh-copy-id node3

MySQL

MySQL Server安装

选择一台机器安装MySQL Server，安装过程略，参考 linux 文档。

创建数据库

create database cmf DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database scm DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database rman DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database metastore DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database sentry DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database nav DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database navms DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

grant all on cmf.* to 'cmf'@'%' identified by '123456';
grant all on scm.* to 'scm'@'%' identified by '123456';
grant all on hive.* to 'hive'@'%' identified by '123456';
grant all on amon.* to 'amon'@'%' identified by '123456';
grant all on rman.* to 'rman'@'%' identified by '123456';
grant all on hue.* to 'hue'@'%' identified by '123456';
grant all on metastore.* to 'hive'@'%' identified by '123456';
grant all on sentry.* to 'sentry'@'%' identified by '123456';
grant all on nav.* to 'nav'@'%' identified by '123456';
grant all on navms .* to navms @'%' identified by '123456';
grant all on oozie.* to 'oozie'@'%' identified by '123456';

flush privileges;

Apache HTTP server

主节点安装

[root@node1 ~]# systemctl start httpd & systemctl enable httpd

httpd的默认网页文件存放位置是/var/www/html

Cloudera Manager

有两种安装方式：

手动安装

• 使用手动在每个节点上安装rpm包

• 修改配置文件，让sever和agent互相通信

• 启动组件

使用CM安装

• 在主节点安装CM

• 在CM中添加从节点

• 通过自定义存储库的方式在从节点上安装

安装相关组件

本文选择第一种安装方式：在每个节点上安装相应的rpm软件包。

node1	node2	node3
server
agent	agent	agent
daemons	daemons	daemons

[root@node123 cloudera-repos]# rpm -i cloudera-manager-deamons-6.3.1-1466458.el7.x86_64.rpm 
[root@node123 cloudera-repos]# rpm -i cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
[root@node1 cloudera-repos]# rpm -i cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm 

修改配置文件

cloudera-scm-agent

[root@node123 ~]# vim /etc/cloudera-scm-agent/config.ini
server_host: node1

cloudera-scm-server

[root@node123 ~]# vim /etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=node3
com.cloudera.cmf.db.name=cmf
com.cloudera.cmf.db.user=cmf
com.cloudera.cmf.db.password=123456

初始化数据库

[root@node1 ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root -h node3

• mysql第一个参数是代表数据类型
• scm是第二个参数，代表哪个库
• root是第三个参数，代表哪个用户

启动

# 主节点启动Server
[root@node1 ~]# systemctl start cloudera-scm-server & systemctl enable cloudera-scm-server
[root@node123 ~]# systemctl start cloudera-scm-agent & systemctl enable cloudera-scm-agent

WebUI

http://node1:7180/cmf/login

账号/密码：admin/admin

使用第一种安装方式，可以看到被CM管理的主机列表

自定义存储库

Apapche 托管

[root@node1 html]# mkdir -p /var/www/html/cloudera-repos/cdh6
[root@node1 html]# mkdir -p /var/www/html/cloudera-repos/cm6

cloudera-repo

vim /etc/yum.repos.d/cloudera-repo.repo

[root@node1 ~]# vim /etc/yum.repos.d/cloudera-repo.repo

[cloudera-repo]
name=cloudera-repo
baseurl=http://node1/cloudera-repos/
enabled=1
gpgcheck=0

安装

安装完成

• 删除本地自定义存储库
• 停止并禁用Apache HTTP Server