网站建设个人博客,中国形象设计网,做网站好处,如何做网站的后台管理目录 Part.01 关于HDP Part.02 核心组件原理 Part.03 资源规划 Part.04 基础环境配置 Part.05 Yum源配置 Part.06 安装OracleJDK Part.07 安装MySQL Part.08 部署Ambari集群 Part.09 安装OpenLDAP Part.10 创建集群 Part.11 安装Kerberos Part.12 安装HDFS Part.13 安装Ranger …目录 Part.01 关于HDP Part.02 核心组件原理 Part.03 资源规划 Part.04 基础环境配置 Part.05 Yum源配置 Part.06 安装OracleJDK Part.07 安装MySQL Part.08 部署Ambari集群 Part.09 安装OpenLDAP Part.10 创建集群 Part.11 安装Kerberos Part.12 安装HDFS Part.13 安装Ranger Part.14 安装YARNMR Part.15 安装HIVE Part.16 安装HBase Part.17 安装Spark2 Part.18 安装Flink Part.19 安装Kafka Part.20 安装Flume
四、基础环境配置
1.SSH免密登录
在hdp01、hdp02上生成公钥配置免密登录到其他节点
ssh-keygen -t rsa -f ~/.ssh/id_rsa -C username_root
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root192.168.111.201
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root192.168.111.202
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root192.168.111.203
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root192.168.111.204
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root192.168.111.2052.ansible安装
配置ansible和rhel-system-roles创建配置文件
mkdir /root/ansible
cd /root/ansible
cp /etc/ansible/ansible.cfg /root/ansible/修改配置文件/root/ansible/ansible.cfg
[defaults]
inventory /root/ansible/inventory
ask_pass false
remote_user root配置inventory文件/root/ansible/inventory
[hdp:children]
nn
dn
[nn]
192.168.111.201 hostnamehdp01
192.168.111.202 hostnamehdp02
[dn]
192.168.111.203 hostnamehdp03
192.168.111.204 hostnamehdp04
192.168.111.205 hostnamehdp053.修改hostname
创建playbook/root/ansible/hostname.yml
---
- name: modify hostnamehosts: alltasks:- name: modify hostname permanentlyraw: echo {{ hostname | quote }} /etc/hostname- name: modify hostname temporarilyshell: hostname {{ hostname | quote }}执行并确认
ansible-playbook /root/ansible/hostname.yml
ansible all -m shell -a hostname4.修改hosts列表
在nn01上修改主机列表/etc/hosts
192.168.111.201 hdp01.hdp.com hdp01
192.168.111.202 hdp02.hdp.com hdp02
192.168.111.203 hdp03.hdp.com hdp03
192.168.111.204 hdp04.hdp.com hdp04
192.168.111.205 hdp05.hdp.com hdp05分发至其他节点
ansible all -m template -a src/etc/hosts dest/etc/hosts5.安装基础软件
安装vim等基础软件/root/ansible/packages.yml
---
- hosts: alltasks:- name: install packagesyum:name:- pciutils- bash-completion- vim- chronystate: present6.关闭firewall和SELinux
关闭firewall
ansible all -m service -a namefirewalld statestopped enabledno
ansible all -m shell -a systemctl status firewalld | grep Active关闭SELinux
ansible all -m selinux -a policytargeted statedisabled
ansible all -m shell -a getenforce7.NTP时钟
以hdp01为时钟源其余节点从nn01进行时钟同步 服务端hdp01 修改配置文件/etc/chrony.conf
# 不指定外部NTP源
# 允许本网段其节点作为客户端访问
allow 192.168.111.0/24
# 如果时间服务可不用则使用本地时间作为标准时间授权层数为10
local stratum 10重启服务
systemctl restart chronyd客户端hdp02-hdp05 安装ntp时钟/root/ansible/timesync.yml
---
- hosts: 192.168.111.202,dnvars:timesync_ntp_servers:- hostname: 192.168.111.201iburst: yesroles:- rhel-system-roles.timesync执行
ansible-playbook /root/ansible/timesync.yml确认时钟同步情况
ansible 192.168.111.202,dn -m shell -a chronyc sources -v8.磁盘分区、文件系统及挂载目录
创建分区parted.yml文件
---
- hosts: alltasks:- name: parted devicesparted:device: {{ item }}number: 1label: gptstate: presentloop:- /dev/sdb- /dev/sdc- /dev/sddignore_errors: yes确认结果
ansible all -m shell -a lsblk -f创建文件系统mkfs.yml文件
---
- hosts: alltasks:- name: mkdir of nnfile:path: /data01state: directorywhen: inventory_hostname in groups[nn]- name: mkdir of dnfile:path: {{ item }}state: directoryloop:- /data01- /data02- /data03when: inventory_hostname in groups[dn]- name: mkfsfilesystem:fstype: xfsdev: {{ item }}loop:- /dev/sdb1- /dev/sdc1- /dev/sdd1ignore_errors: yes- name: mount of nnmount:path: /data01src: /dev/sdb1fstype: xfsstate: mountedwhen: inventory_hostname in groups[nn]- name: mount of dnmount:path: {{ item.p_dir }}src: {{ item.s_dir }}fstype: xfsstate: mountedloop:- { p_dir: /data01, s_dir: /dev/sdb1 }- { p_dir: /data02, s_dir: /dev/sdc1 }- { p_dir: /data03, s_dir: /dev/sdd1 }when: inventory_hostname in groups[dn]
