Cloudera Manger - EC2 설치 [ install in EC2 ]

[ 환경 ] 

RedHat Version 7

3대이상 EC2 Server

 

 

[ EC2 보안그룹 셋팅 ]

해당 Document를 참조 :

https://docs.cloudera.com/management-console/cloud/environments/topics/mc-environment-aws-security-groups.html

 

 

[ 설치 ]

- 관련 패키지 설치

$ sudo yum update -y && sudo yum install wget -y && sudo yum install -y ntp && sudo systemctl start ntpd && sudo systemctl enable ntpd && sudo yum install iptables-services -y

$ sudo chkconfig ntpd on

$ wget ftp://ftp.pbone.net/mirror/ftp.scientificlinux.org/linux/scientific/7.6/x86_64/os/Packages/libtirpc-devel-0.2.4-0.15.el7.x86_64.rpm

$ sudo yum install  libtirpc-devel-0.2.4-0.15.el7.x86_64.rpm

 

- Selinux 방화벽 설정

$ sudo vi /etc/selinux/config

SELINUX=disabled

$ sudo systemctl start iptables

//Saving your firewall rules can be done as follows:

$ sudo service iptables save

$ sudo chkconfig iptables off

$ sudo service ntpd start

$ sudo service iptables stop

$ sudo systemctl disable chronyd.service

 

- Install Hadoop 

$ sudo apt-get install openssh-server

$ sudo apt-get install ssh

$ sudo apt-get install rsync

 

- ssh key 설정

$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

$ chmod 0600 ~/.ssh/authorized_keys

$ ssh localhost

//if still popup password run this

$ ssh-keygen

enter

ssh-add

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

$ ssh localhost

 

- Aws pem파일 설정

winSCP move to name.pem in server

$ ls -l

$ mv name.pem ./.ssh/id_rsa

 

- 멀티클래스터를 위해 EC2 create image

Aws 콘솔에서 인스턴스 우클릭 -> 이미지 -> 이미지 생성

생성된 이미지AMI로 EC2 인스턴스 만들기

 

- get the source installer

$ wget https://archive.cloudera.com/cm6/6.3.1/cloudera-manager-installer.bin

- make it executable

$ sudo chmod u+x cloudera-manager-installer.bin

 

- install

$ sudo ./cloudera-manager-installer.bin

next next next yes ... now u ll get ...

http:publicdns:7180에 접속해서 설치를 계속 수행

next do next next next .. follow this

 

 

 

[ 참고 ]

* Video : 

https://www.youtube.com/watch?v=ZfmlmQwCeZc

* Cloudera Manager Document :

https://docs.cloudera.com/cloudera-manager/7.1.1/installation/topics/cdpdc-version-and-download-information.html

* Cloudera Altus Director Informations : https://docs.cloudera.com/documentation/director/latest/topics/director_deployment_start_launcher.html