在Docker上部署Ambari


最近在搞Docker上部署Ambari

此文梳理了一下整个部署的过程以及部署中遇到的问题和解决办法

下一篇博文,会写:如何在Centos上离线部署Ambari 2.5.0.3

登录服务器,查看服务器基本信息

[root@ai-main ~]# df -Th
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sda2      xfs       100G   40G   61G  40% /
devtmpfs       devtmpfs   16G     0   16G   0% /dev
tmpfs          tmpfs      16G   20K   16G   1% /dev/shm
tmpfs          tmpfs      16G  106M   16G   1% /run
tmpfs          tmpfs      16G     0   16G   0% /sys/fs/cgroup
/dev/sda5      xfs       815G  3.6G  811G   1% /data
/dev/sda1      xfs       497M  143M  354M  29% /boot
tmpfs          tmpfs     3.2G     0  3.2G   0% /run/user/0
[root@ai-main ~]# uname -r
3.10.0-514.el7.x86_64
[root@ai-main ~]# more /etc/redhat-release
CentOS Linux release 7.3.1611 (Core) 

关闭防火墙,关闭SELinux

关闭THP服务,使用两个命令查看

[root@ai-main selinux]# cat /sys/kernel/mm/transparent_hugepage/defrag
[always] madvise never
[root@ai-main selinux]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
这个状态就是启用的

编辑rc.local文件

 vim /etc/rc.d/rc.local
增加下列内容:
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
 echo never > /sys/kernel/mm/transparent_hugepage/enabled
 fi
 if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
 echo never > /sys/kernel/mm/transparent_hugepage/defrag
 fi
保存退出,然后赋予rc.local文件执行权限:
chmod +x /etc/rc.d/rc.local
最后重启系统,以后再检查THP,下面这样就是被禁用了
[root@ai-main ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
[root@ai-main ~]# cat /sys/kernel/mm/transparent_hugepage/defrag 
always madvise [never]

生成SSH秘钥

[root@ai-main ~]# ssh-keygen -t rsa

安装http

安装httpd
[root@ai-main ~]# yum install httpd -y
Installed:
  httpd.x86_64 0:2.4.6-45.el7.centos.4                                                                                                                                                       
Dependency Installed:
  apr.x86_64 0:1.4.8-3.el7                apr-util.x86_64 0:1.5.2-6.el7                httpd-tools.x86_64 0:2.4.6-45.el7.centos.4                mailcap.noarch 0:2.1.41-2.el7               
Complete!

在/etc/yum.repo.d/下创建ambari.repo文件

ambari.repo内容如下:
[HDP-2.6.0.3-8]
name=HDP-2.6.0.3-8
baseurl=http://ai-main/HDP/centos7/
gpgcheck=0
gpgkey=http://ai-main/HDP/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1 

在/etc/yum.repo.d/下创建HDP.repo文件

HDP.repo文件内容如下
[HDP-2.6.0.3-8]
name=HDP-2.6.0.3-8
baseurl=http://ai-main/HDP/centos7/
gpgcheck=0
gpgkey=http://ai-main/HDP/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1 

在/etc/yum.repo.d/下创建HDP-UTILS.repo文件

HDP-UTILS.repo文件内容如下:
[HDP-UTILS-1.1.0.21]
name=HDP-UTILS-1.1.0.21
baseurl=http://ai-main/HDP-UTILS-1.1.0.21-centos7/
gpgcheck=0
gpgkey=http://ai-main/HDP-UTILS-1.1.0.21-centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

开始安装ambari

[root@ai-main yum.repos.d]# yum install -y ambari-server                                                     
Installed:
  ambari-server.x86_64 0:2.5.0.3-7                                                                                                                                                           
Complete!

安装mysql

#yum install mysql-server
安装mysql-server失败,如下:
[root@ai-main java]# yum install mysql-server
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.sina.cn
 * extras: mirrors.sina.cn
 * updates: mirrors.sina.cn
No package mysql-server available.
Error: Nothing to do
查资料发现是CentOS 7 版本将MySQL数据库软件从默认的程序列表中移除,用mariadb代替了

有两种解决办法:

1.安装MariaDB数据库

这里提一嘴,MariaDB数据库

MariaDB数据库管理系统是MySQL的一个分支,主要由开源社区在维护,采用GPL授权许可。

开发这个分支的原因之一是:甲骨文公司收购了MySQL后,有将MySQL闭源的潜在风险,因此社区采用分支的方式来避开这个风险。

MariaDB的目的是完全兼容MySQL,包括API和命令行,使之能轻松成为MySQL的代替品。

安装mariadb后显示的也是 MariaDB [(none)]> ,看起来有点不习惯。于是使用了下面的办法。

2.官网安装mysql

[root@ai-main java]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm   安装mysql有两种方式,一种是安装mariadb,另一种使用网络安装员

[root@ai-main java]#  rpm -ivh mysql-community-release-el7-5.noarch.rpm

Installed:
  mysql-community-client.x86_64 0:5.6.37-2.el7                   mysql-community-libs.x86_64 0:5.6.37-2.el7                   mysql-community-server.x86_64 0:5.6.37-2.el7                  

安装成功后,重启服务,登录mysql

[root@ai-main java]# service mysqld restart
[root@ai-main ~]# chkconfig mysqld on   #设置mysql开机自启

[root@ai-main java]# mysql -u root -p      #登录mysql
Enter password: 
默认没有密码,回车即可
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.37 MySQL Community Server (GPL)
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+
3 rows in set (0.00 sec)
mysql> 

在mysql中使用下面的语句

CREATE DATABASE ambari; 
use ambari; 
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari'; 
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%'; 
CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambari'; 
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost'; 
CREATE USER 'ambari'@'ai-main' IDENTIFIED BY 'ambari'; 
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'ai-main'; 
FLUSH PRIVILEGES; 
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql 
show tables; 
use mysql; 
select Host,User,Password from user where user='ambari'; 
CREATE DATABASE hive; 
use hive; 
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive'; 
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%'; 
CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive'; 
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost'; 
CREATE USER 'hive'@'ai-main' IDENTIFIED BY 'hive'; 
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'ai-main'; 
FLUSH PRIVILEGES; 
CREATE DATABASE oozie; 
use oozie; 
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie'; 
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%'; 
CREATE USER 'oozie'@'localhost' IDENTIFIED BY 'oozie'; 
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost'; 
CREATE USER 'oozie'@'ai-main' IDENTIFIED BY 'oozie'; 
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'ai-main'; 
FLUSH PRIVILEGES;

安装ambari-server

[root@ai-main opt]# ambari-server setup      #开始安装ambari-server
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y     # Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/jdk1.8.0_91
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): ai-main 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (bigdata): 
Re-enter password: 
Configuring ambari database...
Copying JDBC drivers to server resources...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.5.0.3.7.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
[root@ai-main opt]# clear
[root@ai-main opt]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.

到这里在宿主机Ambari环境准备完成了

ambari搭建的主机如下:

ai-main    主机

ai-node1    docker容器1,从节点node1

ai-node2    docker容器2,从节点node2

下面开始安装docker,配置docker环境

机器分布如下:

安装docker

安装docker
[root@ai-main ~]# yum install docker-io -y
Installed:
  docker.x86_64 2:1.12.6-32.git88a4867.el7.centos                                                                                                                                            
Complete!

启动docker

[root@ai-main ~]# service docker start
[root@ai-main opt]# docker search centos       #搜索centos系统
INDEX       NAME                                         DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
docker.io   docker.io/centos                             The official build of CentOS.                   3608      [OK]       
docker.io   docker.io/ansible/centos7-ansible            Ansible on Centos7                              100                  [OK]
docker.io   docker.io/jdeathe/centos-ssh                 CentOS-6 6.9 x86_64 / CentOS-7 7.3.1611 x8...   81                   [OK]
docker.io   docker.io/tutum/centos                       Simple CentOS docker image with SSH access      33                   
docker.io   docker.io/imagine10255/centos6-lnmp-php56    centos6-lnmp-php56                              30                   [OK]
docker.io   docker.io/gluster/gluster-centos             Official GlusterFS Image [ CentOS-7 +  Glu...   20                   [OK]
docker.io   docker.io/kinogmt/centos-ssh                 CentOS with SSH                                 16                   [OK]
docker.io   docker.io/centos/php-56-centos7              PHP 5.6 platform for building and running ...   9                    
docker.io   docker.io/guyton/centos6                     From official centos6 container with full ...   7                    [OK]
docker.io   docker.io/openshift/base-centos7             A Centos7 derived base image for Source-To...   7                    
docker.io   docker.io/openshift/mysql-55-centos7         DEPRECATED: A Centos7 based MySQL v5.5 ima...   6                    
docker.io   docker.io/centos/python-35-centos7           Python 3.5 platform for building and runni...   5                    
docker.io   docker.io/darksheer/centos                   Base Centos Image -- Updated hourly             3                    [OK]
docker.io   docker.io/openshift/jenkins-2-centos7        A Centos7 based Jenkins v2.x image for use...   3                    
docker.io   docker.io/openshift/ruby-20-centos7          DEPRECATED: A Centos7 based Ruby v2.0 imag...   3                    
docker.io   docker.io/blacklabelops/centos               CentOS Base Image! Built and Updates Daily!     1                    [OK]
docker.io   docker.io/indigo/centos-maven                Vanilla CentOS 7 with Oracle Java Developm...   1                    [OK]
docker.io   docker.io/openshift/php-55-centos7           DEPRECATED: A Centos7 based PHP v5.5 image...   1                    
docker.io   docker.io/pivotaldata/centos-gpdb-dev        CentOS image for GPDB development. Tag nam...   1                    
docker.io   docker.io/pivotaldata/centos-mingw           Using the mingw toolchain to cross-compile...   1                    
docker.io   docker.io/jameseckersall/sonarr-centos       Sonarr on CentOS 7                              0                    [OK]
docker.io   docker.io/miko2u/centos6                     CentOS6 日本語環境                                   0                    [OK]
docker.io   docker.io/pivotaldata/centos                 Base centos, freshened up a little with a ...   0                    
docker.io   docker.io/pivotaldata/centos-gcc-toolchain   CentOS with a toolchain, but unaffiliated ...   0                    
docker.io   docker.io/smartentry/centos                  centos with smartentry                          0                    [OK]
[root@ai-main opt]# docker pull centos        #pull centos的镜像
Using default tag: latest
Trying to pull repository docker.io/library/centos ... 
latest: Pulling from docker.io/library/centos
74f0853ba93b: Pull complete 
Digest: sha256:26f74cefad82967f97f3eeeef88c1b6262f9b42bc96f2ad61d6f3fdf544759b8

在Ambari上部署,docker需要指定IP

要自定义docker网络,并需要指定一个空闲的网段

#创建自定义网络,并指定网段
[root@ai-main opt]# docker network create --subnet=172.18.0.0/24 ambari-network   #这里,需要指定一个空闲的网段,原本以为可以使用和主机一样的网段,结果,网络瞬间挂了,还得去服务器那里修改回来,需要注意 17025d3fdfb1d30691cded810aaaafbbce9eb6813e9ef4d2bb499cc36dd94f9d
[root@ai-main ~]# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
3f86861e1b75        ambari-network      bridge              local               
[root@ai-main ~]# ifconfig                #可以看到自定义的docker网段
br-3f86861e1b75: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.0.1  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::42:31ff:fe9b:78d0  prefixlen 64  scopeid 0x20<link>
        ether 02:42:31:9b:78:d0  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

创建好网络后,就要进入docker容器进行环境安装了

docker run -it  --net ambari-network --ip 172.18.0.10  centos:latest 

这个centos是什么都没有的,都需要自己进行安装,进去查看ip地址,查看不了,需要安装net-tools

 yum install net-tools -y
安装后,使用ifconfig就可以查看IP了

安装SSH

先修改root密码:
[root@5f58f4e94fb8 sshd]# passwd root
Changing password for user root.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@5f58f4e94fb8 opt]# yum install -y openssh-server   #安装ssh
Installed:
  openssh-server.x86_64 0:6.6.1p1-35.el7_3                                                                                                                                                   
Dependency Installed:
  fipscheck.x86_64 0:1.4.1-5.el7             fipscheck-lib.x86_64 0:1.4.1-5.el7             openssh.x86_64 0:6.6.1p1-35.el7_3             tcp_wrappers-libs.x86_64 0:7.6-77.el7            
Complete!

在启动ssh中出现了问题

#安装好ssh后,需要在/var/run/下创建sshd目录
[root@5f58f4e94fb8 opt]# mkdir -p /var/run/sshd
#创建好目录后,启动ssh
[root@5f58f4e94fb8 sshd]# /usr/sbin/sshd -D &
[1] 204
#却发现了报错
[root@5f58f4e94fb8 sshd]# Could not load host key: /etc/ssh/ssh_host_rsa_key
Could not load host key: /etc/ssh/ssh_host_ecdsa_key
Could not load host key: /etc/ssh/ssh_host_ed25519_key

解决办法:输入以下命令

root@5f58f4e94fb8 sshd]# ssh-keygen -q -t rsa -b 2048 -f /etc/ssh/ssh_host_rsa_key -N ''
[root@5f58f4e94fb8 sshd]# ssh-keygen -q -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N ''
[root@5f58f4e94fb8 sshd]# ssh-keygen -t dsa -f /etc/ssh/ssh_host_ed25519_key  -N ''
Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_ed25519_key.
Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub.
The key fingerprint is:
32:aa:9c:11:22:86:1d:53:eb:55:e6:67:fa:d0:3e:2e root@5f58f4e94fb8
The key's randomart image is:
+--[ DSA 1024]----+
|    .   o        |
|   . . +         |
|  o . . . o      |
|.. + .   =       |
|+.o . o S .      |
|o. . . o +       |
|  . .     +      |
| . +    E. .     |
|  +      ..      |
+-----------------+
[root@5f58f4e94fb8 sshd]# /usr/sbin/sshd -D &      #ssh启动正常
[2] 208 

启动ssh后,需要在主机上将ssh秘钥拷贝到docker节点

[root@ai-main opt]# ssh-copy-id 172.18.0.10
The authenticity of host '172.18.0.10 (172.18.0.10)' can't be established.
ECDSA key fingerprint is 56:0b:cb:cd:33:85:6e:74:6c:02:50:91:4c:9c:c1:82.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@172.18.0.10's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '172.18.0.10'"
and check to make sure that only the key(s) you wanted were added.

安装java环境

不需要使用网络源下载安装java,将本地的java包复制到docker上,解压,配置环境变量,并source

docker与host容器互传文件(数据)
docker cp 文件路径 容器id:/容器目录
[root@ai-main opt]# docker cp /opt/jdk-8u91-linux-x64.tar.gz 5f58f4e94fb8:/opt/   #需要在主机上操作

基础环境准备好了,将容器commit为一个镜像ambari-env,然后启动两个容器

这里重点要说一下docker的初始化配置

1.需要指定主机名 -h 参数,要不然主机名自动生成

2.需要指定前面配置好的专属网络,指定IP地址  –net 参数指定网络   –ip参数 指定ip

3. -v /etc/localtime:/etc/localtime:ro  参数是设置容器时间与主机时间同步,会在文章后面说明

4.需要添加 -d 参数使其在后台运行

5.容器开启后,进入容器,查看ip地址,修改 /ets/hosts映射文件

6.需要后台开启ssh服务

7.退出时,不能使用ctrl c或者exit退出,这样容器就会停止,使用 ctrl + q+p退出容器

启动两个容器
docker run -itd --name ai-node1 --net ambari-network --ip 172.18.0.10 -h ai-node1  -v /etc/localtime:/etc/localtime:ro  ambari-env  
25f01601c95ffd6bb2d4f4f58f50281d1d7c3cb834e13c2f69a43cbee5737f13
[root@ai-main ~]# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS                    PORTS               NAMES
25f01601c95f        ambari-env          "/bin/bash"         7 seconds ago        Up 6 seconds                                  ai-node1
3f18c6919166        ambari-env          "/bin/bash"         About a minute ago   Up About a minute                             ai-node2
5f58f4e94fb8        docker.io/centos    "/bin/bash"         19 hours ago         Exited (0) 18 hours ago                       node1
1751cc6aa7d2        docker.io/centos    "/bin/bash"         19 hours ago         Exited (0) 19 hours ago                       elegant_hamilton
[root@ai-main ~]# docker attach 25f01601c95f
[root@ai-node1 /]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.0.10  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::42:acff:fe12:a  prefixlen 64  scopeid 0x20<link>
        ether 02:42:ac:12:00:0a  txqueuelen 0  (Ethernet)
        RX packets 8  bytes 648 (648.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
[root@ai-node1 /]# vi /etc/hosts
[root@ai-node1 /]# /usr/sbin/sshd -D & 
[1] 18

容器都开启后,测试与主机的连通性,ssh登录,测试完毕后,回到主机

Ambari安装

选择HDP版本,选择本地源安装,填写本地目录

点击next,上传ssh秘钥文件,下一步

连通性测试

在这个连通,安装的过程中,遇到了问题

ERROR 2017-09-05 17:13:46,069 main.py:244 - Ambari agent machine hostname (ai-main.insightcredit) does not match expected ambari server hostname (ai-main). Aborting registration. Please check hostname, hostname -f and /etc/hosts file to confirm your hostname is setup correctly
主机名和/etc/hosts中的主机名不对应,我去,平时还没注意
[root@ai-main HDP-UTILS-1.1.0.21-centos7]# hostname
ai-main.insightcredit
[root@ai-main HDP-UTILS-1.1.0.21-centos7]# hostname ai-main  #临时修改主机名   vim /etc/sysconfig/network 修改主机名,重启即可
[root@ai-main HDP-UTILS-1.1.0.21-centos7]# hostname
ai-main

下面就是集群角色分配,初始化配置,安装

在安装中遇到了如下问题

Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-279.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-279.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1']
安装时间不一致,这机器时间明显就不一样。。。。


在上面docker初始化提到了,添加-v /etc/localtime:/etc/localtime:ro参数是为了和主机时间同步

在弄这里的时候,博主将docker容器关掉之后,重新启动的,很费事

最后,只需要等待安装就好了

安装后的页面,还有个别小问题需要再去搞一搞


总结一下:

此文介绍了在Docker上部署ambari,如果需要快速搭建开发环境可以参考,但是实际环境中,并不推荐使用,使用docker存数据不太靠谱,集群是伪分布式,服务器资源占用率高

注意:

在重启docker容器时,需要重新开启ssh,重新修改/etc/hosts文件,修改hosts文件后,启动ambari-agent服务,source /etc/profile文件。要不然在ambari中显示主机丢失心跳包,找不到该主机。

如果觉得本博客对您有帮助,可以赞助作者哦~


赫墨拉

我是一个喜爱大数据的小菜鸡,这里是我分享我的成长和经历的博客

You may also like...

8 Responses

  1. 马彪说道:

    可以,有机会我也参考着撘一套。

  2. 李鹏辉说道:

    看起来都高大上,哈哈

  3. 柴健说道:

    您好,有幸拜读了您的文章,目前在学习用三台服务器要搭建一个大数据实验环境用,有两个问题想请教您 1.用docker + ambari这种方式部署和直接服务器上部署,性能上差别大么?
    2.您推荐的这种方法。可以直接部署在三台服务器上么? (每个服务器一个docer这种好呢还是 每个服务器有几个docker节点 然后把所有的docker节点建立统一的集群这种好?)三台服务器两cpu12核/64G内存/4T硬盘 方便留个QQ请教一下么?

  4. 柴健说道:

    您好,有幸拜读了您的文章,目前在学习用三台服务器要搭建一个大数据实验环境用,有两个问题想请教您 1.用docker + ambari这种方式部署和直接服务器上部署,性能上差别大么?
    2.您推荐的这种方法。可以直接部署在三台服务器上么? (每个服务器一个docer这种好呢还是 每个服务器有几个docker节点 然后把所有的docker节点建立统一的集群这种好?)三台服务器两cpu12核/64G内存/4T硬盘 方便留个QQ请教一下么?

    • 赫墨拉说道:

      感谢您的留言。
      当时我在写这篇文章的时候,服务器资源有限,仅有一台服务器,但想在该服务器上实现集群部署,于是使用了docker + ambari。
      现在回答您的问题:
      1.正如我当时的情况,使用docker+ambari的前提是,服务器资源有限,但想实现集群部署,可以使用这种方法。
      关于性能问题:使用docker容器作为集群节点,将会共享宿主机的资源,性能会大打折扣
      在实际应用环境中,不建议使用docker,涉及到系统稳定性和安全性,服务器远比docker可靠,但ambari可以作为搭建、管理大数据平台的一种选择。
      2.鉴于您的服务器资源情况,不推荐使用docker + ambari的形式,直接使用ambari在服务器上搭建集群即可。

      • 匿名说道:

        磁盘不要组raid,datanode挂多个硬盘,提高写速度,单独搭建好点,vmvar的 esxi能把你的物理机搭建成云平台,找个key 以上紧代表个人意见

      • che说道:

        磁盘不要组raid,datanode挂多个硬盘,提高写速度,单独搭建好点,vmvar的 esxi能把你的物理机搭建成云平台,找个key 以上紧代表个人意见

柴健进行回复 取消回复

邮箱地址不会被公开。