Hadoop in Docker

Hadoop in Docker

Protocol for Installation on OpenSuse 15.1

Docker Intro

First create btrfs fs on /var/lib/docker to get decent snapshot support, then

sudo zypper install docker python3-docker-compose
sudo usermod -G docker -a $USER
sudo systemctl enable docker
sudo systemctl start docker
sudo systemctl status docker

You need to relogin or do ssh localhost to activate the group change. Then you can start shell in an opensuse image

docker run --name bigtop -it --entrypoint /bin/bash opensuse/leap:42.3

If you exit and want to restart this container restart and attach

docker container restart bigtop
docker attach bigtop

If you want to start another shell in the running container

docker container exec -it bigtop /bin/bash

If you dont know the name of the container, use this to list them all

docker ps -a

If you want to add a volume to your existing container, that is not as easy as I thought

docker commit bigtop bigtop-image
docker run --name bigtop2 -v /mountpoint -ti --entrypoint /bin/bash bigtop-image

Once you know all is ok, you can replace the old container

docker rm bigtop
docker rename bigtop2 bigtop

Now you have a docker container where you can start bigtop setup in and know how to reconfigure it if something is misssing

Bigtop

Bigtop from Repo

Add repository to opensuse 42.3 (suggest to do that in docker image created above)

zypper in java-1_8_0-openjdk-1.8.0
zypper ar https://artfiles.org/apache.org/bigtop/bigtop-1.4.0/repos/opensuse42.3/bigtop.repo
zypper in bigtop-utils bigtop-jsvc bigtop-tomcat bigtop-groovy

Configure tomcat users

cp -av  /usr/lib/bigtop-tomcat/conf.template /usr/lib/bigtop-tomcat/conf
vi /usr/lib/bigtop-tomcat/conf/tomcat-users.xml

I added this just before the final "</tomcat-users>" line. No idea if this is really ok or necessary

 <role rolename="tomcat"/>
 <role rolename="manager"/>
 <role rolename="manager-gui"/>
 <user username="tomcat" password="my-PW" roles="tomcat,manager,manager-gui"/>
 <user username="joachim" password="my-PW" roles="tomcat,manager,manager-gui"/>
 <user username="julian" password="his-PW" roles="tomcat,manager,manager-gui"/>

Open port 8080 (and maybe 8009 and 8005?) for the container and start /usr/lib/bigtop-tomcat/bin/catalina.sh run

docker run --name bigtop-catalina -p 8080:8080 -p 8005:8005 -p8009:8009 -ti --entrypoint /bin/bash bigtop-image /usr/lib/bigtop-tomcat/bin/catalina.sh run

Check if tomcat is running fine by deploying a "hello world" war

zypper install wget
cd /usr/lib/bigtop-tomcat
mkdir -p webapps
cd webapps
wget https://tomcat.apache.org/tomcat-8.0-doc/appdev/sample/sample.war

Tomcat should find the war file, expand and deploy it automatically. Find it at http://localhost:8080/sample

Groovy that comes with bigtop uses utility which and needs a JDK. Install them

zypper install which java-1_8_0-openjdk-devel
/usr/lib/bigtop-groovy/bin/groovy -version

Bigtop 1.4.0 from tar

Enable docker to run from within containers (bind mount socket and command)

docker run --name bigtop-catalina -p 8080:8080 -p 8005:8005 -p8009:8009 -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -ti --entrypoint /bin/bash bigtop-catalina-image /usr/lib/bigtop-tomcat/bin/catalina.sh run

zypper in tar curl unzip
sed -i 's/en_US.UTF-8//' /etc/sysconfig/language
mkuser -m bigtop
su - bigtop
wget https://apache.lauf-forum.at/bigtop/bigtop-1.4.0/bigtop-1.4.0-project.tar.gz
tar xzvf bigtop-1.4.0-project.tar.gz
cd bigtop-1.4.0