Brain damage caused by BigData buzzword

[Exploring mesos] Install Mesos with Ansible and Vagrant

Intro

The aim of this series of posts is to find use-cases for Mesos ecosystem. You’ll find step-by-step guide installing Mesos and frameworks running on top of it. We’ll start from installing Ansible and Vagrant on your local machine. Vagrant would be responsible for VM creation and management. Ansible would provision VMs with all software we need.

Motivation for tool set

  • Vagrant is good, it can on many popular development platforms anywhere (I didn’t meet any show-stopper problems for Ubuntu, MacOS for the last two years)
  • Ansible has strong community and wide support.
  • Ansible has tons of “roles” (installation directives, Ansible reads them and installs/configures software on target machine).
  • You don’t need to setup client software (Puppet/Chef require clients to be installed on target machines)
  • Ansible is dead-simple. There are some integration quirks for Ansible and Vagrant, and they are minor comparing to other problems we would have to solve.
  • There are plenty Ansible playbooks for Mesos and Mesos ecosystem components, so we are going to use them in order to reach our goal: working Mesos cluster with frameworks installed on top of it.

Limitations

  • Host OS is Ubuntu 14.04, so all installation instructions fits only Ubuntu 14.04.
  • Guest OS are Ubuntu 14.04, so you can’t reuse project to install Mesos on CentOS for example.

I’m using Ubuntu 14.04, so installation guide would fit only this linux distro

Required software

virtualbox 5.0 or greater
Vagrant 1.7.4 or greater

Additional plugins for Vagrant

vagrant-hostsupdater 0.0.11

vagrant-cachier 1.2.1

Ansible 1.9.4

Installing Mesos with minimal software requirements

Checkout project from github https://github.com/seregasheypak/mesos-school and switch to branch release/0.1.0-zookeeper-mesos

Install Ansible roles

That command would read requirements.yml and download roles. java8 roles would be downloaded from Ansible Galaxy, an official repository for Ansible roles. Zookeeper and mesos roles would be downloaded directly from github

java8-oracle role installed from Ansible Galaxy, two other roles would be installed from github. Ansible is pretty flexible at this point. absible-zookeeper role used by Mesos. Mesos uses zookeeper quorum for leader election process. Mesos promises failover and HA. It means you can have redundant master services on several nodes. One of master services is a leader, the others are in stand by mode. When leader master goes down for some reason, election process takes place and one of stand by Mesos masters becomes active.

Ansible-mesos role installs master and slave daemons. Master daemons are managing resources and only one Master daemon has active state at the moment. Slave nodes are responsible for scheduling tasks. Task is a unit of work that some framework wants to execute. Framework asks Mesos master for resources and it gives resource offer in response.

Deploy mesos

Just run from the project root

vargrant up

and wait for a while.

Testing installation

The first thing you should be able to access Mesos master UI: http://192.168.56.10:5050

You could see the message:

 

mesos_master_stand_by

Just wait for a few seconds and you would be redirected to leader Mesos UI. Check that all three slaves are registered.

mesos_master_ui

Let’s check that zookeeper is correctly installed.

ssh to one of mesos nodes (you should be in project root)

Let’s start zookeeper client and check that our three node zookeeper cluster configured correctly

sudo /usr/share/zookeeper/bin/zkCli.sh

We connect to locally running zookeeper instance by default. Let’s create zookeeper node

create /testZk "mesosnode01"

Check that we can see it:

And get node value:

Now it’s time to open one more terminal window and check that other zookeeper sees the same data. If yes, means zookeeper configured correctly and all zookeeper instances see each other and share the same data

Great, zookeeper installed correctly. Let’s setup Marathon (to be continued) for running applications on top of Mesos.

Leave a Reply

Your email address will not be published. Required fields are marked *