Setting up a multiple virtual nodes cluster and running hadoop on top of it using vagrant and puppet

instructionHello, everyone! Happy Apr 4! Finally we got out of the terrible winter right? lol.  Time for some bit advanced geeky stuffs: this tutorial is a continuation of the previous post.  The goal of this tutorial is to show you how to set up a hadoop cdh5 Cloudera version running on few virtual boxes on your machine (In this tutorial, i created 1 master node and 2 slave nodes).  I have written a simple vagrant/puppet script and uploaded it to my Github repo (i.e. My vagrant-hadoop-cluster repo) account so you guys can download the code and please follow the instruction carefully to install and get it running.  Assuming you have followed my instruction, u should be able to get the cluster running like this:

complete-setup

And If you do see something like this, congrats! You have just successfully built a hadoop multi-node cluster on top of your machine.

Notice that, unlike other tech tutorials, I really hate to explain every syntax of one language in my tutorial since nowadays it is so easy to find lot of more complete references already online so there is no point to doubly explain (assuming all of the readers here are technically-able enough to figure out themselves the syntax) here again.   Therefore, my style is to provide the complete out of box working code so you guys figure out yourself the syntaxes and styles of Puppet.  If you guys are more curious, here is the puppet reference that i saved to my bookmark lists whenever i got stuck in the syntax issues.

https://github.com/wwken/Vagrant-Puppet/tree/master/vagrant-hadoop-cluster

Advertisements

One thought on “Setting up a multiple virtual nodes cluster and running hadoop on top of it using vagrant and puppet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s