Hadoop Failover testing - part 3

http://solutionsatexperts.com/test-the-failover-chapter-3/

Cross checks the status of master and slave namenode  using  below hdfs command.

[hduser@mn1~]$hdfs haadmin -getServiceState mn1

[hduser@mn1~]$hdfs haadmin -getServiceState mn2

In our installation guide we are going to show you Failover test using two methods

1. Killing namenode service in master node
2. Manual failover

Let us see the Methods in Details :

1. Killing namenode service in master node

[hduser@mn1~]$kill -9 pid

Check the logs or use dfs haadmin –getServiceState to confirm the automatic failover

2. Manual failover
[hduser@mn1~]$hdfs haadmin -failover mn1 mn2
Start / Stop specific service from group of services in hadoop cluster using below command

example:

[hduser@mn1~]$hadoop-daemon.sh start zkfc
[hduser@mn1~]$hadoop-daemon.sh start namenode

Web based console to check the cluster status
http://mn1:50070 or http://IP:50070
http://mn2:50070 or http://IP:50070


Quorum Journal Manager or  Shared Storage(NFS share)
Method : Quorum Journal Manager

In Hadoop 1. X  there was no automatic failover and no high availability for name node service, namenode service is responsible to maintain metadata.  In Hadoop 2. x elimination of single point of failure , whereas in Hadoop 1.x the basic draw back was a single point of failure for namenode.
In our installation guide for Hadoop 2.3 with high availability which covers all steps required to install and configure Hadoop 2nd generation with high availability.
The method  we are going  to be showing using QuorumJournalManagar.

In our guide we are going configure 3 node cluster, the details are as follows.

server1        :mn1  (Master Node : namenode  , datanode ,
resource manager, DFSZKFailoverController,JournalNode ,
ResourceManager,NodeManager & QuorumPeerMain)

server2       :mn2 (Slave Node: namenode  , datanode ,
DFSZKFailoverController,JournalNode ,NodeManager & QuorumPeerMain)

server3       :dn1(datanode, QuorumPeerMain, & JournalNode)

Software packages and dependencies
hadoop-2.3.0 , zookeeper-3.4.5 &  jdk-7u51-linux-i586.rpm

User account in all the cluster nodes
hduser

Network and Hostname details
mn1    : 192.168.1.39
mn2    : 192.168.1.57
dn1    : 192.168.1.72

Operating system and arch.
OS: Centos 6.5
Arch: 32

Thank You.