Examples of org.apache.hadoop.hdfs.server.namenode.AvatarNode$InvalidatesCleaner

org.apache.hadoop.hdfs.server.namenode.AvatarNode

This is an implementation of the AvatarNode, a hot standby for the NameNode. This is really cool, believe me! The AvatarNode has two avatars.. the Standby avatar and the Active avatar. In the Standby avatar, the AvatarNode is consuming transaction logs generated by the primary (via a transaction log stored in a shared device). Typically, the primary Namenode is writing transactions to a NFS filesystem and the Standby is reading the log from the same NFS filesystem. The Standby is also making periodic checkpoints to the primary namenode. A manual command can switch the AvatarNode from the Standby avatar to the Active avatar. In the Active avatar, the AvatarNode performs precisely the same functionality as a real usual Namenode. The switching from Standby avatar to the Active avatar is fast and can typically occur within seconds. Typically, an adminstrator will run require two shared mount points for transaction logs. It has to be set in fs.name.dir.shared0 and fs.name.dir.shared1 (similarly for edits). Then the adminstrator starts the AvatarNode on two different machines as follows: bin/hadoop org.apache.hadoop.hdfs.server.namenode.AvatarNode -zero -active bin/hadoop org.apache.hadoop.hdfs.server.namenode.AvatarNode -one -standby The first AvatarNode uses fs.name.dir.shared0 while the second AvatarNode uses fs.name.dir.shared1 to write its transaction logs. Also, at startup, the first instance is the primary Namenode and the second instance is the Standby After a while, the adminstrator decides to change the avatar of the second instance to Active. In this case, he/she has to first ensure that the first instance is really really dead. This code does not handle the split-brain scenario where there are two active namenodes in one cluster.

    setUp(3600, "testFailCheckpointOnCorruptImage", true, true);   
    // image will be corrupted for second - manual checkpoint
    h.corruptImage = true;
    
    createEdits(20);
    AvatarNode primary = cluster.getPrimaryAvatar(0).avatar;
    AvatarNode standby = cluster.getStandbyAvatar(0).avatar;
   
    // do second checkpoint
    try {
      h.doCheckpoint();
      fail("Should get IOException here");

    LOG.info("TEST: ----> testCheckpointReprocessEdits");
    TestAvatarCheckpointingHandler h = new TestAvatarCheckpointingHandler(null,
        null, false);   
    setUp("testCheckpointReprocessEdits", true);
    createEdits(20);
    AvatarNode primary = cluster.getPrimaryAvatar(0).avatar;
    AvatarNode standby = cluster.getStandbyAvatar(0).avatar;


    h.reprocessIngest = true;
    // set the handler later no to interfere with the previous checkpoint
    InjectionHandler.set(h);
    // checkpoint should be ok

Examples of org.apache.hadoop.hdfs.server.namenode.AvatarNode$InvalidatesCleaner

Related Classes of org.apache.hadoop.hdfs.server.namenode.AvatarNode$InvalidatesCleaner