List of hsync() Examples

Examples of hsync()

org.apache.hadoop.fs.FSDataOutputStream.hsync()
org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync()
Sync buffered data to DataNodes (flush to disk devices). @param syncFlags Indicate the detailed semantic and actions of the hsync. @throws IOException @see FSDataOutputStream#hsync()
org.apache.hadoop.io.SequenceFile.Writer.hsync()

Examples of org.apache.hadoop.fs.FSDataOutputStream.hsync()

    int count = 0;
    while (count < 2 * 1024 * 1024) {
      out.writeBytes("Data");
      count += 4;
    }
    out.hsync();
    // abort the original stream
    ((DFSOutputStream) out.getWrappedStream()).abort();


    LocatedBlocks locations = cluster.getNameNodeRpc().getBlockLocations(
        file.toString(), 0, count);

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hsync()

      Path path, Class<?> eventClass) throws IOException {
    fileSystem.createNewFile(path);
    FSDataOutputStream out = fileSystem.append(path);
    writable.setClassName(eventClass.getName());
    writable.write(out);
    out.hsync();
    FSDataInputStream in = fileSystem.open(path);
    writable.readFields(in);
    return new HeaderBasedInitialization(out, in);
  }

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hsync()

      }
      dagSummaryDataMap.get(dagId).handleSummaryEvent(proto);
      proto.writeDelimitedTo(newSummaryStream);
    }
    summaryStream.close();
    newSummaryStream.hsync();
    newSummaryStream.close();


    // Set counter for next set of DAGs
    dagAppMaster.setDAGCounter(dagCounter);

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hsync()

      }
      newDAGRecoveryStream.writeInt(eventType.ordinal());
      event.toProtoStream(newDAGRecoveryStream);
    }
    dagRecoveryStream.close();
    newDAGRecoveryStream.hsync();
    newDAGRecoveryStream.close();


    if (!recoveredDAGData.isCompleted
        && !recoveredDAGData.nonRecoverable) {
      if (lastInProgressDAGData.bufferedSummaryEvents != null

View Full Code Here

Examples of org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync()

    hdfs.allowSnapshot(dir);
    hdfs.createSnapshot(dir, "s0");
    
    // 2. create snapshot s1 before appending sub1file1 finishes
    HdfsDataOutputStream out = appendFileWithoutClosing(sub1file1, BLOCKSIZE);
    out.hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH));
    // also append sub1file2
    DFSTestUtil.appendFile(hdfs, sub1file2, BLOCKSIZE);
    hdfs.createSnapshot(dir, "s1");
    out.close();

View Full Code Here

Examples of org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync()

    hdfs.createSnapshot(dir, "s1");
    out.close();
    
    // 3. create snapshot s2 before appending finishes
    out = appendFileWithoutClosing(sub1file1, BLOCKSIZE);
    out.hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH));
    hdfs.createSnapshot(dir, "s2");
    out.close();
    
    // 4. save fsimage before appending finishes
    out = appendFileWithoutClosing(sub1file1, BLOCKSIZE);

View Full Code Here

Examples of org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync()

    hdfs.createSnapshot(dir, "s2");
    out.close();
    
    // 4. save fsimage before appending finishes
    out = appendFileWithoutClosing(sub1file1, BLOCKSIZE);
    out.hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH));
    // dump fsdir
    File fsnBefore = dumpTree2File("before");
    // save the namesystem to a temp file
    File imageFile = saveFSImageToTempFile();

View Full Code Here

Examples of org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync()

    
    hdfs.allowSnapshot(dir);
    hdfs.createSnapshot(dir, "s0");
    
    HdfsDataOutputStream out = appendFileWithoutClosing(sub1file1, BLOCKSIZE);
    out.hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH));      
    
    // save namespace and restart cluster
    hdfs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
    hdfs.saveNamespace();
    hdfs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);

View Full Code Here

Examples of org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync()

    Path file = new Path(dir, "file");
    DFSTestUtil.createFile(hdfs, file, BLOCKSIZE, REPLICATION, seed);
    
    // 1. append without closing stream --> create snapshot
    HdfsDataOutputStream out = appendFileWithoutClosing(file, BLOCKSIZE);
    out.hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH));
    SnapshotTestHelper.createSnapshot(hdfs, dir, "s0");
    out.close();
    
    // check: an INodeFileUnderConstructionWithSnapshot should be stored into s0's
    // deleted list, with size BLOCKSIZE*2

View Full Code Here

Examples of org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync()

    DirectoryDiff last = dirNode.getDiffs().getLast();
    Snapshot s0 = last.snapshot;
    
    // 2. append without closing stream
    out = appendFileWithoutClosing(file, BLOCKSIZE);
    out.hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH));
    
    // re-check nodeInDeleted_S0
    dirNode = (INodeDirectorySnapshottable) fsdir.getINode(dir.toString());
    assertEquals(BLOCKSIZE * 2, fileNode.computeFileSize(s0));

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.