List of hflush() Examples

Examples of hflush()

org.apache.hadoop.fs.FSDataOutputStream.hflush()
org.apache.hadoop.hdfs.DFSOutputStream.hflush()
Flushes out to all replicas of the block. The data is in the buffers of the DNs but not necessarily in the DN's OS buffers. It is a synchronous operation. When it returns, it guarantees that flushed data become visible to new readers. It is not guaranteed that data has been flushed to persistent store on the datanode. Block allocations are persisted on namenode.
org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hflush()
org.apache.hadoop.io.SequenceFile.Writer.hflush()

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

      // create some rbw replicas on disk
      byte[] writeBuf = new byte[fileLen];
      new Random().nextBytes(writeBuf);
      out = fs.create(src);
      out.write(writeBuf);
      out.hflush();
      DataNode dn = cluster.getDataNodes().get(0);
      for (FsVolumeSpi v : dataset(dn).getVolumes()) {
        final FsVolumeImpl volume = (FsVolumeImpl)v;
        File currentDir = volume.getCurrentDir().getParentFile().getParentFile();
        File rbwDir = new File(currentDir, "rbw");

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

    NameNode nn = cluster.getNameNode();
    
    FSDataOutputStream out = fs.create(filePath, REPL_FACTOR);
    try {
      AppendTestUtil.write(out, 0, 10);
      out.hflush();


      // Set up a spy so that we can delay the block report coming
      // from this node.
      DataNode dn = cluster.getDataNodes().get(0);
      DatanodeProtocol spy =

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

      cluster = new MiniDFSCluster.Builder(conf).numDataNodes(numDatanodes)
          .format(true).build();
      FileSystem fs = cluster.getFileSystem();
      fos = fs.create(new Path("tmpfile"));
      fos.write(new byte[] { 0, 1, 2, 3 });
      fos.hflush();
      assertEquals(1, cluster.getNamesystem().getLeaseManager().countLease());


      secondary = startSecondaryNameNode(conf);
      assertEquals(0, secondary.getFSNamesystem().getLeaseManager().countLease());

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

    try {
      for (int i = 0; i < 5; i++) {
        FSDataOutputStream stm = fs.create(new Path("/test-uc-" + i));
        stms.add(stm);
        stm.write(1);
        stm.hflush();
      }
      // Roll edit log so that, when the SBN restarts, it will load
      // the namespace during startup and enter safemode.
      nn0.getRpcServer().rollEditLog();
    } finally {

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

            @Override
            public void run() {
              try {
                while (true) {
                  try {
                    stm.hflush();
                  } catch (IOException ioe) {
                    if (!ioe.toString().contains("DFSOutputStream is closed")) {
                      throw ioe;
                    } else {
                      return;

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

    FSDataOutputStream stm = createFile(fs, p, 1);
    System.out.println("Created file simpleFlush.dat");


    // There have been a couple issues with flushing empty buffers, so do
    // some empty flushes first.
    stm.hflush();
    stm.hflush();
    stm.write(1);
    stm.hflush();
    stm.hflush();

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

    System.out.println("Created file simpleFlush.dat");


    // There have been a couple issues with flushing empty buffers, so do
    // some empty flushes first.
    stm.hflush();
    stm.hflush();
    stm.write(1);
    stm.hflush();
    stm.hflush();


    CountDownLatch countdown = new CountDownLatch(1);

View Full Code Here

Examples of org.apache.hadoop.fs.FSDataOutputStream.hflush()

    // There have been a couple issues with flushing empty buffers, so do
    // some empty flushes first.
    stm.hflush();
    stm.hflush();
    stm.write(1);
    stm.hflush();
    stm.hflush();


    CountDownLatch countdown = new CountDownLatch(1);
    ArrayList<Thread> threads = new ArrayList<Thread>();
    AtomicReference<Throwable> thrown = new AtomicReference<Throwable>();

View Full Code Here

Examples of org.apache.hadoop.hdfs.DFSOutputStream.hflush()

      try {
        // Create a file and make sure a block is allocated for it.
        out = (DFSOutputStream)(fs.create(file).
            getWrappedStream()); 
        out.write(1);
        out.hflush();
        
        // Create a snapshot that includes the file.
        SnapshotTestHelper.createSnapshot((DistributedFileSystem) fs,
            new Path("/"), "s1");

View Full Code Here

Examples of org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hflush()

      assertEquals(file1 + " should be replicated to 1 datanode.", 1,
          stm.getCurrentBlockReplication());


      // write two full blocks.
      writeFile(stm, numBlocks * blockSize);
      stm.hflush();
      assertEquals(file1 + " should still be replicated to 1 datanode.", 1,
          stm.getCurrentBlockReplication());


      // rename file wile keeping it open.
      Path fileRenamed = new Path("/filestatusRenamed.dat");

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.