List of makeQualified() Examples

Examples of makeQualified()

org.apache.hadoop.fs.FileContext.makeQualified()
Make the path fully qualified if it is isn't. A Fully-qualified path has scheme and authority specified and an absolute path. Use the default file system and working dir in this FileContext to qualify. @param path @return qualified path
org.apache.hadoop.fs.FileStatus.makeQualified()
Make the path fully qualified @param fs the file system that the path belongs to
org.apache.hadoop.fs.FileSystem.makeQualified()
Make sure that a path specifies a FileSystem.
org.apache.hadoop.fs.LocalFileSystem.makeQualified()
org.apache.hadoop.fs.LocatedBlockFileStatus.makeQualified()
org.apache.hadoop.fs.LocatedFileStatus.makeQualified()
org.apache.hadoop.fs.Path.makeQualified()
Returns a qualified path object. Deprecated - use {@link #makeQualified(URI,Path)}
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.makeQualified()

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

    
    Properties config = new Properties();
    config.setProperty("mapred.job.tracker.persist.jobstatus.active", "true");
    config.setProperty("mapred.job.tracker.persist.jobstatus.hours", "1");
    config.setProperty("mapred.job.tracker.persist.jobstatus.dir", 
                       fs.makeQualified(TEST_DIR).toString());
    startCluster(true, config);
    JobID jobId = runJob();
    JobClient jc = new JobClient(createJobConf());
    RunningJob rj = jc.getJob(jobId);
    assertNotNull(rj);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

    {
      FileSystem localfs = FileSystem.getLocal(jobConf);
      String TEST_ROOT_DIR =
        new File(System.getProperty("test.build.data","/tmp"))
        .toString().replace(' ', '+');
      Path localIn = localfs.makeQualified
                        (new Path(TEST_ROOT_DIR + "/local/in"));
      Path localOut = localfs.makeQualified
                        (new Path(TEST_ROOT_DIR + "/local/out"));
      result = launchWordCount(jobConf, localIn, localOut,
                               "all your base belong to us", 1, 1);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

      String TEST_ROOT_DIR =
        new File(System.getProperty("test.build.data","/tmp"))
        .toString().replace(' ', '+');
      Path localIn = localfs.makeQualified
                        (new Path(TEST_ROOT_DIR + "/local/in"));
      Path localOut = localfs.makeQualified
                        (new Path(TEST_ROOT_DIR + "/local/out"));
      result = launchWordCount(jobConf, localIn, localOut,
                               "all your base belong to us", 1, 1);
      assertEquals("all\t1\nbase\t1\nbelong\t1\nto\t1\nus\t1\nyour\t1\n", 
                   result.output);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

      for (Path p: srcPaths) {
        FileSystem fs = p.getFileSystem(getConf());
        FileStatus[] statuses = fs.globStatus(p);
        if (statuses != null) {
          for (FileStatus status: statuses) {
            globPaths.add(fs.makeQualified(status.getPath()));
          }
        }
      }
      archive(parentPath, globPaths, archiveName, destPath);
    } catch(IOException ie) {

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

      // get all filtered file names in result list
      FileSystem fs = clusterPath.getFileSystem(conf);
      FileStatus[] matches = fs.listStatus(FileUtil.stat2Paths(fs.globStatus(clusterPath, FILTER)),FILTER);
  
      for (FileStatus match : matches) {
        result.add(fs.makeQualified(match.getPath()));
      }
  
      // iterate through the result path list
      for (Path path : result) {
        SequenceFile.Reader reader= new SequenceFile.Reader(fs, path, conf);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

      throw new InvalidJobConfException("Output directory not set in JobConf.");
    }
    if (outDir != null) {
      FileSystem fs = outDir.getFileSystem(job);
      // normalize the output directory
      outDir = fs.makeQualified(outDir);
      setOutputPath(job, outDir);
      
      // get delegation token for the outDir's file system
      TokenCache.obtainTokensForNamenodes(job.getCredentials(), 
                                          new Path[] {outDir}, job);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

  private String getStagingAreaDirInternal(String user) throws IOException {
    final Path stagingRootDir =
      new Path(conf.get("mapreduce.jobtracker.staging.root.dir",
            "/tmp/hadoop/mapred/staging"));
    final FileSystem fs = stagingRootDir.getFileSystem(conf);
    return fs.makeQualified(new Path(stagingRootDir,
                              user+"/.staging")).toString();
  }


  /**
   * Adds a job to the jobtracker. Make sure that the checks are inplace before

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

        .getDelegationTokenSecretManager().startThreads();
    FileSystem fs = dfsCluster.getFileSystem();
    
    p1 = new Path("file1");
    p2 = new Path("file2");
    p1 = fs.makeQualified(p1);
  }


  @AfterClass
  public static void tearDown() throws Exception {
    if(mrCluster != null)

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

    localFs.create(tmpPath);
    new GenericOptionsParser(conf, args);
    String fileName = conf.get("mapreduce.job.credentials.json");
    assertNotNull("files is null", fileName);
    assertEquals("files option does not match",
      localFs.makeQualified(tmpPath).toString(), fileName);
    
    localFs.delete(new Path(testDir.getAbsolutePath()), true);
  }
}

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

          throw new FileNotFoundException("File "+fileName+" does not exist.");
        }


        LOG.debug("setting conf tokensFile: " + fileName);
        conf.set("mapreduce.job.credentials.json", 
                 localFs.makeQualified(p).toString());
      } catch (IOException e) {
        throw new RuntimeException(e);
      }
    }
  }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.