Examples of HadoopArchives

org.apache.hadoop.tools.HadoopArchives
a archive creation utility. This class provides methods that can be used to create hadoop archives. For understanding of Hadoop archives look at {@link HarFileSystem}.

Examples of org.apache.hadoop.tools.HadoopArchives


  @Override
  public int createHadoopArchive(Configuration conf, Path sourceDir, Path destDir,
      String archiveName) throws Exception {


    HadoopArchives har = new HadoopArchives(conf);
    List<String> args = new ArrayList<String>();


    args.add("-archiveName");
    args.add(archiveName);
    args.add("-p");

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives


  @Override
  public int createHadoopArchive(Configuration conf, Path sourceDir, Path destDir,
      String archiveName) throws Exception {


    HadoopArchives har = new HadoopArchives(conf);
    List<String> args = new ArrayList<String>();


    args.add("-archiveName");
    args.add(archiveName);
    args.add(sourceDir.toString());

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives


  @Override
  public int createHadoopArchive(Configuration conf, Path sourceDir, Path destDir,
      String archiveName) throws Exception {


    HadoopArchives har = new HadoopArchives(conf);
    List<String> args = new ArrayList<String>();


    args.add("-archiveName");
    args.add(archiveName);
    args.add(sourceDir.toString());

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives


  @Override
  public int createHadoopArchive(Configuration conf, Path sourceDir, Path destDir,
      String archiveName) throws Exception {


    HadoopArchives har = new HadoopArchives(conf);
    List<String> args = new ArrayList<String>();


    args.add("-archiveName");
    args.add(archiveName);
    args.add("-p");

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives

            }
//      for (FileStatus ds : fs.globStatus(new Path(dir, "*"))){
//        LOG.info("src : "+ds.getPath().toUri().toString());
//      }


            final HadoopArchives har = new HadoopArchives(newConf);
            int rc = ToolRunner.run(har, args);
            if (rc != 0) {
                throw new Exception("Har returned error code " + rc);
            }

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives


  @Override
  public int createHadoopArchive(Configuration conf, Path sourceDir, Path destDir,
      String archiveName) throws Exception {


    HadoopArchives har = new HadoopArchives(conf);
    List<String> args = new ArrayList<String>();


    if (conf.get("hive.archive.har.parentdir.settable") == null) {
      throw new RuntimeException("hive.archive.har.parentdir.settable is not set");
    }

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives


  @Override
  public int createHadoopArchive(Configuration conf, Path sourceDir, Path destDir,
      String archiveName) throws Exception {


    HadoopArchives har = new HadoopArchives(conf);
    List<String> args = new ArrayList<String>();


    if (conf.get("hive.archive.har.parentdir.settable") == null) {
      throw new RuntimeException("hive.archive.har.parentdir.settable is not set");
    }

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives


  @Override
  public int createHadoopArchive(Configuration conf, Path sourceDir, Path destDir,
      String archiveName) throws Exception {


    HadoopArchives har = new HadoopArchives(conf);
    List<String> args = new ArrayList<String>();


    if (conf.get("hive.archive.har.parentdir.settable") == null) {
      throw new RuntimeException("hive.archive.har.parentdir.settable is not set");
    }

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives

                                rand.nextLong() + "-" + HAR_SUFFIX;
    short metaReplication =
      (short) Integer.parseInt(info.getProperty("metaReplication"));
    // HadoopArchives.HAR_PARTFILE_LABEL is private, so hard-coding the label.
    conf.setLong("har.partfile.size", configMgr.getHarPartfileSize());
    HadoopArchives har = new HadoopArchives(conf);
    String[] args = new String[7];
    args[0] = "-Ddfs.replication=" + metaReplication;
    args[1] = "-archiveName";
    args[2] = harFileSrc;
    args[3] = "-p";

View Full Code Here

Examples of org.apache.hadoop.tools.HadoopArchives

    out.close();
    out = fs.create(filec);
    out.write("c".getBytes());
    out.close();
    Configuration conf = mapred.createJobConf();
    HadoopArchives har = new HadoopArchives(conf);
    String[] args = new String[3];
    //check for destination not specfied
    args[0] = "-archiveName";
    args[1] = "foo.har";
    args[2] = inputPath.toString();

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.