List of globStatus() Examples

Examples of globStatus()

org.apache.hadoop.fs.FileSystem.globStatus()

Return all the files that match filePattern and are not checksum files. Results are sorted by their names.
A filename pattern is composed of regular characters and special pattern matching characters, which are:

?
Matches any single character.

*
Matches zero or more characters.

[abc]
Matches a single character from character set {a,b,c}.

[a-b]
Matches a single character from the character range {a...b}. Note that character a must be lexicographically less than or equal to character b.

[^a]
Matches a single character that is not from character set or range {a}. Note that the ^ character must occur immediately to the right of the opening bracket.

\c
Removes (escapes) any special meaning of character c.

{ab,cd}
Matches a string from the string set {ab, cd}

{ab,c{de,fh}}
Matches a string from the string set {ab, cde, cfh}

@param pathPattern a regular expression specifying a pth pattern @return an array of paths that match the path pattern @throws IOException

Examples of org.apache.hadoop.fs.FileSystem.globStatus()


    try {
      for (String uri : uris) {
        Path srcPath = new Path(uri);
        FileSystem srcFs = getFS(srcPath);
        Path[] srcs = FileUtil.stat2Paths(srcFs.globStatus(srcPath), srcPath);
        for (Path src : srcs) {
          setrep(replication, recursive, srcFs, src, waitList);
        }
      }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()


      try {
        Path srcPat = new Path(uri);
        FileSystem srcFs = getFS(srcPat);


        for (Path src : FileUtil.stat2Paths(srcFs.globStatus(srcPat), srcPat)) {
          Assert.isTrue(srcFs.isFile(src), "Source must be a file");
          i = srcFs.open(src);
          switch (i.readShort()) {
          case 0x1f8b: // RFC 1952
            i.seek(0);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

    PathFilter inputFilter = new MultiPathFilter(filters);
    
    for (int i=0; i < dirs.length; ++i) {
      Path p = dirs[i];
      FileSystem fs = p.getFileSystem(job.getConfiguration()); 
      FileStatus[] matches = fs.globStatus(p, inputFilter);
      if (matches == null) {
        errors.add(new IOException("Input path does not exist: " + p));
      } else if (matches.length == 0) {
        errors.add(new IOException("Input Pattern " + p + " matches 0 files"));
      } else {

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

    }
    PathFilter inputFilter = new MultiPathFilter(filters);


    for (Path p: dirs) {
      FileSystem fs = p.getFileSystem(job); 
      FileStatus[] matches = fs.globStatus(p, inputFilter);
      if (matches == null) {
        errors.add(new IOException("Input path does not exist: " + p));
      } else if (matches.length == 0) {
        errors.add(new IOException("Input Pattern " + p + " matches 0 files"));
      } else {

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

      }
      // do a glob on the srcPaths and then pass it on
      List<Path> globPaths = new ArrayList<Path>();
      for (Path p: srcPaths) {
        FileSystem fs = p.getFileSystem(getConf());
        FileStatus[] statuses = fs.globStatus(p);
        if (statuses != null) {
          for (FileStatus status: statuses) {
            globPaths.add(fs.makeQualified(status.getPath()));
          }
        }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

      Path clusterPath = new Path(clusterPathStr, "*");
      Collection<Path> result = new ArrayList<Path>();
  
      // get all filtered file names in result list
      FileSystem fs = clusterPath.getFileSystem(conf);
      FileStatus[] matches = fs.listStatus(FileUtil.stat2Paths(fs.globStatus(clusterPath, FILTER)),FILTER);
  
      for (FileStatus match : matches) {
        result.add(fs.makeQualified(match.getPath()));
      }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

      inputPathPattern = input;
    }
    
    System.out.println("######"+k);


    FileStatus[] inputFiles = fs.globStatus(inputPathPattern,
        KmeansPublic.FILTER);
    SequenceFile.Writer writer = SequenceFile.createWriter(fs, conf,
        new Path(outFile,"random"), Text.class, Cluster.class);
    List<Cluster> chosenClusters = new ArrayList<Cluster>(k);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

        FileSystem fs = FileSystem.get(conf);
        for (Path path : paths) {
            WGLOG.info("I22003",
                    fs.getUri(),
                    path);
            FileStatus[] results = fs.globStatus(path);
            if (results == null) {
                continue;
            }
            for (FileStatus status : results) {
                doDelete(fs, status, drain);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

        return new Path("target/testing/directio-fs", target);
    }


    private List<Path> find(String target) throws IOException {
        FileSystem fs = FileSystem.get(tester.configuration());
        FileStatus[] list = fs.globStatus(getPath(target));
        if (list == null) {
            return Collections.emptyList();
        }
        List<Path> results = Lists.create();
        for (FileStatus file : list) {

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

        targetDirectory.mkdirs();
        logger.info("copy {} to {}", location, targetDirectory);


        Path path = new Path(location.toPath('/'));
        FileSystem fs = path.getFileSystem(configuration);
        FileStatus[] list = fs.globStatus(path);
        if (list == null) {
            throw new IOException(MessageFormat.format(
                    "Failed to fs -get: source={0}, destination={1}",
                    path,
                    targetDirectory));

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.