List of globStatus() Examples

Examples of globStatus()

org.apache.hadoop.fs.FileSystem.globStatus()

Return all the files that match filePattern and are not checksum files. Results are sorted by their names.
A filename pattern is composed of regular characters and special pattern matching characters, which are:

?
Matches any single character.

*
Matches zero or more characters.

[abc]
Matches a single character from character set {a,b,c}.

[a-b]
Matches a single character from the character range {a...b}. Note that character a must be lexicographically less than or equal to character b.

[^a]
Matches a single character that is not from character set or range {a}. Note that the ^ character must occur immediately to the right of the opening bracket.

\c
Removes (escapes) any special meaning of character c.

{ab,cd}
Matches a string from the string set {ab, cd}

{ab,c{de,fh}}
Matches a string from the string set {ab, cde, cfh}

@param pathPattern a regular expression specifying a pth pattern @return an array of paths that match the path pattern @throws IOException

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

                                                       ClassNotFoundException {
    Configuration conf = CreateNewConfiguration();
    
    Map<String,Integer> dictionary = new HashMap<String,Integer>();
    FileSystem fs = FileSystem.get(dictionaryPath.toUri(), conf);
    FileStatus[] outputFiles = fs.globStatus(new Path(dictionaryPath,
        "part-*"));
    int i = 0;
    for (FileStatus fileStatus : outputFiles) {
      Path path = fileStatus.getPath();
      SequenceFile.Reader reader = new SequenceFile.Reader(fs, path,

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

    try {
      if (!ObjectUtils.isEmpty(uris)) {
        for (String uri : uris) {
          Path src = new Path(uri);
          FileSystem srcFS = getFS(src);
          results.addAll(Arrays.asList(FileUtil.stat2Paths(srcFS.globStatus(src), src)));
        }
      }
    } catch (IOException ex) {
      throw new HadoopException("Cannot execute command " + ex.getMessage(), ex);
    }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

      FileSystem srcFs = getFS(srcpath);
      srcFs.setVerifyChecksum(ignorecrc);
      if (crc && !(srcFs instanceof ChecksumFileSystem)) {
        crc = false;
      }
      FileStatus[] srcs = srcFs.globStatus(srcpath);
      boolean dstIsDir = dst.isDirectory();
      if (srcs.length > 1 && !dstIsDir) {
        throw new IllegalArgumentException("When copying multiple files, "
            + "destination should be a directory.");
      }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()


    for (String src : uris) {
      try {
        Path srcPath = new Path(src);
        final FileSystem fs = getFS(srcPath);
        FileStatus[] statuses = fs.globStatus(srcPath);
        Assert.notEmpty(statuses, "Can not find listing for " + src);
        for (FileStatus s : statuses) {
          Path p = s.getPath();
          results.put(p, fs.getContentSummary(p));
        }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

        }
      }


      for (Path path : srcs) {
        FileSystem srcFs = path.getFileSystem(configuration);
        Path[] from = FileUtil.stat2Paths(srcFs.globStatus(path), path);
        if (!ObjectUtils.isEmpty(from) && from.length > 1 && !isDestDir) {
          throw new IllegalArgumentException(
              "When copying multiple files, destination should be a directory.");
        }
        for (Path fromPath : from) {

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()


    try {
      for (String src : strings) {
        Path srcPath = new Path(src);
        FileSystem srcFs = getFS(srcPath);
        FileStatus[] fileStatus = srcFs.globStatus(srcPath);
        if (summary) {
          for (FileStatus status : fileStatus) {
            results.put(status.getPath(), srcFs.getContentSummary(status.getPath()).getLength());
          }
        }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

  public void getmerge(String src, String localdst, boolean addnl) {
    Path srcPath = new Path(src);
    Path dst = new Path(localdst);
    try {
      FileSystem srcFs = getFS(srcPath);
      Path[] srcs = FileUtil.stat2Paths(srcFs.globStatus(srcPath), srcPath);
      for (int i = 0; i < srcs.length; i++) {
        if (addnl) {
          FileUtil.copyMerge(srcFs, srcs[i], FileSystem.getLocal(configuration), dst, false, configuration,
              "\n");
        }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

    try {
      for (String src : match) {
        Path srcPath = new Path(src);


        FileSystem srcFs = getFS(srcPath);
        FileStatus[] srcs = srcFs.globStatus(srcPath);
        if (!ObjectUtils.isEmpty(srcs)) {
          for (FileStatus status : srcs) {
            ls(status, srcFs, recursive, results);
          }
        }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

        URI srcURI = srcFs.getUri();
        URI dstURI = dstFs.getUri();
        if (srcURI.compareTo(dstURI) != 0) {
          throw new IllegalArgumentException("src and destination filesystems do not match.");
        }
        Path[] srcs = FileUtil.stat2Paths(srcFs.globStatus(srcPath), srcPath);
        if (srcs.length > 1 && !isDstDir) {
          throw new IllegalArgumentException("When moving multiple files, destination should be a directory.");
        }
        for (Path s : srcs) {
          if (!srcFs.rename(s, dstPath)) {

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.globStatus()

    for (String uri : uris) {
      try {
        Path src = new Path(uri);
        FileSystem srcFs = getFS(src);


        for (Path p : FileUtil.stat2Paths(srcFs.globStatus(src), src)) {
          FileStatus status = srcFs.getFileStatus(p);
          if (status.isDir() && !recursive) {
            throw new IllegalStateException("Cannot remove directory \"" + src
                + "\", if recursive deletion was not specified");
          }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.