List of makeQualified() Examples

Examples of makeQualified()

org.apache.hadoop.fs.FileContext.makeQualified()
Make the path fully qualified if it is isn't. A Fully-qualified path has scheme and authority specified and an absolute path. Use the default file system and working dir in this FileContext to qualify. @param path @return qualified path
org.apache.hadoop.fs.FileStatus.makeQualified()
Make the path fully qualified @param fs the file system that the path belongs to
org.apache.hadoop.fs.FileSystem.makeQualified()
Make sure that a path specifies a FileSystem.
org.apache.hadoop.fs.LocalFileSystem.makeQualified()
org.apache.hadoop.fs.LocatedBlockFileStatus.makeQualified()
org.apache.hadoop.fs.LocatedFileStatus.makeQualified()
org.apache.hadoop.fs.Path.makeQualified()
Returns a qualified path object. Deprecated - use {@link #makeQualified(URI,Path)}
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.makeQualified()

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

    if (fs.exists(submitJobDir)) {
      throw new IOException("Not submitting job. Job directory " + submitJobDir
          +" already exists!! This is unexpected.Please check what's there in" +
          " that directory");
    }
    submitJobDir = fs.makeQualified(submitJobDir);
    submitJobDir = new Path(submitJobDir.toUri().getPath());
    FsPermission mapredSysPerms = new FsPermission(JobSubmissionFiles.JOB_DIR_PERMISSION);
    FileSystem.mkdirs(fs, submitJobDir, mapredSysPerms);
    Path filesDir = JobSubmissionFiles.getJobDistCacheFiles(submitJobDir);
    Path archivesDir = JobSubmissionFiles.getJobDistCacheArchives(submitJobDir);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

                FileSystem fileSystem = FileSystem.get(configuration);
                return fileSystem.getWorkingDirectory();
            }
            URI uri = URI.create(result);
            FileSystem fileSystem = FileSystem.get(uri, configuration);
            return fileSystem.makeQualified(new Path(uri));
        } catch (IOException e) {
            WGLOG.error(e, "E00002",
                    profile.getName(),
                    KEY_BASE_PATH,
                    result == null ? "(default)" : result);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

        VariableTable variables = createVariables(context);
        Configuration config = configurations.newInstance();
        FileSystem fs = FileSystem.get(config);
        for (String path : description.getPaths()) {
            String resolved = variables.parse(path, false);
            Path target = fs.makeQualified(new Path(resolved));
            LOG.debug("Deleting file: {}", target);
            boolean succeed = fs.delete(target, true);
            LOG.debug("Deleted file (succeed={}): {}", succeed, target);
        }
        return;

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

        Path path = new Path(resolved);
        Path output = path.getParent();
        Path target;
        if (output == null) {
            LOG.warn("Skipped deleting output directory because it is a base directory: {}", path);
            target = fs.makeQualified(path);
        } else {
            LOG.warn("Output directory will be deleted: {}", output);
            target = fs.makeQualified(output);
        }
        LOG.debug("Deleting output target: {}", target);

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

        if (output == null) {
            LOG.warn("Skipped deleting output directory because it is a base directory: {}", path);
            target = fs.makeQualified(path);
        } else {
            LOG.warn("Output directory will be deleted: {}", output);
            target = fs.makeQualified(output);
        }
        LOG.debug("Deleting output target: {}", target);
        boolean succeed = fs.delete(target, true);
        LOG.debug("Deleted output target (succeed={}): {}", succeed, target);
    }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

     */
    public void cleanWorkingDirectory() throws IOException {
        Configuration conf = configurations.newInstance();
        FileSystem fs = FileSystem.get(conf);
        Path path = new Path(context.getClusterWorkDir());
        Path fullPath = fs.makeQualified(path);
        LOG.debug("クラスタワークディレクトリを初期化します。Path: {}", fullPath);
        boolean deleted = fs.delete(fullPath, true);
        if (deleted) {
            LOG.debug("クラスタワークディレクトリを削除しました。Path: {}", fullPath);
        } else {

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

                LOG.debug("local path matched: " + path);
            }
            return localFileSystem.makeQualified(path);
        }
        FileSystem remoteFileSystem = remotePath.getFileSystem(configuration);
        remotePath = remoteFileSystem.makeQualified(remotePath);
        if (LOG.isDebugEnabled()) {
            LOG.debug("distributed cache is not localized explicitly: " + remotePath);
        }
        if (isLocal(remoteFileSystem) == false) {
            LOG.warn(MessageFormat.format(

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

            if (path.toUri().getScheme() == null) {
                fs = local;
            } else {
                fs = path.getFileSystem(configuration);
            }
            path = fs.makeQualified(path);
            if (fs.exists(path) == false) {
                throw new FileNotFoundException(path.toString());
            }
            results.add(path);
        }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

        for (Path path : paths) {
            if (buf.length() != 0) {
                buf.append(',');
            }
            FileSystem fs = path.getFileSystem(configuration);
            buf.append(fs.makeQualified(path).toString());
        }
        String libjars = buf.toString();
        return libjars;
    }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.makeQualified()

            if (value) {
                if (LOG.isInfoEnabled()) {
                    LOG.info(MessageFormat.format(
                            "Creating transaction info: job={0}, path={1}",
                            jobContext.getJobID(),
                            fs.makeQualified(transactionInfo)));
                }
                FSDataOutputStream output = fs.create(transactionInfo, false);
                boolean closed = false;
                try {
                    PrintWriter writer = new PrintWriter(

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.