List of initOutPathLocalFS() Examples

Examples of initOutPathLocalFS()

eu.stratosphere.core.fs.FileSystem.initOutPathLocalFS()
Initializes output directories on local file systems according to the given write mode. WriteMode.CREATE & parallel output: - A directory is created if the output path does not exist. - An existing directory is reused, files contained in the directory are NOT deleted. - An existing file raises an exception. WriteMode.CREATE & NONE parallel output: - An existing file or directory raises an exception. WriteMode.OVERWRITE & parallel output: - A directory is created if the output path does not exist. - An existing directory is reused, files contained in the directory are NOT deleted. - An existing file is deleted and replaced by a new directory. WriteMode.OVERWRITE & NONE parallel output: - An existing file or directory (and all its content) is deleted Files contained in an existing directory are not deleted, because multiple instances of a DataSinkTask might call this function at the same time and hence might perform concurrent delete operations on the file system (possibly deleting output files of concurrently running tasks). Since concurrent DataSinkTasks are not aware of each other, coordination of delete and create operations would be difficult. @param outPath Output path that should be prepared. @param writeMode Write mode to consider. @param createDirectory True, to initialize a directory at the given path, false otherwise. @return True, if the path was successfully prepared, false otherwise. @throws IOException
org.apache.flink.core.fs.FileSystem.initOutPathLocalFS()
Initializes output directories on local file systems according to the given write mode. WriteMode.CREATE & parallel output: - A directory is created if the output path does not exist. - An existing directory is reused, files contained in the directory are NOT deleted. - An existing file raises an exception. WriteMode.CREATE & NONE parallel output: - An existing file or directory raises an exception. WriteMode.OVERWRITE & parallel output: - A directory is created if the output path does not exist. - An existing directory is reused, files contained in the directory are NOT deleted. - An existing file is deleted and replaced by a new directory. WriteMode.OVERWRITE & NONE parallel output: - An existing file or directory (and all its content) is deleted Files contained in an existing directory are not deleted, because multiple instances of a DataSinkTask might call this function at the same time and hence might perform concurrent delete operations on the file system (possibly deleting output files of concurrently running tasks). Since concurrent DataSinkTasks are not aware of each other, coordination of delete and create operations would be difficult. @param outPath Output path that should be prepared. @param writeMode Write mode to consider. @param createDirectory True, to initialize a directory at the given path, false otherwise. @return True, if the path was successfully prepared, false otherwise. @throws IOException

Examples of eu.stratosphere.core.fs.FileSystem.initOutPathLocalFS()

          // output is not written in parallel and should go to a single file
          
          if(!fs.isDistributedFS()) {
            // prepare local output path
            // checks for write mode and removes existing files in case of OVERWRITE mode
            if(!fs.initOutPathLocalFS(p, writeMode, false)) {
              // output preparation failed! Cancel task.
              throw new IOException("Output path could not be initialized. Canceling task.");
            }
          }

View Full Code Here

Examples of eu.stratosphere.core.fs.FileSystem.initOutPathLocalFS()

          // output is written in parallel into a directory or should always be written to a directory
          
          if(!fs.isDistributedFS()) {
            // File system is not distributed.
            // We need to prepare the output path on each executing node.
            if(!fs.initOutPathLocalFS(p, writeMode, true)) {
              // output preparation failed! Cancel task.
              throw new IOException("Output directory could not be created. Canceling task.");
            }
          }

View Full Code Here

Examples of org.apache.flink.core.fs.FileSystem.initOutPathLocalFS()

      
      if (numTasks == 1 && outputDirectoryMode == OutputDirectoryMode.PARONLY) {
        // output should go to a single file
        
        // prepare local output path. checks for write mode and removes existing files in case of OVERWRITE mode
        if(!fs.initOutPathLocalFS(p, writeMode, false)) {
          // output preparation failed! Cancel task.
          throw new IOException("Output path '" + p.toString() + "' could not be initialized. Canceling task...");
        }
      }
      else {

View Full Code Here

Examples of org.apache.flink.core.fs.FileSystem.initOutPathLocalFS()

        }
      }
      else {
        // numTasks > 1 || outDirMode == OutputDirectoryMode.ALWAYS
        
        if(!fs.initOutPathLocalFS(p, writeMode, true)) {
          // output preparation failed! Cancel task.
          throw new IOException("Output directory '" + p.toString() + "' could not be created. Canceling task...");
        }
      }
    }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.