Examples of org.apache.hadoop.fs.FileContext

• org.apache.hadoop.fs.FileContext
• the hdfs file system hdfs://nnAddress:nnPort/path

While URI names are very flexible, it requires knowing the name or address of the server. For convenience one often wants to access the default system in one's environment without knowing its name/address. This has an additional benefit that it allows one to change one's default fs (e.g. admin moves application from cluster1 to cluster2).

To facilitate this, Hadoop supports a notion of a default file system. The user can set his default file system, although this is typically set up for you in your environment via your default config. A default file system implies a default scheme and authority; slash-relative names (such as /for/bar) are resolved relative to that default FS. Similarly a user can also have working-directory-relative names (i.e. names not starting with a slash). While the working directory is generally in the same default FS, the wd can be in a different FS.

Hence Hadoop path names can be one of:

• fully qualified URI: scheme://authority/path
• slash relative names: /path relative to the default file system
• wd-relative names: path relative to the working dir

Relative paths with scheme (scheme:foo/bar) are illegal.

****The Role of the FileContext and configuration defaults****

The FileContext provides file namespace context for resolving file names; it also contains the umask for permissions, In that sense it is like the per-process file-related state in Unix system. These two properties

• default file system i.e your slash)
• umask

in general, are obtained from the default configuration file in your environment, (@see {@link Configuration}). No other configuration parameters are obtained from the default config as far as the file context layer is concerned. All file system instances (i.e. deployments of file systems) have default properties; we call these server side (SS) defaults. Operation like create allow one to select many properties: either pass them in as explicit parameters or use the SS properties.

The file system related SS defaults are

• the home directory (default is "/user/userName")
• the initial wd (only for local fs)
• replication factor
• block size
• buffer size
• encryptDataTransfer
• checksum option. (checksumType and bytesPerChecksum)

*** Usage Model for the FileContext class ***

Example 1: use the default config read from the $HADOOP_CONFIG/core.xml. Unspecified values come from core-defaults.xml in the release jar.

• myFContext = FileContext.getFileContext(); // uses the default config // which has your default FS
• myFContext.create(path, ...);
• myFContext.setWorkingDir(path)
• myFContext.open (path, ...);

Example 2: Get a FileContext with a specific URI as the default FS

• myFContext = FileContext.getFileContext(URI)
• myFContext.create(path, ...); ...

Example 3: FileContext with local file system as the default

• myFContext = FileContext.getLocalFSFileContext()
• myFContext.create(path, ...);
• ...

Example 4: Use a specific config, ignoring $HADOOP_CONFIG Generally you should not need use a config unless you are doing

• configX = someConfigSomeOnePassedToYou.
• myFContext = getFileContext(configX); // configX is not changed, // is passed down
• myFContext.create(path, ...);
• ...