Examples of org.apache.hadoop.mapred.FileInputFormat

org.apache.hadoop.mapred.FileInputFormat
A base class for file-based {@link InputFormat}.
FileInputFormat is the base class for all file-based InputFormats. This provides a generic implementation of {@link #getSplits(JobConf,int)}. Subclasses of FileInputFormat can also override the {@link #isSplitable(FileSystem,Path)} method to ensure input-files arenot split-up and are processed as a whole by {@link Mapper}s. @deprecated Use {@link org.apache.hadoop.mapreduce.lib.input.FileInputFormat}instead.


    private InputSplit[] getInputSplits(JobConf conf,
        String inputFormatClass, String inputPath, int numSplits)
        throws ClassNotFoundException, IOException {
      // Create a new instance of the input format
      FileInputFormat inputFormat = (FileInputFormat) ReflectionUtils
          .newInstance(Class.forName(inputFormatClass), conf);


      // Set the input path for the left data set
      inputFormat.setInputPaths(conf, inputPath);


      // Get the left input splits
      return inputFormat.getSplits(conf, numSplits);
    }

View Full Code Here

      this.rightIS = split.get(1);
      this.rightReporter = reporter;


      try {
        // Create left record reader
        FileInputFormat leftFIF = (FileInputFormat) ReflectionUtils
            .newInstance(Class.forName(conf
                .get(CartesianInputFormat.LEFT_INPUT_FORMAT)),
                conf);


        leftRR = leftFIF.getRecordReader(split.get(0), conf, reporter);


        // Create right record reader
        rightFIF = (FileInputFormat) ReflectionUtils.newInstance(Class
            .forName(conf
                .get(CartesianInputFormat.RIGHT_INPUT_FORMAT)),

View Full Code Here

TOP

Related Classes of org.apache.hadoop.mapred.FileInputFormat

com.google.common.base.Stopwatch

mrdp.ch5.CartesianProduct$CartesianInputFormat

mrdp.ch5.CartesianProduct$CartesianRecordReader

org.apache.hadoop.fs.BlockLocation

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.LocatedFileStatus

org.apache.hadoop.fs.Path

org.apache.hadoop.fs.PathFilter

org.apache.hadoop.net.NetworkTopology

org.apache.hadoop.net.Node

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.