Examples of org.apache.hadoop.streaming.StreamXmlRecordReader

org.apache.hadoop.streaming.StreamXmlRecordReader
A way to interpret XML fragments as Mapper input records. Values are XML subtrees delimited by configurable tags. Keys could be the value of a certain attribute in the XML subtree, but this is left to the stream processor application. The name-value properties that StreamXmlRecordReader understands are: String begin (chars marking beginning of record) String end (chars marking end of record) int maxrec (maximum record size) int lookahead(maximum lookahead to sync CDATA) boolean slowmatch @author Michel Tourn

      FileSystem fs = file.getFileSystem(job);
      FSDataInputStream fileIn = fs.open(split.getPath());
      if (compressionCodecs != null && compressionCodecs.getCodec(file) != null)
        throw new RuntimeException("Not handling compression!");


      return new StreamXmlRecordReader(fileIn, split, reporter, job, FileSystem.get(job));
    }

View Full Code Here

TOP

Related Classes of org.apache.hadoop.streaming.StreamXmlRecordReader

edu.umd.hooka.CorpusVocabNormalizerAndNumberizer$XMLInput

org.apache.hadoop.io.DataOutputBuffer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.