Examples of com.datasalt.pangool.tuplemr.mapred.lib.input.NullableCSVTokenizer

Package com.datasalt.pangool.tuplemr.mapred.lib.input

Examples of com.datasalt.pangool.tuplemr.mapred.lib.input.NullableCSVTokenizer

com.datasalt.pangool.tuplemr.mapred.lib.input.NullableCSVTokenizer
A custom CSVTokenizer policy that detects and provides null values if needed. Null values can happen in two ways: 1) If strict quoting mode is enabled, anything that is not quoted means null. 2) If a null string is provided, if it is found without quotes then it is null.
The null string provided can be null which means there will be no null string at all.


  }


  @Test(expected = IOException.class)
  public void testMasRecordSize() throws IOException {
    NullableCSVTokenizer tok = new NullableCSVTokenizer('\\', false, null);
    tok.setMaxFieldSize(10);
    CSVReader<String[]> cvs = new CSVReaderBuilder<String[]>(new StringReader(
        "hola,que,\"tal va la vida en este mundo tan cruel\",te,va,yobien"))
        .strategy(new CSVStrategy(',', '"', '\\', false, true)).tokenizer(tok)
        .entryParser(new DefaultCSVEntryParser()).build();

View Full Code Here


  public CSVReader<String[]> getCSVParser(String line, char separator, char quote, char escape,
      boolean strictQuotes, String nullString) {
    return new CSVReaderBuilder<String[]>(new StringReader(line))
        .strategy(new CSVStrategy(separator, quote, '#', false, true))
        .tokenizer(new NullableCSVTokenizer(escape, strictQuotes, nullString))
        .entryParser(new DefaultCSVEntryParser()).build();
  }

View Full Code Here

TOP

Related Classes of com.datasalt.pangool.tuplemr.mapred.lib.input.NullableCSVTokenizer

com.datasalt.pangool.tuplemr.mapred.lib.output.TestNullableCSVTokenizer

java.io.IOException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.