Examples of parquet.io.api.RecordConsumer

parquet.io.api.RecordConsumer

Abstraction for writing records It decouples the striping algorithm from the actual record model example:

 startMessage() startField("A", 0) addValue(1) addValue(2) endField("A", 0) startField("B", 1) startGroup() startField("C", 0) addValue(3) endField("C", 0) endGroup() endField("B", 1) endMessage()

would produce the following message:

 { A: [1, 2] B: { C: 3 } }

@author Julien Le Dem

        this.recordConsumer = consumer;
    }


    @Override
    public void write(Object value) {
        RecordConsumer consumer = recordConsumer;
        String[] ns = names;
        PropertyExtractor[] ps = properties;
        ValueWriter[] vs = drivers;
        consumer.startMessage();
        for (int index = 0, n = ns.length; index < n; index++) {
            ValueOption<?> property = ps[index].extract(value);
            if (property.isNull() == false) {
                String name = ns[index];
                consumer.startField(name, index);
                vs[index].write(property, consumer);
                consumer.endField(name, index);
            }
        }
        consumer.endMessage();
    }

View Full Code Here

    }


  }


  public RecordConsumer getRecordWriter(ColumnWriteStore columns) {
    RecordConsumer recordWriter = new MessageColumnIORecordConsumer(columns);
    if (DEBUG) recordWriter = new RecordConsumerLoggingWrapper(recordWriter);
    return validating ? new ValidatingRecordConsumer(recordWriter, getType()) : recordWriter;
  }

View Full Code Here

TOP

Related Classes of parquet.io.api.RecordConsumer

com.asakusafw.directio.hive.parquet.DataModelWriteSupport

parquet.io.MessageColumnIO

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.