Examples of org.kitesdk.data.spi.FieldPartitioner

org.kitesdk.data.spi.FieldPartitioner

Partitions values for a named field.

Used by a {@link org.kitesdk.data.PartitionStrategy} to calculate which partition an entitybelongs in, based on the value of a given field, called the source field. A field partitioner can, in some cases, provide meaningful cardinality hints to query systems. A good example of this is a hash partitioner which always knows the number of buckets produced by the function.

Implementations of {@link FieldPartitioner} are immutable.
@param < S> The type of the source field in the entity. The partition function mustaccept values of this type. @param < T> The type of the target field, which is the type of the return value of thepartition function. @see org.kitesdk.data.PartitionStrategy

    List<Object> values = Lists.newArrayList();
    int i = 0;
    for (String part : parts) {
      Iterator<String> split = Splitter.on('=').split(part).iterator();
      String fieldName = split.next();
      FieldPartitioner fp = fieldPartitioners.get(i++);
      if (!fieldName.equals(fp.getName())) {
        throw new IllegalArgumentException(String.format("Unrecognized partition name " +
            "'%s' in partition %s, expecting '%s'.", fieldName, partitionUri,
            fp.getName()));
      }
      if (!split.hasNext()) {
        throw new IllegalArgumentException(String.format("Missing partition value for " +
            "'%s' in partition %s.", fieldName, partitionUri));
      }

View Full Code Here

0 1

TOP

Related Classes of org.kitesdk.data.spi.FieldPartitioner

org.kitesdk.data.crunch.CrunchDatasets$AvroStorageKey

org.kitesdk.data.hbase.DaoView

org.kitesdk.data.hbase.impl.KeySchema

org.kitesdk.data.spi.filesystem.FileSystemDataset

org.kitesdk.data.spi.filesystem.FileSystemDatasetRepository

org.kitesdk.data.spi.filesystem.PathConversion

org.kitesdk.data.TestPartitionExpression

org.kitesdk.data.TestPartitionStrategy

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.