Examples of cascading.operation.Aggregator

cascading.operation.Aggregator
An Aggregator takes the set of all values associated with a unique grouping and returns zero or more values. {@link cascading.operation.aggregator.MaxValue}, {@link cascading.operation.aggregator.MinValue}, {@link cascading.operation.aggregator.Count}, and {@link cascading.operation.aggregator.Average} are good examples.
Aggregator implementations should be reentrant. There is no guarantee an Aggregator instance will be executed in a unique vm, or by a single thread. The {@link #start(cascading.flow.FlowProcess,AggregatorCall)}method provides a mechanism for maintaining a 'context' object to hold intermediate values.
Note {@link TupleEntry} instances are reused internally so should not be stored. Instead use the TupleEntry or Tuplecopy constructors to make safe copies.
Since Aggregators can be chained, and Cascading pipelines all operation results, any Aggregators coming ahead of the current Aggregator must return a value before the {@link #complete(cascading.flow.FlowProcess,AggregatorCall)}method on this Aggregator is called. Subsequently, if any previous Aggregators return more than one Tuple result, this complete() method will be called for each Tuple emitted.
Thus it is a best practice to implement a {@link Buffer} when emitting more than one, or zero Tuple results. @see AggregatorCall @see OperationCall

    pipeline = new GroupBy(pipeline, new Fields("resource"));


    // For every Tuple group
    // count the number of occurrences of "word" and store result in
    // a field named "count"
    Aggregator count = new Count(new Fields("resource"));
    pipeline = new Every(pipeline, count);




    // create a SINK tap to write to the default filesystem
    // by default, TextLine writes all fields out

View Full Code Here

    }


  @Test
  public void testAverage()
    {
    Aggregator aggregator = new Average();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testCount()
    {
    Aggregator aggregator = new Count();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testFirst()
    {
    Aggregator aggregator = new First();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testFirstN()
    {
    Aggregator aggregator = new First( 3 );


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testLast()
    {
    Aggregator aggregator = new Last();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testMax()
    {
    Aggregator aggregator = new Max();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testMin()
    {
    Aggregator aggregator = new Min();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testSum()
    {
    Aggregator aggregator = new Sum();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

    }


  @Test
  public void testMaxValue()
    {
    Aggregator aggregator = new MaxValue();


    Tuple[] arguments = new Tuple[]{new Tuple( new Double( 1.0 ) ), new Tuple( new Double( 3.0 ) ),
                                    new Tuple( new Double( 2.0 ) ), new Tuple( new Double( 4.0 ) ),
                                    new Tuple( new Double( -5.0 ) )};

View Full Code Here

0 1

TOP

Related Classes of cascading.operation.Aggregator

cascading.BasicPipesPlatformTest

cascading.JoinFieldedPipesPlatformTest

cascading.operation.aggregator.AggregatorsTest

com.manning.hip.ch12.cascading.PopularLogResources

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.