List of openForRead() Examples

Examples of openForRead()

cascading.tap.MultiSourceTap.openForRead()
cascading.tap.Tap.openForRead()
Method openForRead opens the resource represented by this Tap instance for reading.
{@code input} value may be null, if so, sub-classes must inquire with the underlying {@link Scheme}via {@link Scheme#sourceConfInit(cascading.flow.FlowProcess,Tap,Object)} to get the properinput type and instantiate it before calling {@code super.openForRead()}.
Note the returned iterator will return the same instance of {@link cascading.tuple.TupleEntry} on every call,thus a copy must be made of either the TupleEntry or the underlying {@code Tuple} instance if they are to bestored in a Collection. @param flowProcess of type FlowProcess @param input of type Input @return TupleEntryIterator @throws java.io.IOException when the resource cannot be opened
cascading.tap.hadoop.Hfs.openForRead()
cascading.tap.hadoop.Lfs.openForRead()
cascading.tap.local.FileTap.openForRead()
com.nexirius.util.TextFile.openForRead()
The file is opened and the file id and version are read. Checks the file_id for equality (else throws TextFileException). If a null file_id is given the id check is skipped
org.drools.examples.broker.events.StockTickPersister.openForRead()

Examples of cascading.tap.Tap.openForRead()

        FlowConnector flowConnector = platform.makeFlowConnector();
        Flow flow = flowConnector.connect(in, FetchPipe.makeSinkMap(statusSink, contentSink), fetchPipe);
        flow.complete();
        
        Tap validate = platform.makeTap(platform.makeBinaryScheme(FetchedDatum.FIELDS), contentPath);
        TupleEntryIterator tupleEntryIterator = validate.openForRead(platform.makeFlowProcess());
        Assert.assertFalse(tupleEntryIterator.hasNext());
        
        validate = platform.makeTap(platform.makeBinaryScheme(StatusDatum.FIELDS), statusPath);
        tupleEntryIterator = validate.openForRead(platform.makeFlowProcess());

View Full Code Here

Examples of cascading.tap.Tap.openForRead()

        Tap validate = platform.makeTap(platform.makeBinaryScheme(FetchedDatum.FIELDS), contentPath);
        TupleEntryIterator tupleEntryIterator = validate.openForRead(platform.makeFlowProcess());
        Assert.assertFalse(tupleEntryIterator.hasNext());
        
        validate = platform.makeTap(platform.makeBinaryScheme(StatusDatum.FIELDS), statusPath);
        tupleEntryIterator = validate.openForRead(platform.makeFlowProcess());
        
        int numEntries = 0;
        while (tupleEntryIterator.hasNext()) {
            numEntries += 1;
            TupleEntry entry = tupleEntryIterator.next();

View Full Code Here

Examples of cascading.tap.Tap.openForRead()

        FlowConnector flowConnector = platform.makeFlowConnector();
        Flow flow = flowConnector.connect(in, FetchPipe.makeSinkMap(statusSink, contentSink), fetchPipe);
        flow.complete();
        
        Tap validate = platform.makeTap(platform.makeBinaryScheme(FetchedDatum.FIELDS), contentPath);
        TupleEntryIterator tupleEntryIterator = validate.openForRead(platform.makeFlowProcess());
        Assert.assertTrue(tupleEntryIterator.hasNext());
        tupleEntryIterator.next();
        Assert.assertFalse(tupleEntryIterator.hasNext());


        validate = platform.makeTap(platform.makeBinaryScheme(StatusDatum.FIELDS), statusPath);

View Full Code Here

Examples of cascading.tap.Tap.openForRead()

        Assert.assertTrue(tupleEntryIterator.hasNext());
        tupleEntryIterator.next();
        Assert.assertFalse(tupleEntryIterator.hasNext());


        validate = platform.makeTap(platform.makeBinaryScheme(StatusDatum.FIELDS), statusPath);
        tupleEntryIterator = validate.openForRead(platform.makeFlowProcess());
        
        int numSkippedEntries = 0;
        int numFetchedEntries = 0;
        while (tupleEntryIterator.hasNext()) {
            TupleEntry entry = tupleEntryIterator.next();

View Full Code Here

Examples of cascading.tap.Tap.openForRead()


    Flow flow = new LocalFlowConnector().connect( flowDef );


    flow.complete();


    TupleEntryIterator iterator = resultsTap.openForRead( flow.getFlowProcess() );


    while( iterator.hasNext() )
      System.out.println( iterator.next() );


    iterator.close();

View Full Code Here

Examples of cascading.tap.Tap.openForRead()

    assertEquals( Hfs.CombinedInputFormat.class, inputFormat.getClass() );
    InputSplit[] splits = inputFormat.getSplits( conf, 1 );


    assertEquals( 1, splits.length );


    validateLength( source.openForRead( process ), 10 );
    }
  }

View Full Code Here

Examples of cascading.tap.Tap.openForRead()

  @Test
  public void testTupleEntryNextTwice() throws IOException
    {
    Tap tap = getPlatform().getTextFile( inputFileNums10 );


    TupleEntryIterator iterator = tap.openForRead( getPlatform().getFlowProcess() );


    int count = 0;
    while( iterator.hasNext() )
      {
      iterator.next();

View Full Code Here

Examples of cascading.tap.Tap.openForRead()

    }
    tec.close();


    // read results back out
    Tap outputTap = new Hfs(new ProtobufScheme("value", Example.Person.class), "/tmp/input");
    TupleEntryIterator iter = outputTap.openForRead(new HadoopFlowProcess(), null);
    List<Tuple> tuples = new ArrayList<Tuple>();
    while (iter.hasNext()) {
      tuples.add(iter.next().getTupleCopy());
    }

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForRead()


    TupleEntryIterator reader = null;


    try
      {
      reader = temp.openForRead( new HadoopFlowProcess( jobConf ) );


      if( !reader.hasNext() )
        throw new FlowException( "step state path is empty: " + temp.getIdentifier() );


      return reader.next().getString( 0 );

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForRead()

    Map<Object, Object> properties = new HashMap<Object, Object>(){{
      put("io.serializations", new JobConf().get("io.serializations") + "," + ProtobufSerialization.class.getName());
    }};
    new HadoopFlowConnector(properties).connect(t, sink, groupByPipe).complete();


    TupleEntryIterator tei = sink.openForRead(new HadoopFlowProcess(new JobConf()));
    Set<Tuple> tuples = new HashSet<Tuple>();
    while (tei.hasNext()) {
      tuples.add(tei.next().getTupleCopy());
    }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.