Examples of cascading.tuple.Tuple

cascading.tuple.Tuple
A Tuple represents a set of values. Consider a Tuple the same as a database record where every value is a column in that table.
A "tuple stream" is a set of Tuple instances passed consecutively through a Pipe assembly.
Tuples work in tandem with {@link Fields} and the {@link TupleEntry} classes. A TupleEntry holds an instance ofFields and a Tuple. It allows a tuple to be accessed by its field names, and will help maintain consistent types if any are given on the Fields instance. That is, if a field is specified at an Integer, calling {@link #set(int,Object)}with a String will force the String to be coerced into a Integer instance.
For managing custom types, see the {@link CoercibleType} interface which extends {@link Type}.
Tuple instances created by user code, by default, are mutable (or modifiable). Tuple instances created by the system are immutable (or unmodifiable, tested by calling {@link #isUnmodifiable()}).
For example tuples returned by {@link cascading.operation.FunctionCall#getArguments()}, will always be unmodifiable. Thus they must be copied if they will be changed by user code or cached in the local context. See the Tuple copy constructor, or {@code *Copy()} methodson {@link TupleEntry}.
Because a Tuple can hold any Object type, it is suitable for storing custom types. But all custom types must have a serialization support per the underlying framework.
For Hadoop, a {@link org.apache.hadoop.io.serializer.Serialization} implementationmust be registered with Hadoop. For further performance improvements, see the {@link cascading.tuple.hadoop.SerializationToken} Java annotation. @see org.apache.hadoop.io.serializer.Serialization @see cascading.tuple.hadoop.SerializationToken

    @Override
    public void operate(FlowProcess flowProcess, FunctionCall<NullContext> funcCall) {
        ParsedDatum datum = new ParsedDatum(funcCall.getArguments());
        Text key = new Text(datum.getUrl());
        Text value = new Text(datum.getTitle() + '\n' + datum.getParsedText());
        Tuple keyVal = new Tuple(key, value);
        funcCall.getOutputCollector().add(keyVal);
    }

View Full Code Here

        long fetchTime = System.currentTimeMillis();
        PartitioningKey groupingKey = new PartitioningKey("key", 1);
        FetchSetDatum pfd1 = new FetchSetDatum(urls, fetchTime, 0, groupingKey.getValue(), groupingKey.getRef());
        pfd1.setLastList(true);
        
        Tuple t = pfd1.getTuple();
        
        FetchSetDatum pfd2 = new FetchSetDatum(t);
        
        Assert.assertEquals(pfd1, pfd2);
    }

View Full Code Here

        
        String value3 = "value3\nwith\nreturns";
        headers.add("key3", value3);
        FetchedDatum datum = new FetchedDatum(url, url, 0, headers, new ContentBytes(), "", 0);
        
        Tuple tuple = datum.getTuple();
        
        FetchedDatum newDatum = new FetchedDatum(tuple);
        
        HttpHeaders newHeaders = newDatum.getHeaders();
        assertEquals("value", newHeaders.getFirst(key1));

View Full Code Here

            // Now we might need to remain the address, if this user has aliases.
            if (EMAIL_ALIASES.containsKey(address)) {
              address = EMAIL_ALIASES.get(address);
            }


            Tuple tuple = new Tuple(messageId, address, name, 0.0);
            functionCall.getOutputCollector().add(tuple);


            String replyId = metadata.get(Metadata.RELATION);


            if (replyId != null) {
              double score = analyzeReply(_content.toString());
              if (score > 0.0) {
                tuple = new Tuple(replyId, null, null, score);
                functionCall.getOutputCollector().add(tuple);
              }
            }
          } catch (Exception e) {
        LOGGER.error("Exception parsing email message", e);

View Full Code Here

      }


      emailNames.setLength(emailNames.length() - 2);
    }
    
    bufferCall.getOutputCollector().add(new Tuple(emailAddress, emailNames.toString(), summedScore));
  }

View Full Code Here

        HttpHeaders headers = new HttpHeaders();
        String key1 = "key\twith\ttabs";
        String value1 = "value1";
        headers.add(key1, value1);
        
        Tuple t = headers.toTuple();
        HttpHeaders newHeaders = new HttpHeaders(t);
        assertEquals(1, newHeaders.getNames().size());
        assertEquals(value1, newHeaders.getFirst(key1));
        
        String key2 = "key\n\r\fwith lots of funky chars";

View Full Code Here

      
      score += entry.getDouble(FieldNames.SCORE);
    }
    
    if (email != null) {
      bufferCall.getOutputCollector().add(new Tuple(email, name, score));
    }
  }

View Full Code Here

        TupleEntryCollector collector = funcCall.getOutputCollector();
        PageResult[] pageResults = datum.getPageResults();
        if (pageResults.length > 0) {
            for (PageResult pageResult : pageResults) {
                String outResult = String.format("%s\t%s\t%s", pageResult.getSourceUrl(), pageResult.getImageUrl(), pageResult.getDescription());
                collector.add(new Tuple(outResult));
            }
        }
    }

View Full Code Here

        OperationCall<NullContext> oc = Mockito.mock(OperationCall.class);
        BufferCall<NullContext> bc = Mockito.mock(BufferCall.class);
        
        TupleEntryCollector collector = Mockito.mock(TupleEntryCollector.class);
        
        Mockito.when(bc.getGroup()).thenReturn(new TupleEntry(new Tuple("http://localhost:8089")));
        Mockito.when(bc.getArgumentsIterator()).thenReturn(getGroupedurlDatumList("http://localhost:8089").iterator());
        Mockito.when(bc.getOutputCollector()).thenReturn(collector);
        
        TestWebServer server = null;

View Full Code Here

        OperationCall<NullContext> oc = Mockito.mock(OperationCall.class);
        BufferCall<NullContext> bc = Mockito.mock(BufferCall.class);
        
        TupleEntryCollector collector = Mockito.mock(TupleEntryCollector.class);
        
        Mockito.when(bc.getGroup()).thenReturn(new TupleEntry(new Tuple("http://localhost:8089")));
        Mockito.when(bc.getArgumentsIterator()).thenReturn(getGroupedurlDatumList("http://localhost:8089").iterator());
        Mockito.when(bc.getOutputCollector()).thenReturn(collector);
        
        TestWebServer server = null;

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of cascading.tuple.Tuple

bixo.config.BixoPlatform

bixo.datum.FetchSetDatum

bixo.datum.ParsedDatum

bixo.examples.webmining.AnalyzedDatum

bixo.operations.FilterAndScoreByUrlAndRobotsTest

cascading.BasicPipesPlatformTest

cascading.BufferPipesPlatformTest

cascading.CoGroupFieldedPipesPlatformTest

cascading.DistanceUseCasePlatformTest

cascading.FieldedPipesPlatformTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.