Examples of cascading.operation.regex.RegexParser

cascading.operation.regex.RegexParser
Class RegexParser is used to extract a matched regex from an incoming argument value.
Sometimes its useful to parse out a value from a key/value pair in a string, if the key exists. If the key does not exist, returning an empty string instead of failing is typically expected.
The following regex can extract a value from {@code key1=value1&key2=value2} if key1 exists, otherwise anempty string is returned:
```
(?<=key1=)[^&]*|$
```


    Tap source = new Hfs( new TextLine( new Fields( "offset", "line" ) ), inputFileApache );


    Pipe pipe = new Pipe( "test" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );


    pipe = new Each( pipe, new InsertRawBytes( new Fields( "bytes" ), "inserted text as bytes", true, true ), Fields.ALL );


    Fields bytes = new Fields( "bytes" );
    bytes.setComparator( "bytes", new BytesComparator() );

View Full Code Here


    Tap source = new Hfs( new TextLine( sourceFields ), inputFileApache );


    Pipe pipe = new Pipe( "test" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "offset", "ip" ) );


    pipe = new GroupBy( pipe, new Fields( "offset" ) );


    pipe = new Every( pipe, new Count(), new Fields( "offset", "count" ) );

View Full Code Here


    Tap source = getPlatform().getTextFile( inputFileApache );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );
    pipe = new GroupBy( "reduce", pipe, new Fields( "ip" ) );
    pipe = new Every( pipe, new Count(), new Fields( "ip", "count" ) );


    Tap sink = getPlatform().getTextFile( getOutputPath( "none/tap" ), SinkMode.REPLACE );
    Tap trap = getPlatform().getTextFile( getOutputPath( "none/trap" ), SinkMode.REPLACE );

View Full Code Here


    Tap source = getPlatform().getTextFile( inputFileApache );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );


    // always fail
    pipe = new Each( pipe, new Fields( "ip" ), new TestFunction( new Fields( "test" ), null ), Fields.ALL );


    pipe = new GroupBy( "reduce", pipe, new Fields( "ip" ) );

View Full Code Here


    Tap source = getPlatform().getTextFile( inputFileApache );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );


    // always fail
    pipe = new Each( pipe, new Fields( "ip" ), new TestFunction( new Fields( "test" ), null ), Fields.ALL );


    pipe = new GroupBy( "reduce", pipe, new Fields( "ip" ) );

View Full Code Here


    Tap source = getPlatform().getTextFile( inputFileApache );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );


    pipe = new GroupBy( "reduce", pipe, new Fields( "ip" ) );
    pipe = new Every( pipe, new Count(), new Fields( "ip", "count" ) );
    pipe = new Every( pipe, new TestFailAggregator( new Fields( "fail" ), failAt ), new Fields( "ip", "count" ) );

View Full Code Here


    Tap source = getPlatform().getTextFile( inputFileApache );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );


    // always fail
    pipe = new Each( pipe, new TestFunction( new Fields( "test" ), new Tuple( 1 ), 1 ), Fields.ALL );
    pipe = new Each( pipe, new TestFunction( new Fields( "test2" ), new Tuple( 2 ), 2 ), Fields.ALL );
    pipe = new Each( pipe, new TestFunction( new Fields( "test3" ), new Tuple( 3 ), 3 ), Fields.ALL );

View Full Code Here


    Tap source = getPlatform().getTextFile( inputFileApache );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );


    // always fail
    pipe = new Each( pipe, AssertionLevel.VALID, new AssertNotEquals( "75.185.76.245" ) );
    pipe = new GroupBy( pipe, new Fields( "ip" ) );
    pipe = new Each( pipe, AssertionLevel.VALID, new AssertNotEquals( "68.46.103.112" ) );

View Full Code Here


    Tap source = getPlatform().getTextFile( inputFileApache );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );


    // always fail
    pipe = new Each( pipe, new Fields( "ip" ), new TestFunction( new Fields( "test" ), null ), Fields.ALL );


    pipe = new GroupBy( "reduce", pipe, new Fields( "ip" ) );

View Full Code Here


    Tap source = getPlatform().getTap( scheme, inputFileApache, SinkMode.KEEP );


    Pipe pipe = new Pipe( "map" );


    pipe = new Each( pipe, new Fields( "line" ), new RegexParser( new Fields( "ip" ), "^[^ ]*" ), new Fields( "ip" ) );
    pipe = new GroupBy( pipe, new Fields( "ip" ) );
    pipe = new Every( pipe, new Count(), new Fields( "ip", "count" ) );


    Tap sink = getPlatform().getTap( scheme, getOutputPath( "trapsourcesink/sink" ), SinkMode.REPLACE );

View Full Code Here

0 1 2 3 4 5 6

TOP

Related Classes of cascading.operation.regex.RegexParser

bixo.examples.webmining.DemoWebMiningWorkflow

cascading.BasicPipesPlatformTest

cascading.BasicTrapPlatformTest

cascading.FieldedPipesPlatformTest

cascading.flow.AssemblyPlannerPlatformTest

cascading.flow.FlowStrategiesPlatformTest

cascading.flow.hadoop.BuildJobsHadoopPlatformTest

cascading.flow.SubAssemblyPlatformTest$TestAssembly

cascading.operation.assertion.BuildAssertionsPlatformTest

cascading.operation.debug.BuildDebugPlatformTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.