Examples of io.netty.handler.codec.LengthFieldBasedFrameDecoder

io.netty.handler.codec.LengthFieldBasedFrameDecoder
A decoder that splits the received {@link ByteBuf}s dynamically by the value of the length field in the message. It is particularly useful when you decode a binary message which has an integer header field that represents the length of the message body or the whole message.
{@link LengthFieldBasedFrameDecoder} has many configuration parameters sothat it can decode any message with a length field, which is often seen in proprietary client-server protocols. Here are some example that will give you the basic idea on which option does what.
2 bytes length field at offset 0, do not strip header
The value of the length field in this example is 12 (0x0C) which represents the length of "HELLO, WORLD". By default, the decoder assumes that the length field represents the number of the bytes that follows the length field. Therefore, it can be decoded with the simplistic parameter combination.
```
 lengthFieldOffset   = 0 lengthFieldLength   = 2 lengthAdjustment    = 0 initialBytesToStrip = 0 (= do not strip header) BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes) +--------+----------------+      +--------+----------------+ | Length | Actual Content |----->| Length | Actual Content | | 0x000C | "HELLO, WORLD" |      | 0x000C | "HELLO, WORLD" | +--------+----------------+      +--------+----------------+ 
```
2 bytes length field at offset 0, strip header
Because we can get the length of the content by calling {@link ByteBuf#readableBytes()}, you might want to strip the length field by specifying initialBytesToStrip. In this example, we specified 2, that is same with the length of the length field, to strip the first two bytes.
```
 lengthFieldOffset   = 0 lengthFieldLength   = 2 lengthAdjustment    = 0 initialBytesToStrip = 2 (= the length of the Length field) BEFORE DECODE (14 bytes)         AFTER DECODE (12 bytes) +--------+----------------+      +----------------+ | Length | Actual Content |----->| Actual Content | | 0x000C | "HELLO, WORLD" |      | "HELLO, WORLD" | +--------+----------------+      +----------------+ 
```
2 bytes length field at offset 0, do not strip header, the length field represents the length of the whole message
In most cases, the length field represents the length of the message body only, as shown in the previous examples. However, in some protocols, the length field represents the length of the whole message, including the message header. In such a case, we specify a non-zero lengthAdjustment. Because the length value in this example message is always greater than the body length by 2, we specify -2 as lengthAdjustment for compensation.
```
 lengthFieldOffset   =  0 lengthFieldLength   =  2 lengthAdjustment    = -2 (= the length of the Length field) initialBytesToStrip =  0 BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes) +--------+----------------+      +--------+----------------+ | Length | Actual Content |----->| Length | Actual Content | | 0x000E | "HELLO, WORLD" |      | 0x000E | "HELLO, WORLD" | +--------+----------------+      +--------+----------------+ 
```
3 bytes length field at the end of 5 bytes header, do not strip header
The following message is a simple variation of the first example. An extra header value is prepended to the message. lengthAdjustment is zero again because the decoder always takes the length of the prepended data into account during frame length calculation.
```
 lengthFieldOffset   = 2 (= the length of Header 1) lengthFieldLength   = 3 lengthAdjustment    = 0 initialBytesToStrip = 0 BEFORE DECODE (17 bytes)                      AFTER DECODE (17 bytes) +----------+----------+----------------+      +----------+----------+----------------+ | Header 1 |  Length  | Actual Content |----->| Header 1 |  Length  | Actual Content | |  0xCAFE  | 0x00000C | "HELLO, WORLD" |      |  0xCAFE  | 0x00000C | "HELLO, WORLD" | +----------+----------+----------------+      +----------+----------+----------------+ 
```
3 bytes length field at the beginning of 5 bytes header, do not strip header
This is an advanced example that shows the case where there is an extra header between the length field and the message body. You have to specify a positive lengthAdjustment so that the decoder counts the extra header into the frame length calculation.
```
 lengthFieldOffset   = 0 lengthFieldLength   = 3 lengthAdjustment    = 2 (= the length of Header 1) initialBytesToStrip = 0 BEFORE DECODE (17 bytes)                      AFTER DECODE (17 bytes) +----------+----------+----------------+      +----------+----------+----------------+ |  Length  | Header 1 | Actual Content |----->|  Length  | Header 1 | Actual Content | | 0x00000C |  0xCAFE  | "HELLO, WORLD" |      | 0x00000C |  0xCAFE  | "HELLO, WORLD" | +----------+----------+----------------+      +----------+----------+----------------+ 
```
2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field
This is a combination of all the examples above. There are the prepended header before the length field and the extra header after the length field. The prepended header affects the lengthFieldOffset and the extra header affects the lengthAdjustment. We also specified a non-zero initialBytesToStrip to strip the length field and the prepended header from the frame. If you don't want to strip the prepended header, you could specify 0 for initialBytesToSkip.
```
 lengthFieldOffset   = 1 (= the length of HDR1) lengthFieldLength   = 2 lengthAdjustment    = 1 (= the length of HDR2) initialBytesToStrip = 3 (= the length of HDR1 + LEN) BEFORE DECODE (16 bytes)                       AFTER DECODE (13 bytes) +------+--------+------+----------------+      +------+----------------+ | HDR1 | Length | HDR2 | Actual Content |----->| HDR2 | Actual Content | | 0xCA | 0x000C | 0xFE | "HELLO, WORLD" |      | 0xFE | "HELLO, WORLD" | +------+--------+------+----------------+      +------+----------------+ 
```
2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field, the length field represents the length of the whole message
Let's give another twist to the previous example. The only difference from the previous example is that the length field represents the length of the whole message instead of the message body, just like the third example. We have to count the length of HDR1 and Length into lengthAdjustment. Please note that we don't need to take the length of HDR2 into account because the length field already includes the whole header length.
```
 lengthFieldOffset   =  1 lengthFieldLength   =  2 lengthAdjustment    = -3 (= the length of HDR1 + LEN, negative) initialBytesToStrip =  3 BEFORE DECODE (16 bytes)                       AFTER DECODE (13 bytes) +------+--------+------+----------------+      +------+----------------+ | HDR1 | Length | HDR2 | Actual Content |----->| HDR2 | Actual Content | | 0xCA | 0x0010 | 0xFE | "HELLO, WORLD" |      | 0xFE | "HELLO, WORLD" | +------+--------+------+----------------+      +------+----------------+ 
```
@see LengthFieldPrepender

          .option(ChannelOption.SO_KEEPALIVE, true).childOption(ChannelOption.SO_KEEPALIVE, true)
          .childHandler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel socketChannel) throws Exception {
              socketChannel.pipeline().addLast(
                  new LengthFieldBasedFrameDecoder(MAX_FRAME_LENGTH, LENGTH_FIELD_OFFSET,
                      LENGTH_FIELD_LENGTH, LENGTH_ADJUSTMENT, INITIAL_BYTES_TO_STRIP));
              socketChannel.pipeline().addLast(new HelixIPCCallbackHandler());
            }
          }).bind(new InetSocketAddress(config.getPort()));

View Full Code Here

                                                                        final int lengthFieldLength, final int lengthAdjustment,
                                                                        final int initialBytesToStrip) {
        return new DefaultChannelHandlerFactory() {
            @Override
            public ChannelHandler newChannelHandler() {
                return new LengthFieldBasedFrameDecoder(maxFrameLength, lengthFieldOffset, lengthFieldLength, lengthAdjustment, initialBytesToStrip);
            }
        };
    }

View Full Code Here

  @Override
    public void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();


        pipeline.addLast("info", new SendConnectInfoHandler(clientMsg));
        pipeline.addLast("frame", new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE, 0, 4, 0, 4));
        pipeline.addLast("decoder", DECODER);


        pipeline.addLast("handler", CLIENTHANDLER);
    }

View Full Code Here

        new PipelineConfigurator<RemoteRxEvent, RemoteRxEvent>(){
          @Override
          public void configureNewPipeline(ChannelPipeline pipeline) {
//            pipeline.addFirst(new LoggingHandler(LogLevel.ERROR)); // uncomment to enable debug logging        
            pipeline.addLast("frameEncoder", new LengthFieldPrepender(4)); // 4 bytes to encode length
            pipeline.addLast("frameDecoder", new LengthFieldBasedFrameDecoder(524288, 0, 4, 0, 4)); // max frame = half MB
          }
        }, new RxEventPipelineConfigurator()))
      .connect()
      // send subscription request, get input stream
      .flatMap(new Func1<ObservableConnection<RemoteRxEvent, RemoteRxEvent>, Observable<RemoteRxEvent>>(){

View Full Code Here

        new PipelineConfigurator<RemoteRxEvent, RemoteRxEvent>(){
          @Override
          public void configureNewPipeline(ChannelPipeline pipeline) {
//            pipeline.addFirst(new LoggingHandler(LogLevel.ERROR)); // uncomment to enable debug logging  
            pipeline.addLast("frameEncoder", new LengthFieldPrepender(4)); // 4 bytes to encode length
            pipeline.addLast("frameDecoder", new LengthFieldBasedFrameDecoder(524288, 0, 4, 0, 4)); // max frame = half MB
          }
        }, new RxEventPipelineConfigurator()),   
        new RemoteObservableConnectionHandler(configuredObservables, builder.getIngressPolicy(), blockUntilCompleted,
            metrics));
    this.server = server;

View Full Code Here

              // FixedLengthFrameDecoder (same as used with the
              // non-authenticated pipeline) after authentication
              // completes (as in non-auth pipeline below).
              PipelineUtils.addLastWithExecutorCheck(
                  "length-field-based-frame-decoder",
                  new LengthFieldBasedFrameDecoder(1024, 0, 4, 0, 4),
                  handlerToUseExecutionGroup, executionGroup, ch);
              PipelineUtils.addLastWithExecutorCheck("request-encoder",
                  new RequestEncoder(conf), handlerToUseExecutionGroup,
                  executionGroup, ch);
              // The following pipeline component responds to the server's SASL

View Full Code Here

            PipelineUtils.addLastWithExecutorCheck("compressionEncoder",
                conf.getNettyCompressionEncoder(),
                handlerToUseExecutionGroup, executionGroup, ch);
          }
          PipelineUtils.addLastWithExecutorCheck("requestFrameDecoder",
              new LengthFieldBasedFrameDecoder(1024 * 1024 * 1024, 0, 4, 0, 4),
              handlerToUseExecutionGroup, executionGroup, ch);
          PipelineUtils.addLastWithExecutorCheck("requestDecoder",
              new RequestDecoder(conf, inByteCounter),
              handlerToUseExecutionGroup, executionGroup, ch);
          // Removed after authentication completes:
          PipelineUtils.addLastWithExecutorCheck("saslServerHandler",
              saslServerHandlerFactory.newHandler(conf),
              handlerToUseExecutionGroup, executionGroup, ch);
          PipelineUtils.addLastWithExecutorCheck("authorizeServerHandler",
              new AuthorizeServerHandler(), handlerToUseExecutionGroup,
              executionGroup, ch);
          PipelineUtils.addLastWithExecutorCheck("requestServerHandler",
              requestServerHandlerFactory.newHandler(workerRequestReservedMap,
                  conf, myTaskInfo, exceptionHandler),
              handlerToUseExecutionGroup, executionGroup, ch);
          // Removed after authentication completes:
          PipelineUtils.addLastWithExecutorCheck("responseEncoder",
              new ResponseEncoder(), handlerToUseExecutionGroup,
              executionGroup, ch);
        } else {
          LOG.info("start: Using Netty without authentication.");
/*end[HADOOP_NON_SECURE]*/
          // Store all connected channels in order to ensure that we can close
          // them on stop(), or else stop() may hang waiting for the
          // connections to close on their own
          ch.pipeline().addLast("connectedChannels",
              new ChannelInboundHandlerAdapter() {
                @Override
                public void channelActive(ChannelHandlerContext ctx)
                  throws Exception {
                  accepted.add(ctx.channel());
                  ctx.fireChannelActive();
                }
              });
          PipelineUtils.addLastWithExecutorCheck("serverInboundByteCounter",
              inByteCounter, handlerToUseExecutionGroup, executionGroup, ch);
          if (conf.doCompression()) {
            PipelineUtils.addLastWithExecutorCheck("compressionDecoder",
                conf.getNettyCompressionDecoder(),
                handlerToUseExecutionGroup, executionGroup, ch);
          }
          PipelineUtils.addLastWithExecutorCheck("serverOutboundByteCounter",
              outByteCounter, handlerToUseExecutionGroup, executionGroup, ch);
          if (conf.doCompression()) {
            PipelineUtils.addLastWithExecutorCheck("compressionEncoder",
                conf.getNettyCompressionEncoder(),
                handlerToUseExecutionGroup, executionGroup, ch);
          }
          PipelineUtils.addLastWithExecutorCheck("requestFrameDecoder",
              new LengthFieldBasedFrameDecoder(1024 * 1024 * 1024, 0, 4, 0, 4),
              handlerToUseExecutionGroup, executionGroup, ch);
          PipelineUtils.addLastWithExecutorCheck("requestDecoder",
              new RequestDecoder(conf, inByteCounter),
              handlerToUseExecutionGroup, executionGroup, ch);
          PipelineUtils.addLastWithExecutorCheck("requestServerHandler",

View Full Code Here

    // maxFrameLength = 2G
    // lengthFieldOffset = 0
    // lengthFieldLength = 8
    // lengthAdjustment = -8, i.e. exclude the 8 byte length itself
    // initialBytesToStrip = 8, i.e. strip out the length field itself
    return new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE, 0, 8, -8, 8);
  }

View Full Code Here

public class LengthBasedInitializer extends ChannelInitializer<Channel> {


    @Override
    protected void initChannel(Channel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        pipeline.addLast(new LengthFieldBasedFrameDecoder(65 * 1024, 0, 8));
        pipeline.addLast(new FrameHandler());
    }

View Full Code Here


public class LengthFieldBasedFrameDecoderTest {
    @Test
    public void testFailSlowTooLongFrameRecovery() throws Exception {
        EmbeddedChannel ch = new EmbeddedChannel(
                new LengthFieldBasedFrameDecoder(5, 0, 4, 0, 4, false));


        for (int i = 0; i < 2; i ++) {
            assertFalse(ch.writeInbound(Unpooled.wrappedBuffer(new byte[] { 0, 0, 0, 2 })));
            try {
                assertTrue(ch.writeInbound(Unpooled.wrappedBuffer(new byte[] { 0, 0 })));

View Full Code Here

0 1

TOP

Related Classes of io.netty.handler.codec.LengthFieldBasedFrameDecoder

com.github.diamond.client.netty.ClientChannelInitializer

com.manning.nettyinaction.chapter8.LengthBasedInitializer

io.netty.buffer.ByteBuf

io.netty.handler.codec.frame.LengthFieldBasedFrameDecoderTest

io.reactivex.netty.RemoteObservable

io.reactivex.netty.RemoteRxServer

org.apache.camel.component.netty4.ChannelHandlerFactories

org.apache.giraph.comm.netty.NettyClient

org.apache.giraph.comm.netty.NettyServer

org.apache.helix.ipc.netty.NettyHelixIPCService

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of io.netty.handler.codec.LengthFieldBasedFrameDecoder

2 bytes length field at offset 0, do not strip header

2 bytes length field at offset 0, strip header

2 bytes length field at offset 0, do not strip header, the length field represents the length of the whole message

3 bytes length field at the end of 5 bytes header, do not strip header

3 bytes length field at the beginning of 5 bytes header, do not strip header

2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field

2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field, the length field represents the length of the whole message

Related Classes of io.netty.handler.codec.LengthFieldBasedFrameDecoder