Examples of co.cask.cdap.data2.transaction.stream.AbstractStreamFileConsumer

Package co.cask.cdap.data2.transaction.stream

Examples of co.cask.cdap.data2.transaction.stream.AbstractStreamFileConsumer

co.cask.cdap.data2.OperationException

A {@link StreamConsumer} that read events from stream file and uses a table to store consumer states.

State table ROW key schema:

 {@code row_key =   group_id = 8 bytes consumer group id stream_file_offset =      partition_start = 8 bytes timestamp of partition start time partition_end = 8 bytes timestamp of partition end time name_prefix = DataOutput UTF-8 output. sequence_id = 4 bytes stream file sequence id offset = 8 bytes offset inside the stream file}

The state table has single column ( {@link QueueEntryRow#COLUMN_FAMILY}: {@link QueueEntryRow#STATE_COLUMN_PREFIX}) to store state. The state value:

 {@code state_value =    write_pointer = 8 bytes Transaction write point of the consumer who update this state. instance_id = 4 bytes Instance id of the consumer who update this state. state = ConsumerEntryState.getState(), either CLAIMED or PROCESSED}

      hTable.increment(increment);
    } catch (IOException e) {
      // figure out whether this is an illegal increment
      // currently there is not other way to extract that from the HBase exception than string match
      if (e.getMessage() != null && e.getMessage().contains("isn't 64 bits wide")) {
        throw new OperationException(StatusCode.ILLEGAL_INCREMENT, e.getMessage(), e);
      }
      throw e;
    }
    hTable.flushCommits();
  }

View Full Code Here

      return Bytes.toLong(result.getValue(DATA_COLUMN_FAMILY, column));
    } catch (IOException e) {
      // figure out whether this is an illegal increment
      // currently there is not other way to extract that from the HBase exception than string match
      if (e.getMessage() != null && e.getMessage().contains("isn't 64 bits wide")) {
        throw new OperationException(StatusCode.ILLEGAL_INCREMENT, e.getMessage(), e);
      }
      throw e;
    }
  }

View Full Code Here

      byte[] oldVal = (oldValue == null) ? null : Bytes.toBytes(oldValue);
      byte[] newVal = (newValue == null) ? null : Bytes.toBytes(newValue);


      return aggregatesTable.swap(row, col, oldVal, newVal);
    } catch (Exception e) {
      throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
    }
  }

View Full Code Here

          increments.put(Bytes.toBytes(tag.getTag()), (long) tag.getValue());
        }
        aggregatesTable.increment(rowKey, increments);
      }
    } catch (Exception e) {
      throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
    }
  }

View Full Code Here

  public void delete(String contextPrefix) throws OperationException {
    Preconditions.checkArgument(contextPrefix != null, "null context not allowed");
    try {
      aggregatesTable.deleteAll(entityCodec.encodeWithoutPadding(MetricsEntityType.CONTEXT, contextPrefix));
    } catch (Exception e) {
      throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
    }
  }

View Full Code Here

    byte[] endRow = getRawPaddedKey(contextPrefix, metricPrefix, runId, 0xff);
    try {
      aggregatesTable.deleteRange(startRow, endRow, tags == null ? null : Bytes.toByteArrays(tags),
                                  getFilter(contextPrefix, metricPrefix, runId));
    } catch (Exception e) {
      throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
    }
  }

View Full Code Here

   */
  public void clear() throws OperationException {
    try {
      aggregatesTable.deleteAll(new byte[] { });
    } catch (Exception e) {
      throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
    }
  }

View Full Code Here

    }


    try {
      timeSeriesTable.put(table);
    } catch (Exception e) {
      throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
    }
  }

View Full Code Here

  public void delete(String contextPrefix) throws OperationException {
    Preconditions.checkArgument(contextPrefix != null, "null context not allowed for delete");
    try {
      timeSeriesTable.deleteAll(entityCodec.encodeWithoutPadding(MetricsEntityType.CONTEXT, contextPrefix));
    } catch (Exception e) {
      throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
    }
  }

View Full Code Here

          Bytes.concat(contextPair.getSecond(), metricPair.getSecond(), tagPair.getSecond(),
                       FOUR_ONE_BYTES, runIdPair.getSecond()))));


        timeSeriesTable.deleteRange(startRow, endRow, null, filter);
      } catch (Exception e) {
        throw new OperationException(StatusCode.INTERNAL_ERROR, e.getMessage(), e);
      }
    }
  }

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of co.cask.cdap.data2.transaction.stream.AbstractStreamFileConsumer

co.cask.cdap.api.dataset.DatasetAdmin

co.cask.cdap.api.dataset.DatasetDefinition

co.cask.cdap.api.dataset.DatasetSpecification

co.cask.cdap.api.dataset.module.DatasetModule

co.cask.cdap.api.dataset.table.ConflictDetection

co.cask.cdap.api.dataset.table.Result

co.cask.cdap.api.dataset.table.Row

co.cask.cdap.api.dataset.table.Scanner

co.cask.cdap.api.security.ACL

co.cask.cdap.common.hooks.MetricsReporterHook

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.