Examples of org.hbase.async.Scanner

org.hbase.async.Scanner
Creates a scanner to read data sequentially from HBase.
This class is not synchronized as it's expected to be used from a single thread at a time. It's rarely (if ever?) useful to scan concurrently from a shared scanner using multiple threads. If you want to optimize large table scans using extra parallelism, create a few scanners and give each of them a partition of the table to scan. Or use MapReduce.
Unlike HBase's traditional client, there's no method in this class to explicitly open the scanner. It will open itself automatically when you start scanning by calling {@link #nextRows()}. Also, the scanner will automatically call {@link #close} when it reaches the end key. If, however,you would like to stop scanning before reaching the end key, you must call {@link #close} before disposing of the scanner. Note thatit's always safe to call {@link #close} on a scanner.
If you keep your scanner open and idle for too long, the RegionServer will close the scanner automatically for you after a timeout configured on the server side. When this happens, you'll get an {@link UnknownScannerException} when you attempt to use the scanner again.Also, if you scan too slowly (e.g. you take a long time between each call to {@link #nextRows()}), you may prevent HBase from splitting the region if the region is also actively being written to while you scan. For heavy processing you should consider using MapReduce.
A {@code Scanner} is not re-usable. Should you want to scan the same rowsor the same table again, you must create a new one.
A note on passing {@code byte} arrays in argument
None of the method that receive a {@code byte[]} in argument will copy it.For more info, please refer to the documentation of {@link HBaseRpc}.
A note on passing {@code String}s in argument
All strings are assumed to use the platform's default charset.

    Deferred.group(client.put(put1), client.put(put2),
                   client.put(put3)).join();
    // Scan the same 3 rows created above twice.
    for (int i = 0; i < 2; i++) {
      LOG.info("------------ iteration #" + i);
      final Scanner scanner = client.newScanner(table);
      scanner.setStartKey("s0");
      scanner.setStopKey("s9");
      // Callback class to keep scanning recursively.
      class cb implements Callback<Object, ArrayList<ArrayList<KeyValue>>> {
        private int n = 0;
        public Object call(final ArrayList<ArrayList<KeyValue>> rows) {
          if (rows == null) {
            return null;
          }
          n++;
          try {
            assertSizeIs(1, rows);
            final ArrayList<KeyValue> kvs = rows.get(0);
            final KeyValue kv = kvs.get(0);
            assertSizeIs(1, kvs);
            assertEq("s" + n, kv.key());
            assertEq("q", kv.qualifier());
            assertEq("v" + n, kv.value());
            return scanner.nextRows(1).addCallback(this);
          } catch (AssertionError e) {
            // Deferred doesn't catch Errors on purpose, so transform any
            // assertion failure into an Exception.
            throw new RuntimeException("Asynchronous failure", e);
          }
        }
      }
      try {
        scanner.nextRows(1).addCallback(new cb()).join();
      } finally {
        scanner.close().join();
      }
    }
  }

View Full Code Here

    final PutRequest put1 = new PutRequest(table, "k", family, "a", "val1");
    final PutRequest put2 = new PutRequest(table, "k", family, "b", "val2");
    final PutRequest put3 = new PutRequest(table, "k", family, "c", "val3");
    Deferred.group(client.put(put1), client.put(put2),
                   client.put(put3)).join();
    final Scanner scanner = client.newScanner(table);
    scanner.setFamily(family);
    scanner.setQualifiers(new byte[][] { { 'a' }, { 'c' } });
    final ArrayList<ArrayList<KeyValue>> rows = scanner.nextRows(2).join();
    assertSizeIs(1, rows);
    final ArrayList<KeyValue> kvs = rows.get(0);
    assertSizeIs(2, kvs);
    assertEq("val1", kvs.get(0).value());
    assertEq("val3", kvs.get(1).value());

View Full Code Here

                                           family, "q", "krfv2");
    final PutRequest put3 = new PutRequest(table, "krf this is Accepted too",
                                           family, "q", "krfv3");
    Deferred.group(client.put(put1), client.put(put2),
                   client.put(put3)).join();
    final Scanner scanner = client.newScanner(table);
    scanner.setFamily(family);
    scanner.setStartKey("krf ");
    scanner.setStopKey("krf!");
    scanner.setKeyRegexp("[Aa]ccept(ed)?");
    final ArrayList<ArrayList<KeyValue>> rows = scanner.nextRows().join();
    assertSizeIs(2, rows);
    ArrayList<KeyValue> kvs = rows.get(0);
    assertSizeIs(1, kvs);
    assertEq("krfv1", kvs.get(0).value());
    kvs = rows.get(1);

View Full Code Here

    final PutRequest put2 = new PutRequest(table, "cpf1", family, "qa2", "v2");
    final PutRequest put3 = new PutRequest(table, "cpf2", family, "qa3", "v3");
    final PutRequest put4 = new PutRequest(table, "cpf2", family, "qb4", "v4");
    Deferred.group(Deferred.group(client.put(put1), client.put(put2)),
                   Deferred.group(client.put(put3), client.put(put4))).join();
    final Scanner scanner = client.newScanner(table);
    scanner.setFamily(family);
    scanner.setStartKey("cpf1");
    scanner.setStopKey("cpf3");
    scanner.setFilter(new ColumnPrefixFilter("qa"));
    final ArrayList<ArrayList<KeyValue>> rows = scanner.nextRows().join();
    assertSizeIs(2, rows);
    ArrayList<KeyValue> kvs = rows.get(0);
    assertSizeIs(2, kvs);
    assertEq("v1", kvs.get(0).value());
    assertEq("v2", kvs.get(1).value());

View Full Code Here

    final PutRequest put2 = new PutRequest(table, "crf1", family, "qb2", "v2");
    final PutRequest put3 = new PutRequest(table, "crf2", family, "qc3", "v3");
    final PutRequest put4 = new PutRequest(table, "crf2", family, "qd4", "v4");
    Deferred.group(Deferred.group(client.put(put1), client.put(put2)),
                   Deferred.group(client.put(put3), client.put(put4))).join();
    final Scanner scanner = client.newScanner(table);
    scanner.setFamily(family);
    scanner.setStartKey("crf1");
    scanner.setStopKey("crf3");
    scanner.setFilter(new ColumnRangeFilter("qb", true, "qd4", false));
    final ArrayList<ArrayList<KeyValue>> rows = scanner.nextRows().join();
    assertSizeIs(2, rows);  // One KV from row "fl1" and one from "fl2".
    ArrayList<KeyValue> kvs = rows.get(0);
    assertSizeIs(1, kvs);
    assertEq("v2", kvs.get(0).value());
    kvs = rows.get(1);

View Full Code Here

    final PutRequest put1 = new PutRequest(table, "fc1", family, "a", "v1");
    final PutRequest put2 = new PutRequest(table, "fc2", family, "a", "v2");
    final PutRequest put3 = new PutRequest(table, "fc3", family, "a", "v3");
    Deferred.group(client.put(put1), client.put(put2), client.put(put3)).join();


    final Scanner binary_scanner = client.newScanner(table);
    binary_scanner.setStartKey("fc1");
    binary_scanner.setStopKey("fc4");
    binary_scanner.setFilter(
        new RowFilter(CompareOp.LESS, new BinaryComparator(Bytes.UTF8("fc2"))));
    final ArrayList<ArrayList<KeyValue>> binary_rows =
        binary_scanner.nextRows().join();
    assertSizeIs(1, binary_rows);
    assertSizeIs(1, binary_rows.get(0));
    assertEq("v1", binary_rows.get(0).get(0).value());


    final Scanner prefix_scanner = client.newScanner(table);
    prefix_scanner.setStartKey("fc1");
    prefix_scanner.setStopKey("fc4");
    prefix_scanner.setFilter(
        new RowFilter(CompareOp.GREATER_OR_EQUAL,
            new BinaryPrefixComparator(Bytes.UTF8("fc2"))));
    final ArrayList<ArrayList<KeyValue>> prefix_rows =
        prefix_scanner.nextRows().join();
    assertSizeIs(2, prefix_rows);
    assertSizeIs(1, prefix_rows.get(0));
    assertEq("v2", prefix_rows.get(0).get(0).value());
    assertSizeIs(1, prefix_rows.get(1));
    assertEq("v3", prefix_rows.get(1).get(0).value());


    final Scanner bit_scanner = client.newScanner(table);
    bit_scanner.setStartKey("fc1");
    bit_scanner.setStopKey("fc4");
    bit_scanner.setFilter(
        new RowFilter(CompareOp.EQUAL,
            new BitComparator(Bytes.UTF8("fc2"), BitComparator.BitwiseOp.XOR)));
    final ArrayList<ArrayList<KeyValue>> bit_rows =
        bit_scanner.nextRows().join();
    assertSizeIs(2, bit_rows);
    assertSizeIs(1, bit_rows.get(0));
    assertEq("v1", bit_rows.get(0).get(0).value());
    assertSizeIs(1, bit_rows.get(1));
    assertEq("v3", bit_rows.get(1).get(0).value());


    final Scanner regex_scanner = client.newScanner(table);
    regex_scanner.setStartKey("fc1");
    regex_scanner.setStopKey("fc4");
    regex_scanner.setFilter(
        new RowFilter(CompareOp.EQUAL, new RegexStringComparator("fc2")));
    final ArrayList<ArrayList<KeyValue>> regex_rows =
        regex_scanner.nextRows().join();
    assertSizeIs(1, regex_rows);
    assertSizeIs(1, regex_rows.get(0));
    assertEq("v2", regex_rows.get(0).get(0).value());


    final Scanner substring_scanner = client.newScanner(table);
    substring_scanner.setStartKey("fc1");
    substring_scanner.setStopKey("fc4");
    substring_scanner.setFilter(
        new RowFilter(CompareOp.EQUAL, new SubstringComparator("2")));
    final ArrayList<ArrayList<KeyValue>> substring_rows =
        substring_scanner.nextRows().join();
    assertSizeIs(1, substring_rows);
    assertSizeIs(1, substring_rows.get(0));
    assertEq("v2", substring_rows.get(0).get(0).value());
  }

View Full Code Here

        42);
    Deferred.group(
        Deferred.group(client.put(put1), client.put(put2)),
        Deferred.group(client.put(put3), client.put(put4))).join();


    final Scanner row_scanner = client.newScanner(table);
    row_scanner.setStartKey("cf1");
    row_scanner.setStopKey("cf4");
    row_scanner.setFilter(
        new RowFilter(CompareOp.NOT_EQUAL, new BinaryComparator(Bytes.UTF8("cf2"))));
    final ArrayList<ArrayList<KeyValue>> row_rows =
        row_scanner.nextRows().join();
    assertSizeIs(2, row_rows);
    assertSizeIs(1, row_rows.get(0));
    assertEq("v1", row_rows.get(0).get(0).value());
    assertSizeIs(2, row_rows.get(1));
    assertEq("v3", row_rows.get(1).get(0).value());
    assertEq("v4", row_rows.get(1).get(1).value());


    final Scanner family_scanner = client.newScanner(table);
    family_scanner.setFilter(
        new FamilyFilter(CompareOp.LESS_OR_EQUAL,
            new BinaryComparator(Bytes.UTF8("aSomeOtherFamily"))));
    final ArrayList<ArrayList<KeyValue>> family_rows =
        family_scanner.nextRows().join();
    assertNull(family_rows);


    final Scanner qualifier_scanner = client.newScanner(table);
    qualifier_scanner.setStartKey("cf1");
    qualifier_scanner.setStopKey("cf4");
    qualifier_scanner.setFilter(
        new QualifierFilter(CompareOp.GREATER,
            new BinaryComparator(Bytes.UTF8("b"))));
    final ArrayList<ArrayList<KeyValue>> qualifier_rows =
        qualifier_scanner.nextRows().join();
    assertSizeIs(1, qualifier_rows);
    assertSizeIs(2, qualifier_rows.get(0));
    assertEq("v3", qualifier_rows.get(0).get(0).value());
    assertEq("v4", qualifier_rows.get(0).get(1).value());


    final Scanner value_scanner = client.newScanner(table);
    value_scanner.setStartKey("cf1");
    value_scanner.setStopKey("cf4");
    value_scanner.setFilter(
        new ValueFilter(CompareOp.GREATER_OR_EQUAL,
            new BinaryComparator(Bytes.UTF8("v3"))));
    final ArrayList<ArrayList<KeyValue>> value_rows =
        value_scanner.nextRows().join();
    assertSizeIs(1, value_rows);
    assertSizeIs(2, value_rows.get(0));
    assertEq("v3", value_rows.get(0).get(0).value());
    assertEq("v4", value_rows.get(0).get(1).value());


    final Scanner dependent_scanner = client.newScanner(table);
    dependent_scanner.setMaxNumKeyValues(-1);
    dependent_scanner.setFilter(
        new DependentColumnFilter(Bytes.UTF8(family), Bytes.UTF8("dep")));
    final ArrayList<ArrayList<KeyValue>> dependent_rows =
        dependent_scanner.nextRows().join();
    assertSizeIs(1, dependent_rows);
    assertSizeIs(2, dependent_rows.get(0));
    assertEq("v3", dependent_rows.get(0).get(0).value());
    assertEq("v4", dependent_rows.get(0).get(1).value());


    final Scanner dependent_value_scanner = client.newScanner(table);
    dependent_value_scanner.setMaxNumKeyValues(-1);
    dependent_value_scanner.setFilter(
        new DependentColumnFilter(
            Bytes.UTF8(family),
            Bytes.UTF8("dep"),
            true,
            CompareOp.EQUAL,
            new BinaryComparator(Bytes.UTF8("v4"))));
    final ArrayList<ArrayList<KeyValue>> dependent_value_rows =
        dependent_value_scanner.nextRows().join();
    assertSizeIs(1, dependent_value_rows);
    assertSizeIs(1, dependent_value_rows.get(0));
    assertEq("v3", dependent_value_rows.get(0).get(0).value());
  }

View Full Code Here

    final PutRequest put6 = new PutRequest(table, "fl2", family, "qd6", "v6");
    Deferred.group(Deferred.group(client.put(put1), client.put(put2),
                                  client.put(put3)),
                   Deferred.group(client.put(put4), client.put(put5),
                                  client.put(put6))).join();
    final Scanner scanner = client.newScanner(table);
    scanner.setFamily(family);
    scanner.setStartKey("fl0");
    scanner.setStopKey("fl9");
    scanner.setFilter(new FilterList(filters));
    final ArrayList<ArrayList<KeyValue>> rows = scanner.nextRows().join();
    assertSizeIs(2, rows);  // One KV from row "fl1" and one from "fl2".
    ArrayList<KeyValue> kvs = rows.get(0);
    assertSizeIs(1, kvs);   // KV from "fl1":
    assertEq("v2", kvs.get(0).value());
    kvs = rows.get(1);

View Full Code Here

    final PutRequest put2 =
      new PutRequest(tableBytes, Bytes.UTF8("tf2"), familyBytes, qualifier, Bytes.UTF8("v2"), 2L);
    final PutRequest put3 =
      new PutRequest(tableBytes, Bytes.UTF8("tf3"), familyBytes, qualifier, Bytes.UTF8("v3"), 3L);
    Deferred.group(client.put(put1), client.put(put2), client.put(put3)).join();
    final Scanner scanner = client.newScanner(table);
    scanner.setFamily(family);
    scanner.setStartKey(Bytes.UTF8("tf"));
    scanner.setStopKey(Bytes.UTF8("tf4"));
    scanner.setFilter(new TimestampsFilter(1L, 3L));
    final ArrayList<ArrayList<KeyValue>> rows = scanner.nextRows().join();
    assertSizeIs(2, rows);
    assertSizeIs(1, rows.get(0));
    assertEq("v1", rows.get(0).get(0).value());
    assertSizeIs(1, rows.get(1));
    assertEq("v3", rows.get(1).get(0).value());

View Full Code Here

    LOG.info("Increments coalesced: " + stats.hitCount());
    LOG.info("Increments sent to HBase: " + stats.missCount());
    LOG.info("  due to cache evictions: " + stats.evictionCount());


    LOG.info("Reading all counters back from HBase and checking values...");
    final Scanner scanner = client.newScanner(table);
    scanner.setStartKey(key(0));
    scanner.setStopKey(key(NUM_ROWS));
    scanner.setFamily(family);
    scanner.setQualifier(QUALIFIER);
    ArrayList<ArrayList<KeyValue>> rows;
    final long expected = nthreads * ICV_PER_ROW;
    while ((rows = scanner.nextRows().join()) != null) {
      for (final ArrayList<KeyValue> row : rows) {
        final long value = Bytes.getLong(row.get(0).value());
        if (value != expected) {
          LOG.error("Invalid count in " + row.get(0) + ": " + value);
          failed = true;

View Full Code Here

0 1 2

TOP

Related Classes of org.hbase.async.Scanner

HBaseIA.TwitBase.AsyncUsersTool

net.opentsdb.core.TsdbQuery

net.opentsdb.tools.DumpSeries

net.opentsdb.tools.Fsck

net.opentsdb.tools.MetaPurge

net.opentsdb.tools.MetaSync

net.opentsdb.tools.TreeSync

net.opentsdb.tools.UidManager

net.opentsdb.tree.Branch

net.opentsdb.tree.Tree

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of org.hbase.async.Scanner

A note on passing {@code byte} arrays in argument

A note on passing {@code String}s in argument

Related Classes of org.hbase.async.Scanner