Examples of org.apache.hadoop.hbase.client.TableSnapshotScanner

org.apache.hadoop.hbase.client.TableSnapshotScanner
A Scanner which performs a scan over snapshot files. Using this class requires copying the snapshot to a temporary empty directory, which will copy the snapshot reference files into that directory. Actual data files are not copied.
This also allows one to run the scan from an online or offline hbase cluster. The snapshot files can be exported by using the {@link ExportSnapshot} tool, to a pure-hdfs cluster, and this scanner can be used torun the scan directly over the snapshot files. The snapshot should not be deleted while there are open scanners reading from snapshot files.
An internal RegionScanner is used to execute the {@link Scan} obtainedfrom the user for each region in the snapshot.
HBase owns all the data and snapshot files on the filesystem. Only the HBase user can read from snapshot files and data files. HBase also enforces security because all the requests are handled by the server layer, and the user cannot read from the data files directly. To read from snapshot files directly from the file system, the user who is running the MR job must have sufficient permissions to access snapshot and reference files. This means that to run mapreduce over snapshot files, the job has to be run as the HBase user or the user must have group or other priviledges in the filesystem (See HBASE-8369). Note that, given other users access to read from snapshot/data files will completely circumvent the access control enforced by HBase. @see TableSnapshotInputFormat

    restoreDir.getFileSystem(conf).delete(restoreDir, true);
    snapshotRestoreTimer.stop();


    Scan scan = getScan();
    scanOpenTimer.start();
    TableSnapshotScanner scanner = new TableSnapshotScanner(conf, restoreDir, snapshotName, scan);
    scanOpenTimer.stop();


    long numRows = 0;
    long numCells = 0;
    scanTimer.start();
    while (true) {
      Result result = scanner.next();
      if (result == null) {
        break;
      }
      numRows++;


      numCells += result.rawCells().length;
    }
    scanTimer.stop();
    scanner.close();


    ScanMetrics metrics = scanner.getScanMetrics();
    long totalBytes = metrics.countOfBytesInResults.get();
    double throughput = (double)totalBytes / scanTimer.elapsedTime(TimeUnit.SECONDS);
    double throughputRows = (double)numRows / scanTimer.elapsedTime(TimeUnit.SECONDS);
    double throughputCells = (double)numCells / scanTimer.elapsedTime(TimeUnit.SECONDS);

View Full Code Here

    restoreDir.getFileSystem(conf).delete(restoreDir, true);
    snapshotRestoreTimer.stop();


    Scan scan = getScan();
    scanOpenTimer.start();
    TableSnapshotScanner scanner = new TableSnapshotScanner(conf, restoreDir, snapshotName, scan);
    scanOpenTimer.stop();


    long numRows = 0;
    long numCells = 0;
    scanTimer.start();
    while (true) {
      Result result = scanner.next();
      if (result == null) {
        break;
      }
      numRows++;


      numCells += result.rawCells().length;
    }
    scanTimer.stop();
    scanner.close();


    ScanMetrics metrics = scanner.getScanMetrics();
    long totalBytes = metrics.countOfBytesInResults.get();
    double throughput = (double)totalBytes / scanTimer.elapsedTime(TimeUnit.SECONDS);
    double throughputRows = (double)numRows / scanTimer.elapsedTime(TimeUnit.SECONDS);
    double throughputCells = (double)numCells / scanTimer.elapsedTime(TimeUnit.SECONDS);

View Full Code Here

TOP

Related Classes of org.apache.hadoop.hbase.client.TableSnapshotScanner

org.apache.hadoop.fs.Path

org.apache.hadoop.hbase.HRegionInfo

org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription

org.apache.hadoop.hbase.ScanPerformanceEvaluation

org.apache.hadoop.hbase.snapshot.SnapshotManifest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.