Examples of HRegion

org.apache.hadoop.hbase.HRegion
HRegion stores data for a certain region of a table. It stores all columns for each row. A given table consists of one or more HRegions.
We maintain multiple HStores for a single HRegion.
An HStore is a set of rows with some column data; together, they make up all the data for the rows.
Each HRegion has a 'startKey' and 'endKey'.
The first is inclusive, the second is exclusive (except for the final region) The endKey of region 0 is the same as startKey for region 1 (if it exists). The startKey for the first region is null. The endKey for the final region is null.
The HStores have no locking built-in. All row-level locking and row-level atomicity is provided by the HRegion.
An HRegion is defined by its table and its key extent.
It consists of at least one HStore. The number of HStores should be configurable, so that data which is accessed together is stored in the same HStore. Right now, we approximate that by building a single HStore for each column family. (This config info will be communicated via the tabledesc.) The HTableDescriptor contains metainfo about the HRegion's table. regionName is a unique identifier for this HRegion. (startKey, endKey] defines the keyspace for this HRegion.
org.apache.hadoop.hbase.regionserver.HRegion
HRegion stores data for a certain region of a table. It stores all columns for each row. A given table consists of one or more HRegions.
We maintain multiple HStores for a single HRegion.
An Store is a set of rows with some column data; together, they make up all the data for the rows.
Each HRegion has a 'startKey' and 'endKey'.
The first is inclusive, the second is exclusive (except for the final region) The endKey of region 0 is the same as startKey for region 1 (if it exists). The startKey for the first region is null. The endKey for the final region is null.
Locking at the HRegion level serves only one purpose: preventing the region from being closed (and consequently split) while other operations are ongoing. Each row level operation obtains both a row lock and a region read lock for the duration of the operation. While a scanner is being constructed, getScanner holds a read lock. If the scanner is successfully constructed, it holds a read lock until it is closed. A close takes out a write lock and consequently will block for ongoing operations and will block new operations from starting while the close is in progress.
An HRegion is defined by its table and its key extent.
It consists of at least one Store. The number of Stores should be configurable, so that data which is accessed together is stored in the same Store. Right now, we approximate that by building a single Store for each column family. (This config info will be communicated via the tabledesc.)
The HTableDescriptor contains metainfo about the HRegion's table. regionName is a unique identifier for this HRegion. (startKey, endKey] defines the keyspace for this HRegion.

Examples of org.apache.hadoop.hbase.regionserver.HRegion

    HColumnDescriptor hcd = (new HColumnDescriptor(CF_NAME)).setMaxVersions(MAX_VERSIONS).
        setDataBlockEncoding(encoding).
        setBlocksize(BLOCK_SIZE).
        setBloomFilterType(BloomType.NONE).
        setCompressTags(compressTags);
    HRegion region = testUtil.createTestRegion(TABLE_NAME, hcd);


    //write the data, but leave some in the memstore
    doPuts(region);


    //verify correctness when memstore contains data
    doGets(region);


    //verify correctness again after compacting
    region.compactStores();
    doGets(region);




    Map<DataBlockEncoding, Integer> encodingCounts = cache.getEncodingCountsForTest();

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

        // from special EMPTY_START_ROW to next region's startKey
        HRegionInfo newRegion = new HRegionInfo(htd.getName(),
            HConstants.EMPTY_START_ROW, next.getStartKey());


        // TODO test
        HRegion region = HBaseFsckRepair.createHDFSRegionDir(conf, newRegion, htd);
        LOG.info("Table region start key was not empty.  Created new empty region: "
            + newRegion + " " +region);
        fixes++;
      }

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

        HTableDescriptor htd = getTableInfo().getHTD();
        // from curEndKey to EMPTY_START_ROW
        HRegionInfo newRegion = new HRegionInfo(htd.getName(), curEndKey,
            HConstants.EMPTY_START_ROW);


        HRegion region = HBaseFsckRepair.createHDFSRegionDir(conf, newRegion, htd);
        LOG.info("Table region end key was not empty. Created new empty region: " + newRegion
            + " " + region);
        fixes++;
      }

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

                + Bytes.toStringBinary(holeStopKey)
                + ".  Creating a new regioninfo and region "
                + "dir in hdfs to plug the hole.");
        HTableDescriptor htd = getTableInfo().getHTD();
        HRegionInfo newRegion = new HRegionInfo(htd.getName(), holeStartKey, holeStopKey);
        HRegion region = HBaseFsckRepair.createHDFSRegionDir(conf, newRegion, htd);
        LOG.info("Plugged hold by creating new empty region: "+ newRegion + " " +region);
        fixes++;
      }

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

        // create new empty container region.
        HTableDescriptor htd = getTableInfo().getHTD();
        // from start key to end Key
        HRegionInfo newRegion = new HRegionInfo(htd.getName(), range.getFirst(),
            range.getSecond());
        HRegion region = HBaseFsckRepair.createHDFSRegionDir(conf, newRegion, htd);
        LOG.info("Created new empty container region: " +
            newRegion + " to contain regions: " + Joiner.on(",").join(overlap));
        debugLsr(region.getRegionDir());


        // all target regions are closed, should be able to safely cleanup.
        boolean didFix= false;
        Path target = region.getRegionDir();
        for (HbckInfo contained : overlap) {
          LOG.info("Merging " + contained  + " into " + target );
          int merges = mergeRegionDirs(target, contained);
          if (merges > 0) {
            didFix = true;

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

        Bytes.toString(orphanRegionRange.getSecond()) + ")");


    // create new region on hdfs.  move data into place.
    HRegionInfo hri = new HRegionInfo(template.getName(), orphanRegionRange.getFirst(), orphanRegionRange.getSecond());
    LOG.info("Creating new region : " + hri);
    HRegion region = HBaseFsckRepair.createHDFSRegionDir(conf, hri, template);
    Path target = region.getRegionDir();


    // rename all the data to new region
    mergeRegionDirs(target, hi);
    fixes++;
  }

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

    Configuration c = conf;
    HRegionInfo rootHRI = new HRegionInfo(HRegionInfo.ROOT_REGIONINFO);
    MasterFileSystem.setInfoFamilyCachingForRoot(false);
    HRegionInfo metaHRI = new HRegionInfo(HRegionInfo.FIRST_META_REGIONINFO);
    MasterFileSystem.setInfoFamilyCachingForMeta(false);
    HRegion root = HRegion.createHRegion(rootHRI, rootdir, c,
        HTableDescriptor.ROOT_TABLEDESC);
    HRegion meta = HRegion.createHRegion(metaHRI, rootdir, c,
        HTableDescriptor.META_TABLEDESC);
    MasterFileSystem.setInfoFamilyCachingForRoot(true);
    MasterFileSystem.setInfoFamilyCachingForMeta(true);


    // Add first region from the META table to the ROOT region.

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

    // we can rebuild, move old root and meta out of the way and start
    LOG.info("HDFS regioninfo's seems good.  Sidelining old .META.");
    Path backupDir = sidelineOldRootAndMeta();


    LOG.info("Creating new .META.");
    HRegion meta = createNewRootAndMeta();


    // populate meta
    List<Put> puts = generatePuts(tablesInfo);
    if (puts == null) {
      LOG.fatal("Problem encountered when creating new .META. entries.  " +
        "You may need to restore the previously sidelined -ROOT- and .META.");
      return false;
    }
    meta.put(puts.toArray(new Put[0]));
    meta.close();
    meta.getLog().closeAndDelete();
    LOG.info("Success! .META. table rebuilt.");
    LOG.info("Old -ROOT- and .META. are moved into " + backupDir);
    return true;
  }

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

          }
          super.doWrite(info, logKey, logEdit, htd);
        };
      };
      hlog.rollWriter();
      HRegion region = null;
      try {
        region = openRegion(fs, rootRegionDir, htd, hlog);
        long putTime = runBenchmark(new HLogPutBenchmark(region, htd, numIterations, noSync), numThreads);
        logBenchmarkResult("Summary: threads=" + numThreads + ", iterations=" + numIterations,
          numIterations * numThreads, putTime);

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion

    // Mock an HRegion
    byte [] tableName = Bytes.toBytes("table");
    byte [][] familyNames = new byte [][] { FAMILY };
    HTableDescriptor htd = new HTableDescriptor();
    htd.addFamily(new HColumnDescriptor(Bytes.toBytes("f1")));
    HRegion region = mockRegion(tableName, familyNames, regionRootDir);
    HLog hlog = region.getLog();


    // Spin up N threads to each perform M log operations
    LogWriter [] incrementors = new LogWriter[numThreads];
    for (int i=0; i<numThreads; i++) {
      incrementors[i] = new LogWriter(region, tableName, hlog, i,

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.