Examples of org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper

Package org.apache.hadoop.hbase.snapshot

Examples of org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper

org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper
Helper to Restore/Clone a Snapshot
The helper assumes that a table is already created, and by calling restore() the content present in the snapshot will be restored as the new content of the table.
Clone from Snapshot: If the target table is empty, the restore operation is just a "clone operation", where the only operations are:
- for each region in the snapshot create a new region (note that the region will have a different name, since the encoding contains the table name)
- for each file in the region create a new HFileLink to point to the original file.
- restore the logs, if any
Restore from Snapshot:
- for each region in the table verify which are available in the snapshot and which are not
  - if the region is not present in the snapshot, remove it.
  - if the region is present in the snapshot
    - for each file in the table region verify which are available in the snapshot
      
      if the hfile is not present in the snapshot, remove it
      if the hfile is present, keep it (nothing to do)
    - for each file in the snapshot region but not in the table
      
      create a new HFileLink that point to the original file
- for each region in the snapshot not present in the current table state
  - create a new region and for each file in the region create a new HFileLink (This is the same as the clone operation)
- restore the logs, if any

    Path tableDir = new Path(tableRootDir, tableName);


    try {
      // 1. Execute the on-disk Clone
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(conf, fs,
          snapshot, snapshotDir, hTableDescriptor, tableDir, monitor);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // Clone operation should not have stuff to restore or remove
      Preconditions.checkArgument(!metaChanges.hasRegionsToRestore(),
          "A clone should not have regions to restore");
      Preconditions.checkArgument(!metaChanges.hasRegionsToRemove(),

View Full Code Here

      this.masterServices.getTableDescriptors().add(hTableDescriptor);


      // 2. Execute the on-disk Restore
      LOG.debug("Starting restore snapshot=" + SnapshotDescriptionUtils.toString(snapshot));
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(
          masterServices.getConfiguration(), fs,
          snapshot, snapshotDir, hTableDescriptor, tableDir, monitor);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // 3. Applies changes to .META.
      hris.clear();
      if (metaChanges.hasRegionsToAdd()) hris.addAll(metaChanges.getRegionsToAdd());
      if (metaChanges.hasRegionsToRestore()) hris.addAll(metaChanges.getRegionsToRestore());

View Full Code Here


    MonitoredTask status = TaskMonitor.get().createStatus(
        "Restoring  snapshot '" + snapshotName + "' to directory " + tableDir);
    ForeignExceptionDispatcher monitor = new ForeignExceptionDispatcher();


    RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
        snapshotDesc, snapshotDir, htd, tableDir, monitor, status);
    helper.restoreHdfsRegions();
  }

View Full Code Here

      this.masterServices.getTableDescriptors().add(hTableDescriptor);


      // 2. Execute the on-disk Restore
      LOG.debug("Starting restore snapshot=" + ClientSnapshotDescriptionUtils.toString(snapshot));
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(
          masterServices.getConfiguration(), fs,
          snapshot, snapshotDir, hTableDescriptor, rootDir, monitor, status);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // 3. Forces all the RegionStates to be offline
      //
      // The AssignmentManager keeps all the region states around
      // with no possibility to remove them, until the master is restarted.

View Full Code Here

      this.masterServices.getTableDescriptors().add(hTableDescriptor);


      // 2. Execute the on-disk Restore
      LOG.debug("Starting restore snapshot=" + SnapshotDescriptionUtils.toString(snapshot));
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(
          masterServices.getConfiguration(), fs,
          snapshot, snapshotDir, hTableDescriptor, tableDir, monitor, status);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // 3. Forces all the RegionStates to be offline
      //
      // The AssignmentManager keeps all the region states around
      // with no possibility to remove them, until the master is restarted.

View Full Code Here

    Path tableDir = new Path(tableRootDir, tableName);


    try {
      // 1. Execute the on-disk Clone
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(conf, fs,
          snapshot, snapshotDir, hTableDescriptor, tableDir, monitor, status);
      metaChanges = restoreHelper.restoreHdfsRegions();


      // Clone operation should not have stuff to restore or remove
      Preconditions.checkArgument(!metaChanges.hasRegionsToRestore(),
          "A clone should not have regions to restore");
      Preconditions.checkArgument(!metaChanges.hasRegionsToRemove(),

View Full Code Here

    Path rootDir = fileSystemManager.getRootDir();


    try {
      // 1. Execute the on-disk Clone
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(conf, fs,
          snapshot, snapshotDir, hTableDescriptor, tableRootDir, monitor, status);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // Clone operation should not have stuff to restore or remove
      Preconditions.checkArgument(!metaChanges.hasRegionsToRestore(),
          "A clone should not have regions to restore");
      Preconditions.checkArgument(!metaChanges.hasRegionsToRemove(),

View Full Code Here

      this.masterServices.getTableDescriptors().add(hTableDescriptor);


      // 2. Execute the on-disk Restore
      LOG.debug("Starting restore snapshot=" + ClientSnapshotDescriptionUtils.toString(snapshot));
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(
          masterServices.getConfiguration(), fs,
          snapshot, snapshotDir, hTableDescriptor, rootDir, monitor, status);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // 3. Applies changes to .META.
      hris.clear();
      status.setStatus("Preparing to restore each region");
      if (metaChanges.hasRegionsToAdd()) hris.addAll(metaChanges.getRegionsToAdd());

View Full Code Here

    Path tableDir = new Path(tableRootDir, tableName);


    try {
      // 1. Execute the on-disk Clone
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(conf, fs,
          snapshot, snapshotDir, hTableDescriptor, tableDir, monitor);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // Clone operation should not have stuff to restore or remove
      Preconditions.checkArgument(!metaChanges.hasRegionsToRestore(),
          "A clone should not have regions to restore");
      Preconditions.checkArgument(!metaChanges.hasRegionsToRemove(),

View Full Code Here

      this.masterServices.getTableDescriptors().add(hTableDescriptor);


      // 2. Execute the on-disk Restore
      LOG.debug("Starting restore snapshot=" + ClientSnapshotDescriptionUtils.toString(snapshot));
      Path snapshotDir = SnapshotDescriptionUtils.getCompletedSnapshotDir(snapshot, rootDir);
      RestoreSnapshotHelper restoreHelper = new RestoreSnapshotHelper(
          masterServices.getConfiguration(), fs,
          snapshot, snapshotDir, hTableDescriptor, tableDir, monitor);
      RestoreSnapshotHelper.RestoreMetaChanges metaChanges = restoreHelper.restoreHdfsRegions();


      // 3. Applies changes to .META.
      hris.clear();
      if (metaChanges.hasRegionsToAdd()) hris.addAll(metaChanges.getRegionsToAdd());
      if (metaChanges.hasRegionsToRestore()) hris.addAll(metaChanges.getRegionsToRestore());

View Full Code Here

0 1 2

TOP

Related Classes of org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper

org.apache.hadoop.fs.Path

org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher

org.apache.hadoop.hbase.HRegionInfo

org.apache.hadoop.hbase.HTableDescriptor

org.apache.hadoop.hbase.io.HFileLink

org.apache.hadoop.hbase.io.Reference

org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat

org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler

org.apache.hadoop.hbase.master.snapshot.RestoreSnapshotHandler

org.apache.hadoop.hbase.monitoring.MonitoredTask

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.