Examples of com.salesforce.hbase.index.covered.data.IndexMemStore$MemStoreScanner

• com.salesforce.hbase.index.covered.data.IndexMemStore
  Like the HBase {@link MemStore}, but without all that extra work around maintaining snapshots and sizing (for right now). We still support the concurrent access (in case indexes are built in parallel).

  We basically wrap a KeyValueSkipListSet, just like a regular MemStore, except we are:

  1. not dealing with
     • space considerations
     • a snapshot set
  2. ignoring memstore timestamps in favor of deciding when we want to overwrite keys based on how we obtain them
  3. ignoring time range updates (so {@link KeyValueScanner#shouldUseScanner(Scan,SortedSet,long)} isn't supported from {@link #getScanner()}).

  We can ignore the memstore timestamps because we know that anything we get from the local region is going to be MVCC visible - so it should just go in. However, we also want overwrite any existing state with our pending write that we are indexing, so that needs to clobber the KVs we get from the HRegion. This got really messy with a regular memstore as each KV from the MemStore frequently has a higher MemStoreTS, but we can't just up the pending KVs' MemStoreTs because a memstore relies on the MVCC readpoint, which generally is less than {@link Long#MAX_VALUE}.

  By realizing that we don't need the snapshot or space requirements, we can go much faster than the previous implementation. Further, by being smart about how we manage the KVs, we can drop the extra object creation we were doing to wrap the pending KVs (which we did previously to ensure they sorted before the ones we got from the HRegion). We overwrite {@link KeyValue}s when we add them from external sources {@link #add(KeyValue,boolean)}, but then don't overwrite existing keyvalues when read them from the underlying table (because pending keyvalues should always overwrite current ones) - this logic is all contained in LocalTableState. @see LocalTableState

Related Classes of com.salesforce.hbase.index.covered.data.IndexMemStore$MemStoreScanner