Examples of Checkpoint

cascading.pipe.Checkpoint
The Checkpoint pipe, if supported by the current planner, will force data to be persisted at the point in the tuple stream an instance of Checkpoint is inserted into the pipe assembly.
If a checkpoint {@link cascading.tap.Tap} is added to the {@link cascading.flow.FlowDef} via the{@link cascading.flow.FlowDef#addCheckpoint(Checkpoint,cascading.tap.Tap)} method, that Tap instancewill be used to capture the intermediate result sets.
It is required that any Scheme used as a checkpoint must source {@link cascading.tuple.Fields#UNKNOWN} andsink {@link cascading.tuple.Fields#ALL}.
If used with a {@link cascading.scheme.hadoop.TextDelimited} {@link cascading.scheme.Scheme} class andthe {@code hasHeader} value is {@code true}, a header with the resolved field names will be written to the file.
This is especially useful for debugging complex flows.
For the {@link cascading.flow.hadoop.HadoopFlowConnector} and Hadoop platform, a Checkpoint will force a newMapReduce job ( {@link cascading.flow.hadoop.HadoopFlowStep} into the {@link cascading.flow.Flow} plan.
This can be important when used in conjunction with a {@link HashJoin} where all the operations upstreamfrom the HashJoin significantly filter out data allowing it to fit in memory.
com.linkedin.databus.core.Checkpoint
This class represents the state of a consumer consuming events from a Databus server for a single timeline (physical partition). There are two main types of checkpoints:
- Online consumption - used for consuming events a relay
- Bootstrap - used for consuming events from a bootstrap server
The type of a checkpoint is determined by the consumptionMode parameter {@link #getConsumptionMode()}.
Online-Consumption checkpoints Online-consumption checkpoints {@link DbusClientMode#ONLINE_CONSUMPTION} represent the state ofa consumer of event stream from a relay. The main properties of the checkpoint are:
- {@code consumption_mode} - must be {@link DbusClientMode#ONLINE_CONSUMPTION}
- {@code windowScn} - the sequence number (SCN) of the current window; -1 denotes"flexible" checkpoint (see below). If the SCN is 0, and tsNescs is greater than 0 then the relay may (if capable) stream events that have timestamp greater than or equal to tsNsecs. However, the relay MUST ensure that it does not miss any events that have a timestamp greater than or equal to tsNsecs. TODO: Until we have this capability in the relays we don't have to define the exact behavior
- {@code prevScn} - the sequence number (SCN) of the window before current; -1 means"unknown".
- {@code windowOffset} - the number of events processed from the current window;-1 means the entire window has been processed including the end-of-window event and prevScn must be equal to windowScn.
- {@code tsNsecs} - optional value that is set to the timestamp of the EOP event in the window ofevents with the highest SCN that has been successfully consumed. If tsNsecs is greater than 0 then the value of windowScn must not be -1 (see discussion on flexible checkpoints below).
Flexible online-consumption checkpoints Used by consumers which do not care from where they start consuming. The relay will make the best effort to serve whatever data it has. Flexible checkpoints can be created using {@link #createFlexibleCheckpoint()} or invoking {@link #setFlexible()} on an existing Checkpoint.If a flexible checkpoint has tsNsecs set, the value of tsNsecs must be -1 (unset value).
Bootstrap checkpoints These are common fields for snapshot and catchup bootsrap checkpoints (see below).
- {@code bootstrap_since_scn} - the SCN of the last fully-consumed window or 0 for a full bootstrap. Must be >= 0.
- {@code bootstrap_start_scn} - the last SCN written to the snapshot when the snapshot started; must be setbefore any data is read, i.e. {@code bootstrap_snapshot_source_index} > 0 or{@code snapshot_offset} > 0 or {@code bootstrap_target_scn} != -1
- {@code bootstrap_target_scn} - the last SCN written to the log tables when the snapshot of the last sourcecompleted. It provides an upper bound of how much dirty data might have been read while snapshotting. It specifies the SCN up to which catch-up should be performed to guarantee consistency of the bootstrap results.
- {@code bootstrap_start_tsnsecs}- (optional) the timestamp of the EOP event of the highest window successfully processed by the client before the client fell off the relay. This value is optionally set by the bootstrap client before bootstrapping begins, and is never changed during the entire bootstrap sequence (snapshot and catchup phases).
Bootstrap snapshot checkpoints Bootstrap snapshot checkpoints ( {@link DbusClientMode#BOOTSTRAP_SNAPSHOT}) represent a consumer in the SNAPSHOT phase of bootstrapping. The main properties of checkpoints are:
- {@code consumption_mode} - must be {@link DbusClientMode#BOOTSTRAP_SNAPSHOT}
- {@code bootstrap_snapshot_source_index} - the index of the current source being snapshotted or the indexof the next source if {@code bootstrap_snapshot_offet} == -1
- {@code snapshot_source} - the name of the current source being snapshotted or the name of the next sourceif {@code bootstrap_snapshot_offet} == -1
- {@code bootstrap_snapshot_offset} - number of rows successfully read for the current snapshot source; if -1,all rows have been read
- {@code bootstrap_snapshot_file_record_offset} - Applicable for V3 bootstrap only; refers to the offset of therecord within the AVRO block
- {@code storage_cluster_name} - Applicable for V3 bootstrap only; refers to name of espresso storage cluster
Bootstrap catchup checkpoints Bootstrap catchup checkpoints ( {@link DbusClientMode#BOOTSTRAP_CATCHUP}) represent a consumer in the CATCHUP phase of bootstrapping. The main properties of checkpoints are:
- {@code consumption_mode} - must be {@link DbusClientMode#BOOTSTRAP_CATCHUP}
- {@code bootstrap_snapshot_source_index} - the index of the last snapshot source
- {@code bootstrap_snapshot_offset} - should always be -1
- {@code catchup_source} - the name of the current catch-up source or the name of the next sourceif {@code windowScn} == -1
- {@code windowScn} - the sequence number (SCN) of the current window being caught-up;
- {@code windowOffset} - the number of events processed from the current window;
@see CheckpointMult for multi-partition checkpoints
com.persistit.CheckpointManager.Checkpoint
com.pointcliki.dizgruntled.logic.Checkpoint
A logic to display the level checkpoint flags @author Hugheth @since alpha 2.6
com.sun.grid.jgdi.configuration.Checkpoint
fr.imag.adele.apam.pax.test.performance.util.Checkpoint
org.agilewiki.jfile.transactions.db.Checkpoint
A checkpoint request is sent to a database to write all state to disk.
org.apache.jackrabbit.jcr2spi.operation.Checkpoint
Checkout...
org.archive.checkpointing.Checkpoint
Represents a single checkpoint, by its name and main store directory. @contributor gojomo
org.exist.storage.txn.Checkpoint
@author wolf
org.infinispan.test.fwk.CheckPoint
Behaves more or less like a map of {@link java.util.concurrent.Semaphore}s. One thread will wait for an event via {@code await(...)} or {@code awaitStrict(...)}, and one or more other threads will trigger the event via {@code trigger(...)} or {@code triggerForever(...)}. @author Dan Berindei @since 5.3
org.jpos.transaction.participant.CheckPoint
ptolemy.backtrack.Checkpoint
The class of checkpoint objects. A checkpoint object represents the smallest entity on which checkpoints are created and managed. It monitors one or more objects. When the {@link #rollback(long,boolean)} function is called, allthe monitored objects are rolled back to their previous states, defined by a timestamp. @author Thomas Feng @version $Id: Checkpoint.java,v 1.13.4.1 2008/03/25 22:11:40 cxh Exp $ @since Ptolemy II 5.1 @Pt.ProposedRating Red (tfeng) @Pt.AcceptedRating Red (tfeng)

Examples of com.linkedin.databus.core.Checkpoint

  /** Test bootstrap transition: Target_Scn_Success : Happy Path */
  public void testTransition_TargetScnSuccessHappyPath() throws Exception
  {
    BootstrapPullThread bsPuller = createBootstrapPullThread(false, false, false, false, false, null, 12000, 1, true);


    Checkpoint cp = _ckptHandlerSource1.createInitialBootstrapCheckpoint(null, 0L);
    //TODO remove
    //cp.setSnapshotSource("source1");
    //cp.setCatchupSource("source1");
    //cp.setConsumptionMode(DbusClientMode.BOOTSTRAP_SNAPSHOT);
    cp.setBootstrapServerInfo(_serverInfoName);


    bsPuller.getComponentStatus().start();
    ConnectionState connState = bsPuller.getConnectionState();
    connState.switchToBootstrap(cp);
    testTransitionCase(bsPuller, StateId.BOOTSTRAP, StateId.REQUEST_START_SCN, cp);


    bsPuller.getMessageQueue().clear();
    testTransitionCase(bsPuller, StateId.REQUEST_START_SCN, StateId.START_SCN_RESPONSE_SUCCESS, null);
    bsPuller.getMessageQueue().clear();
    Map<Long, List<RegisterResponseEntry>> entries = new HashMap<Long, List<RegisterResponseEntry>>();


    entries.put(1L, new ArrayList<RegisterResponseEntry>());
    connState.setSourcesSchemas(entries);


    testTransitionCase(bsPuller, StateId.START_SCN_RESPONSE_SUCCESS, StateId.REQUEST_STREAM, null);
    bsPuller.getMessageQueue().clear();


    connState.getSourcesNameMap().put("source1", new IdNamePair(1L, "source1"));
    connState.getSourceIdMap().put(1L, new IdNamePair(1L, "source1"));
    testTransitionCase(bsPuller, StateId.REQUEST_STREAM, StateId.STREAM_REQUEST_SUCCESS, null);


    bsPuller.getMessageQueue().clear();
    testTransitionCase(bsPuller, StateId.STREAM_REQUEST_SUCCESS, StateId.STREAM_RESPONSE_DONE, null);


    bsPuller.getMessageQueue().clear();
    cp.setSnapshotOffset(-1);
    testTransitionCase(bsPuller, StateId.STREAM_RESPONSE_DONE, StateId.REQUEST_TARGET_SCN, null);
    Assert.assertEquals(cp.getConsumptionMode(), DbusClientMode.BOOTSTRAP_SNAPSHOT, "Consumption Mode check");
    Assert.assertFalse(cp.isBootstrapTargetScnSet());


    bsPuller.getMessageQueue().clear();
    testTransitionCase(bsPuller, StateId.REQUEST_TARGET_SCN, StateId.TARGET_SCN_RESPONSE_SUCCESS, null);


    bsPuller.getMessageQueue().clear();
    entries = new HashMap<Long, List<RegisterResponseEntry>>();


    entries.put(1L, new ArrayList<RegisterResponseEntry>());
    connState.setSourcesSchemas(entries);


    testTransitionCase(bsPuller, StateId.TARGET_SCN_RESPONSE_SUCCESS, StateId.REQUEST_STREAM, null);
    Assert.assertEquals(cp.getConsumptionMode(), DbusClientMode.BOOTSTRAP_CATCHUP, "Consumption Mode check");
    Assert.assertTrue(cp.isBootstrapTargetScnSet());
    Assert.assertEquals(cp.getBootstrapTargetScn().longValue(), 10L);
    Assert.assertEquals(cp.getWindowScn(), cp.getBootstrapStartScn().longValue());
    Assert.assertEquals(cp.getWindowOffset().longValue(), 0L);
  }

View Full Code Here

Examples of com.linkedin.databus.core.Checkpoint

  {
    // targetScn = 50 < 100 = startScn
    BootstrapPullThread bsPuller = createBootstrapPullThread(false, false, false, false, false, null, 12000, 1, true,
                                                             100, 50);


    Checkpoint cp = _ckptHandlerSource1.createInitialBootstrapCheckpoint(null, 0L);
    //TODO remove
    //cp.setSnapshotSource("source1");
    //cp.setCatchupSource("source1");
    //cp.setConsumptionMode(DbusClientMode.BOOTSTRAP_SNAPSHOT);
    cp.setBootstrapServerInfo(_serverInfoName);


    bsPuller.getComponentStatus().start();
    ConnectionState connState = bsPuller.getConnectionState();
    connState.switchToBootstrap(cp);
    testTransitionCase(bsPuller, StateId.BOOTSTRAP, StateId.REQUEST_START_SCN, cp);


    bsPuller.getMessageQueue().clear();
    testTransitionCase(bsPuller, StateId.REQUEST_START_SCN, StateId.START_SCN_RESPONSE_SUCCESS, null);
    bsPuller.getMessageQueue().clear();
    Map<Long, List<RegisterResponseEntry>> entries = new HashMap<Long, List<RegisterResponseEntry>>();


    entries.put(1L, new ArrayList<RegisterResponseEntry>());
    connState.setSourcesSchemas(entries);


    testTransitionCase(bsPuller, StateId.START_SCN_RESPONSE_SUCCESS, StateId.REQUEST_STREAM, null);
    bsPuller.getMessageQueue().clear();


    connState.getSourcesNameMap().put("source1", new IdNamePair(1L, "source1"));
    connState.getSourceIdMap().put(1L, new IdNamePair(1L, "source1"));
    testTransitionCase(bsPuller, StateId.REQUEST_STREAM, StateId.STREAM_REQUEST_SUCCESS, null);


    bsPuller.getMessageQueue().clear();
    testTransitionCase(bsPuller, StateId.STREAM_REQUEST_SUCCESS, StateId.STREAM_RESPONSE_DONE, null);


    bsPuller.getMessageQueue().clear();
    cp.setSnapshotOffset(-1);
    testTransitionCase(bsPuller, StateId.STREAM_RESPONSE_DONE, StateId.REQUEST_TARGET_SCN, null);
    Assert.assertEquals(cp.getConsumptionMode(), DbusClientMode.BOOTSTRAP_SNAPSHOT, "Consumption Mode check");
    Assert.assertFalse(cp.isBootstrapTargetScnSet());


    bsPuller.getMessageQueue().clear();
//    testTransitionCase(bsPuller, StateId.REQUEST_TARGET_SCN, StateId.TARGET_SCN_RESPONSE_SUCCESS, null);
    testTransitionCase(bsPuller, StateId.REQUEST_TARGET_SCN, StateId.TARGET_SCN_RESPONSE_ERROR, null);


    bsPuller.getMessageQueue().clear();
    testTransitionCase(bsPuller, StateId.TARGET_SCN_RESPONSE_ERROR, StateId.PICK_SERVER, null);
    Assert.assertEquals(cp.getConsumptionMode(), DbusClientMode.BOOTSTRAP_SNAPSHOT, "Consumption Mode check");
    Assert.assertFalse(cp.isBootstrapTargetScnSet());
    Assert.assertEquals(cp.getBootstrapTargetScn().longValue(), Checkpoint.UNSET_BOOTSTRAP_TARGET_SCN);


  }

View Full Code Here

Examples of com.persistit.CheckpointManager.Checkpoint

        if (_closed.get() || !_initialized.get()) {
            return null;
        }
        cleanup();
        _journalManager.pruneObsoleteTransactions();
        final Checkpoint result = _checkpointManager.checkpoint();
        _journalManager.pruneObsoleteTransactions();
        return result;
    }

View Full Code Here

Examples of com.pointcliki.dizgruntled.logic.Checkpoint

    } else if (logic.equals("ExitTrigger")) {
      l = new Fort();
    } else if (logic.equals("FortressFlag")) {
      l = new FortressFlag();
    } else if (logic.equals("CheckpointTrigger")) {
      l = new Checkpoint();
    } else if (logic.equals("InGameText")) {
      l = new HelpBook();
    } else if (logic.equals("GruntPuddle")) {
      l = new GruntPuddle();
    } else if (logic.equals("GruntCreationPoint")) {

View Full Code Here

Examples of com.sun.grid.jgdi.configuration.Checkpoint




                try {


                    // Create a new checkpoint object which intialized with default values
                    Checkpoint ckpt = ConfigurationFactory.createCheckpointWithDefaults();
                    ckpt.setName("sample");
                    ckpt.setCkptCommand("/usr/bin/ckpt");
                    ckpt.setCkptDir("/tmp");


                    jgdi.addCheckpoint(ckpt);
                    try {
                        System.out.println("Checkpoint " + ckpt.getName() + " successfully added");
                        ckpt = jgdi.getCheckpoint(ckpt.getName());
                        ckpt.setRestCommand("/tmp/blubber");
                        jgdi.updateCheckpoint(ckpt);
                        System.out.println("Checkpoint " + ckpt.getName() + " successfully modified");
                    } finally {
                        jgdi.deleteCheckpoint(ckpt);
                        System.out.println("Checkpoint " + ckpt.getName() + " successfully deleted");
                    }


                    Thread.sleep(60000);
                } finally {
                    evc.close();

View Full Code Here

Examples of fr.imag.adele.apam.pax.test.performance.util.Checkpoint

      
      fibonacci.callsInit(0);
      
      for (int i = 0; i <= SIZE; i++) {
        
        Checkpoint p1 = new Checkpoint();


        int value = fibonacci.compute(i);


        Checkpoint p2 = new Checkpoint();


        MeasureHolder holder = Measure.time(p1, p2);


        System.err.println(String.format("%d\t%d\t%s\t%s\t%d", i,
            value, holder.usedMemory(), holder.asMili(),

View Full Code Here

Examples of org.agilewiki.jfile.transactions.db.Checkpoint

                Eval eval = new Eval(block.getTimestamp());
                if (generateCheckpoints) {
                    RP evalrp = new RP() {
                        @Override
                        public void processResponse(Object response) throws Exception {
                            Checkpoint checkpoint = new Checkpoint(
                                    block.getCurrentPosition(),
                                    block.getTimestamp(), block.getFileName());
                            checkpoint.send(TransactionProcessor.this, getParent(), rp);
                        }
                    };
                    eval.send(TransactionProcessor.this, evaluator, evalrp);
                } else {
                    eval.send(TransactionProcessor.this, evaluator, rp);

View Full Code Here

Examples of org.apache.jackrabbit.jcr2spi.operation.Checkpoint

        Operation co = Checkout.create(nodeState, this);
        workspaceManager.execute(co);
    }


    public NodeEntry checkpoint(NodeState nodeState) throws RepositoryException {
        Checkpoint cp = Checkpoint.create(nodeState, this);
        workspaceManager.execute(cp);
        return workspaceManager.getHierarchyManager().getNodeEntry(cp.getNewVersionId());
    }

View Full Code Here

Examples of org.archive.checkpointing.Checkpoint

        return Checkpoint.class.isAssignableFrom(cls);
    }


    @Override
    public void validate(Object target, Errors errors) {
        Checkpoint cp = ((CheckpointService)target).getRecoveryCheckpoint();
        if(cp==null) {
            return; 
        }
        if(!Checkpoint.hasValidStamp(cp.getCheckpointDir().getFile())) {
            errors.rejectValue(
                "recoveryCheckpoint.checkpointDir",
                null,
                "Configured recovery checkpoint "+cp.getName()
                +" incomplete: lacks valid stamp file.");
        }
    }

View Full Code Here

Examples of org.exist.storage.txn.Checkpoint

     * @param switchLogFiles
     * @throws TransactionException
     */
    public void checkpoint(long txnId, boolean switchLogFiles) throws TransactionException {
        LOG.debug("Checkpoint reached");
        writeToLog(new Checkpoint(txnId));
        if (switchLogFiles)
            // if we switch files, we don't need to sync.
            // the file will be removed anyway.
            {flushBuffer();}
        else

View Full Code Here

0 1 2 3 4 5 6

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.