Examples of com.cloudera.cdk.morphline.base.FaultTolerance

com.cloudera.cdk.morphline.base.FaultTolerance
Mission critical, large-scale online production systems need to make progress without downtime despite some issues. Some program exceptions tend to be transient, in which case the corresponding task can be retried. Examples include network connection errors, timeouts, etc. These are called recoverable exceptions. The isIgnoringRecoverableExceptions should only be enabled if an exception misclassification bug has been identified.

    
    int batchSize = SEQ_NUM2.incrementAndGet() % 2 == 0 ? 100 : 1;
    DocumentLoader testServer = new SolrServerDocumentLoader(solrServer, batchSize);
    MorphlineContext solrMorphlineContext = new SolrMorphlineContext.Builder()
      .setDocumentLoader(testServer)
      .setExceptionHandler(new FaultTolerance(false, false, SolrServerException.class.getName()))
      .setMetricRegistry(new MetricRegistry()).build();
    
    MorphlineHandlerImpl impl = new MorphlineHandlerImpl();
    impl.setMorphlineContext(solrMorphlineContext);

View Full Code Here

  }


  @Override
  public void configure(Context context) {
    if (morphlineContext == null) {
      FaultTolerance faultTolerance = new FaultTolerance(
          context.getBoolean(FaultTolerance.IS_PRODUCTION_MODE, false), 
          context.getBoolean(FaultTolerance.IS_IGNORING_RECOVERABLE_EXCEPTIONS, false),
          context.getString(FaultTolerance.RECOVERABLE_EXCEPTION_CLASSES));
      
      morphlineContext = new MorphlineContext.Builder()

View Full Code Here

    }
  }
  
  @Test
  public void testFaultTolerance() throws Exception {
    FaultTolerance tolerance = new FaultTolerance(true, false);
    tolerance = new FaultTolerance(true, true, IOException.class.getName());
    tolerance.handleException(new IOException(), new Record());
    
    tolerance = new FaultTolerance(true, true, IOException.class.getName());
    tolerance.handleException(new RuntimeException(), new Record());


    tolerance = new FaultTolerance(true, true, IOException.class.getName());
    tolerance.handleException(new RuntimeException(new IOException()), new Record());


    tolerance = new FaultTolerance(true, false, IOException.class.getName());
    try {
      tolerance.handleException(new IOException(), new Record());
      fail();
    } catch (MorphlineRuntimeException e) {
      ; // expected
    }


    tolerance = new FaultTolerance(false, false, IOException.class.getName());
    try {
      tolerance.handleException(new IOException(), new Record());
      fail();
    } catch (MorphlineRuntimeException e) {
      ; // expected
    }


    tolerance = new FaultTolerance(false, false, IOException.class.getName());
    try {
      tolerance.handleException(new RuntimeException(), new Record());
      fail();
    } catch (MorphlineRuntimeException e) {
      ; // expected
    }
    
    tolerance = new FaultTolerance(true, true, Error.class.getName());
    try {
      tolerance.handleException(new Error(), new Record());
      fail();
    } catch (Error e) {
      ; // expected
    }
  }

View Full Code Here

    return new PipeBuilder().build(config, null, collector, createMorphlineContext());
  }


  private MorphlineContext createMorphlineContext() {
    return new MorphlineContext.Builder()
      .setExceptionHandler(new FaultTolerance(false, false, SolrServerException.class.getName()))
      .setMetricRegistry(new MetricRegistry())
      .build();
  }

View Full Code Here


  private MorphlineContext createMorphlineContext() {
    return new SolrMorphlineContext.Builder()
      .setDocumentLoader(testServer)
//      .setDocumentLoader(new CollectingDocumentLoader(100))
      .setExceptionHandler(new FaultTolerance(false, false, SolrServerException.class.getName()))
      .setMetricRegistry(new MetricRegistry())
      .build();
  }

View Full Code Here

TOP

Related Classes of com.cloudera.cdk.morphline.base.FaultTolerance

com.cloudera.cdk.morphline.api.MorphlineTest

com.cloudera.cdk.morphline.solr.AbstractSolrMorphlineTest

com.cloudera.cdk.morphline.solr.AbstractSolrMorphlineZkTest

org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl

org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink

com.cloudera.cdk.morphline.api.MorphlineRuntimeException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.