Examples of org.apache.uima.analysis_engine.AnalysisEngine

org.apache.uima.analysis_engine.AnalysisEngine
An Analysis Engine is a component responsible for analyzing unstructured information, discovering and representing semantic content. Unstructured information includes, but is not restricted to, text documents.
An AnalysisEngine operates on an "analysis structure" (implemented by {@link org.apache.uima.cas.CAS}). The CAS contains the artifact to be processed as well as semantic information already inferred from that artifact. The AnalysisEngine analyzes this information and adds new information to the CAS.
To create an instance of an Analysis Engine, an application should call {@link org.apache.uima.UIMAFramework#produceAnalysisEngine(ResourceSpecifier)}.
A typical application interacts with the Analysis Engine interface as follows:
1. Call {@link #newCAS()} to create a new Common Analysis System appropriate for thisAnalysisEngine.
2. Use the {@link CAS} interface to populate the CAS with the artifact to beanalyzed any information known about this document (e.g. the language of a text document).
3. Optionally, create a {@link org.apache.uima.analysis_engine.ResultSpecification} thatidentifies the results you would like this AnalysisEngine to generate (e.g. people, places, and dates), and call the {#link {@link #setResultSpecification(ResultSpecification)} method.
4. Call {@link #process(CAS)} - the AnalysisEngine will perform its analysis.
5. Retrieve the results from the {@link CAS}.
6. Call {@link CAS#reset()} to clear out the CAS and prepare for processing anew artifact.
7. Repeat steps 2 through 6 for each artifact to be processed.
Important: It is highly recommended that you reuse CAS objects rather than calling newCAS() prior to each analysis. This is because CAS objects may be expensive to create and may consume a significant amount of memory.
Instead of using the {@link CAS} interface, applications may wish to use the Java-object-based{@link JCas} interface. In that case, the call to newCAS from step 1 above wouldbe replaced by {@link #newJCas()}, and the {@link #process(JCas)} method would be used.
Analysis Engine implementations may or may not be capable of simultaneously processing multiple documents in a multithreaded environment. See the documentation associated with the implementation or factory method (e.g. ( {@link org.apache.uima.UIMAFramework#produceAnalysisEngine(ResourceSpecifier)}) that you are using.

   * @see org.apache.uima.analysis_engine.impl.AnalysisEngineImplBase#processAndOutputNewCASes(org.apache.uima.jcas.JCas)
   */
  @Override
  public JCasIterator processAndOutputNewCASes(JCas aJCas) throws AnalysisEngineProcessException {
    enterProcess(); // start timer for collecting performance stats
    AnalysisEngine ae = null;
    try {
      ae = getAeFromPool();
      return ae.processAndOutputNewCASes(aJCas);
    } finally {
      if (ae != null) {
        mPool.releaseAnalysisEngine(ae);
      }
      exitProcess(); // stop timer for collecting performance stats

View Full Code Here

              .getXMLParser()
              .parseAnalysisEngineDescription(
                      new XMLInputSource(
                              JUnitExtension
                                      .getFile("TextAnalysisEngineImplTest/AggregateWithSegmenterForErrorTest.xml")));
      AnalysisEngine ae = UIMAFramework.produceAnalysisEngine(aggSegDesc);
      
      CAS cas = ae.newCAS();
      for (int i = 0; i < 2; i++) // verify we can do this more than once
      {
        FlowControllerForErrorTest.reset();
        cas.setDocumentText("Line one\nLine two\nERROR");
        CasIterator iter = ae.processAndOutputNewCASes(cas);
        assertTrue(iter.hasNext());
        CAS outCas = iter.next();
        assertEquals("Line one", outCas.getDocumentText());
        outCas.release();
        assertTrue(iter.hasNext());
        outCas = iter.next();
        assertEquals("Line two", outCas.getDocumentText());
        outCas.release();
        try {
          UIMAFramework.getLogger().setLevel(Level.OFF);  // Suppress logging of expected exception
          assertTrue(iter.hasNext());
          outCas = iter.next();
          fail(); // the above should throw an exception
        } catch (AnalysisEngineProcessException e) {
          UIMAFramework.getLogger().setLevel(Level.INFO); // Restore to apparent default of INFO
        }
        //check that FlowController was notified twice, once for the 
        //segment's flow and once for the complete document's flow
        assertEquals(2, FlowControllerForErrorTest.abortedDocuments.size());
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("ERROR"));
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("Line one\nLine two\nERROR"));


        cas.reset();
      }


      // nested aggregate
      AnalysisEngineDescription nestedAggSegDesc = UIMAFramework
              .getXMLParser()
              .parseAnalysisEngineDescription(
                      new XMLInputSource(
                              JUnitExtension
                                      .getFile("TextAnalysisEngineImplTest/NestedAggregateSegmenterForErrorTest.xml")));
      ae = UIMAFramework.produceAnalysisEngine(nestedAggSegDesc);
      cas = ae.newCAS();
      for (int i = 0; i < 2; i++) // verify we can do this more than once
      {
        FlowControllerForErrorTest.reset();
        cas.setDocumentText("Line one\nLine two\nERROR");
        CasIterator iter = ae.processAndOutputNewCASes(cas);
        assertTrue(iter.hasNext());
        CAS outCas = iter.next();
        assertEquals("Line one", outCas.getDocumentText());
        outCas.release();
        assertTrue(iter.hasNext());
        outCas = iter.next();
        assertEquals("Line two", outCas.getDocumentText());
        outCas.release();
        try {
          UIMAFramework.getLogger().setLevel(Level.OFF);  // Suppress logging of expected exception
          assertTrue(iter.hasNext());
          outCas = iter.next();
          fail(); // the above should throw an exception
        } catch (AnalysisEngineProcessException e) {
          UIMAFramework.getLogger().setLevel(Level.INFO); // Restore to apparent default of INFO
        }
        //check that FlowController was notified three times, once for the 
        //segment's flow and twice for the complete document's flow (once
        //in each aggregate)
        assertEquals(3, FlowControllerForErrorTest.abortedDocuments.size());
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("ERROR"));
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("Line one\nLine two\nERROR"));
        FlowControllerForErrorTest.abortedDocuments.remove("Line one\nLine two\nERROR");
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("Line one\nLine two\nERROR"));
        
        cas.reset();
      }


      // 2 segmenters
      AnalysisEngineDescription twoSegDesc = UIMAFramework
              .getXMLParser()
              .parseAnalysisEngineDescription(
                      new XMLInputSource(
                              JUnitExtension
                                      .getFile("TextAnalysisEngineImplTest/AggregateWith2SegmentersForErrorTest.xml")));
      ae = UIMAFramework.produceAnalysisEngine(twoSegDesc);
      cas = ae.newCAS();
      for (int i = 0; i < 2; i++) // verify we can do this more than once
      {
        FlowControllerForErrorTest.abortedDocuments.clear();
        cas.setDocumentText("One\tTwo\nThree\tERROR");
        CasIterator iter = ae.processAndOutputNewCASes(cas);
        assertTrue(iter.hasNext());
        CAS outCas = iter.next();
        assertEquals("One", outCas.getDocumentText());
        outCas.release();
        assertTrue(iter.hasNext());
        outCas = iter.next();
        assertEquals("Two", outCas.getDocumentText());
        outCas.release();
        assertTrue(iter.hasNext());
        outCas = iter.next();
        assertEquals("Three", outCas.getDocumentText());
        outCas.release();
        try {
          UIMAFramework.getLogger().setLevel(Level.OFF);  // Suppress logging of expected exception
          assertTrue(iter.hasNext());
          outCas = iter.next();
          fail(); // the above should throw an exception
        } catch (AnalysisEngineProcessException e) {
          UIMAFramework.getLogger().setLevel(Level.INFO); // Restore to apparent default of INFO
        }
        //check that FlowController was notified three times, once for each level of granularity
        assertEquals(3, FlowControllerForErrorTest.abortedDocuments.size());
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("ERROR"));
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("Three\tERROR"));
        assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("One\tTwo\nThree\tERROR"));
        
        cas.reset();
      }


      // segmenter that requests too many CASes
      AnalysisEngineDescription segmenterDesc = UIMAFramework.getXMLParser()
              .parseAnalysisEngineDescription(
                      new XMLInputSource(JUnitExtension
                              .getFile("TextAnalysisEngineImplTest/BadSegmenter.xml")));
      ae = UIMAFramework.produceAnalysisEngine(segmenterDesc);
      cas = ae.newCAS();
      cas.setDocumentText("Line one\nLine two\nLine three");
      CasIterator iter = ae.processAndOutputNewCASes(cas);
      assertTrue(iter.hasNext());
      CAS outCas = iter.next(); // first call OK
      outCas.release();
      assertTrue(iter.hasNext());
      // next call should fail with AnalysisEngineProcessException
      try {
        UIMAFramework.getLogger().setLevel(Level.OFF);  // Suppress logging of expected exception
        iter.next();
        fail(); // should not get here
      } catch (AnalysisEngineProcessException e) {
        UIMAFramework.getLogger().setLevel(Level.INFO); // Restore to apparent default of INFO
      }
      
      // bad segmenter in an aggregate
      AnalysisEngineDescription aggWithBadSegmenterDesc = UIMAFramework.getXMLParser()
      .parseAnalysisEngineDescription(
              new XMLInputSource(JUnitExtension
                      .getFile("TextAnalysisEngineImplTest/AggregateWithBadSegmenterForErrorTest.xml")));
      ae = UIMAFramework.produceAnalysisEngine(aggWithBadSegmenterDesc);
      FlowControllerForErrorTest.reset();
      cas = ae.newCAS();
      cas.setDocumentText("Line one\nLine two\nLine three");
      iter = ae.processAndOutputNewCASes(cas);
      assertTrue(iter.hasNext());
      outCas = iter.next(); // first call OK
      outCas.release();
      assertTrue(FlowControllerForErrorTest.abortedDocuments.isEmpty());
      assertTrue(FlowControllerForErrorTest.failedAEs.isEmpty());
      // next call should fail with AnalysisEngineProcessException
      try {
        UIMAFramework.getLogger().setLevel(Level.OFF);  // Suppress logging of expected exception
        if (iter.hasNext()) {
          iter.next();
        }
        fail(); // should not get here
      } catch (AnalysisEngineProcessException e) {
        UIMAFramework.getLogger().setLevel(Level.INFO); // Restore to apparent default of INFO
      }
      assertEquals(1, FlowControllerForErrorTest.abortedDocuments.size());
      assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("Line one\nLine two\nLine three"));
      assertEquals(1,FlowControllerForErrorTest.failedAEs.size());
      assertTrue(FlowControllerForErrorTest.failedAEs.contains("Segmenter"));


      //configure AE to continue after error
      ae = UIMAFramework.produceAnalysisEngine(aggWithBadSegmenterDesc);
      ae.setConfigParameterValue("ContinueOnFailure", Boolean.TRUE);
      ae.reconfigure();
      FlowControllerForErrorTest.reset();


      cas.reset();
      cas.setDocumentText("Line one\nLine two\nLine three");
      iter = ae.processAndOutputNewCASes(cas);
      assertTrue(iter.hasNext());
      outCas = iter.next(); // first call OK
      outCas.release();
      assertTrue(FlowControllerForErrorTest.abortedDocuments.isEmpty());
      assertTrue(FlowControllerForErrorTest.failedAEs.isEmpty());

View Full Code Here

      // primitive
      AnalysisEngineDescription segmenterDesc = UIMAFramework.getXMLParser()
              .parseAnalysisEngineDescription(
                      new XMLInputSource(JUnitExtension
                              .getFile("TextAnalysisEngineImplTest/NewlineResegmenter.xml")));
      AnalysisEngine ae = UIMAFramework.produceAnalysisEngine(segmenterDesc);
      CAS inputCas1 = ae.newCAS();
      Type sdiType = inputCas1.getTypeSystem().getType(
              "org.apache.uima.examples.SourceDocumentInformation");
      Feature uriFeat = sdiType.getFeatureByBaseName("uri");
      inputCas1.setDocumentText("This is");
      FeatureStructure sdiFS = inputCas1.createFS(sdiType);
      sdiFS.setStringValue(uriFeat, "cas1");
      inputCas1.getIndexRepository().addFS(sdiFS);
      CAS inputCas2 = ae.newCAS();
      inputCas2.setDocumentText(" one.\nThis is");
      FeatureStructure sdiFS2 = inputCas2.createFS(sdiType);
      sdiFS2.setStringValue(uriFeat, "cas2");
      inputCas2.getIndexRepository().addFS(sdiFS2);
      CAS inputCas3 = ae.newCAS();
      inputCas3.setDocumentText(" two.\n");
      FeatureStructure sdiFS3 = inputCas3.createFS(sdiType);
      sdiFS3.setStringValue(uriFeat, "cas3");
      inputCas3.getIndexRepository().addFS(sdiFS3);


      // input first CAS. Should be no segments yet.
      CasIterator iter = ae.processAndOutputNewCASes(inputCas1);
      assertFalse(iter.hasNext());
      // input second CAS. We should get back one segment.
      iter = ae.processAndOutputNewCASes(inputCas2);
      assertTrue(iter.hasNext());
      CAS outCas = iter.next();
      assertEquals("This is one.", outCas.getDocumentText());
      // -- check SourceDocumentInformation FSs
      Iterator<AnnotationFS> sdiIter = outCas.getAnnotationIndex(sdiType).iterator();
      assertTrue(sdiIter.hasNext());
      AnnotationFS outSdiFs = (AnnotationFS) sdiIter.next();
      assertEquals("This is", outSdiFs.getCoveredText());
      assertEquals("cas1", outSdiFs.getStringValue(uriFeat));
      assertTrue(sdiIter.hasNext());
      outSdiFs = (AnnotationFS) sdiIter.next();
      assertEquals(" one.", outSdiFs.getCoveredText());
      assertEquals("cas2", outSdiFs.getStringValue(uriFeat));
      assertFalse(sdiIter.hasNext());
      // --
      assertFalse(iter.hasNext());


      // input third CAS. We should get back one more segment.
      iter = ae.processAndOutputNewCASes(inputCas3);
      assertTrue(iter.hasNext());
      outCas = iter.next();
      assertEquals("This is two.", outCas.getDocumentText());
      // -- check SourceDocumentInformation FSs
      sdiIter = outCas.getAnnotationIndex(sdiType).iterator();

View Full Code Here

    try {
      //This test uses an aggregate AE fails if the document text is set to "ERROR".
      AnalysisEngineDescription aeDesc = UIMAFramework.getXMLParser()
              .parseAnalysisEngineDescription(
                      new XMLInputSource(JUnitExtension.getFile("TextAnalysisEngineImplTest/AggregateForErrorTest.xml")));
      AnalysisEngine ae = UIMAFramework.produceAnalysisEngine(aeDesc);
      FlowControllerForErrorTest.reset();
      CAS cas = ae.newCAS();
      //try document that should succeed
      cas.setDocumentText("This is OK");
      ae.process(cas);
      //flow controller should not be notified
      assertTrue(FlowControllerForErrorTest.abortedDocuments.isEmpty());
      assertTrue(FlowControllerForErrorTest.failedAEs.isEmpty());
      
      //now one that fails
      cas.reset();
      cas.setDocumentText("ERROR");
      try {
        UIMAFramework.getLogger().setLevel(Level.OFF);  // Suppress logging of expected exception
        ae.process(cas);
        fail();
      }
      catch(AnalysisEngineProcessException e) {
        UIMAFramework.getLogger().setLevel(Level.INFO); // Restore to apparent default of INFO
      }
      assertEquals(1, FlowControllerForErrorTest.abortedDocuments.size());
      assertTrue(FlowControllerForErrorTest.abortedDocuments.contains("ERROR"));
      assertEquals(1, FlowControllerForErrorTest.failedAEs.size());
      assertTrue(FlowControllerForErrorTest.failedAEs.contains("ErrorAnnotator"));
    
      //AE should still be able to process a new document now
      FlowControllerForErrorTest.reset();
      cas.reset();
      cas.setDocumentText("This is OK");
      ae.process(cas);
      assertTrue(FlowControllerForErrorTest.abortedDocuments.isEmpty());
      assertTrue(FlowControllerForErrorTest.failedAEs.isEmpty());
      
      //configure AE to continue after error
      ae.setConfigParameterValue("ContinueOnFailure", Boolean.TRUE);
      ae.reconfigure();
      cas.reset();
      cas.setDocumentText("ERROR");
      UIMAFramework.getLogger().setLevel(Level.OFF);  // Suppress logging of expected exception
      ae.process(cas); //should not throw exception now
      UIMAFramework.getLogger().setLevel(Level.INFO); // Restore to apparent default of INFO
      
      //document should not have aborted, but FC should have been notified of the failiure
      assertEquals(0, FlowControllerForErrorTest.abortedDocuments.size());
      assertEquals(1, FlowControllerForErrorTest.failedAEs.size());

View Full Code Here

  }
  
  public void testMissingSuper() throws Exception {
    try {
      // initialize simple primitive TextAnalysisEngine
      AnalysisEngine ae1 = new PrimitiveAnalysisEngine_impl();
      AnalysisEngineDescription primitiveDesc = new AnalysisEngineDescription_impl();
      primitiveDesc.setFrameworkImplementation(Constants.JAVA_FRAMEWORK_NAME);
      primitiveDesc.setPrimitive(true);
      primitiveDesc.setAnnotatorImplementationName(AnnotatorMissingSuper.class.getCanonicalName());
      ae1.initialize(primitiveDesc, null);
      ae1.process(ae1.newCAS());
    } catch (Exception e) {
      JUnitExtension.handleException(e);
    }
  }

View Full Code Here

   */
  public void doFullValidation() throws ResourceInitializationException {
    // attempt to instantiate AE in "verification mode"
    Map<String, Object> m = new HashMap<String, Object>();
    m.put(AnalysisEngineImplBase.PARAM_VERIFICATION_MODE, Boolean.TRUE);
    AnalysisEngine ae = UIMAFramework.produceAnalysisEngine(this, m);
    validateSofaMappings();
    ae.newCAS();
  }

View Full Code Here

  public void doFullValidation(ResourceManager aResourceManager)
          throws ResourceInitializationException {
    // attempt to instantiate AE in "verification mode"
    Map<String, Object> m = new HashMap<String, Object>();
    m.put(AnalysisEngineImplBase.PARAM_VERIFICATION_MODE, Boolean.TRUE);
    AnalysisEngine ae = UIMAFramework.produceAnalysisEngine(this, aResourceManager, m);
    validateSofaMappings();
    ae.newCAS();
  }

View Full Code Here

      resultSpecForComponents.addCapabilities(getAllComponentCapabilities(), false);


      // now iterate over components and call their setResultSpecification methods
      Iterator<AnalysisEngine> componentIter = _getASB().getComponentAnalysisEngines().values().iterator();
      while (componentIter.hasNext()) {
        AnalysisEngine ae = (AnalysisEngine) componentIter.next();
        ae.setResultSpecification(resultSpecForComponents);
      }
    }
  }

View Full Code Here

        }
      }
      //call components in the order specified in the flow
      if (orderedNodes != null) {
        for (int i = 0; i < orderedNodes.length; i++) {
          AnalysisEngine component = (AnalysisEngine)components.remove(orderedNodes[i]);  
          component.collectionProcessComplete();
        }
      }
      //now call remaining components in arbitrary order
      Iterator<AnalysisEngine> iter = components.values().iterator();
      while (iter.hasNext()) {

View Full Code Here

      trace.addEvent(procEvt);


      // now add subevents for each component
      Iterator<AnalysisEngine> aeIter = _getASB().getComponentAnalysisEngines().values().iterator();
      while (aeIter.hasNext()) {
        AnalysisEngine ae = aeIter.next();
        if (ae instanceof AnalysisEngineImplBase) {
          ProcessTrace subPT = ((AnalysisEngineImplBase) ae).buildProcessTraceFromMBeanStats();
          if (subPT.getEvents().size() > 0) {
            procEvt.addSubEvent((ProcessTraceEvent) subPT.getEvents().get(0));
          }

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of org.apache.uima.analysis_engine.AnalysisEngine

dkpro.similarity.example.WithDKPro

net.sf.nlpshell.MainUima

org.apache.clerezza.uima.utils.AEProvider

org.apache.clerezza.uima.utils.AEProviderTest

org.apache.ctakes.assertion.eval.AssertionEvalBasedOnModifier

org.apache.ctakes.assertion.eval.AssertionEvaluation

org.apache.ctakes.constituency.parser.ae.ParserEvaluationAnnotator

org.apache.ctakes.constituency.parser.util.CommandLineParserUtil

org.apache.ctakes.core.ae.SHARPKnowtatorXMLReader

org.apache.ctakes.core.ae.SimpleSegmentAnnotatorTests

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.