Examples of at.ofai.gate.virtualdocuments.AnnotatedDocumentTransformer

at.ofai.gate.virtualdocuments.AnnotatedDocumentTransformer
A class that can be used to extract a string (with {@link #getStringForDocument(Document,String)}) or a sequence of annotations (with {@link #getIterator(Document,String)}from an annotated document based on a list of annotation specifications and a list of processing options which have to be provided when an object of this class is created. The annotation specifications describe which annotations and features to pick in which order of preference and whether to use the document text covered by an annotation, the value of a feature of an annotation, or some constant string value in case an annotation type or annotation with some feature is found. The processing options specify if and how to separate the strings which are found based on the annotation specification or how to look for the next matching annotation after an annotation has been found.
The annotation specifications list consists of entries as described below, where the order of the entries expresses decreasing priority:
- annotationtype - if an annotation type "annotationtype" is found, take the text covered by that annotation
- annotationtype->value - this matches in the same way as the previous specification but takes the value specified instead of the text covered by the annotation. is found, take the value specified after the arrow
- annotationtype.featurename - if an annotation of type "annotationtype" is found with a non-null value for the feature with the name "featurename", take the value of the feature.
- annotationtype.featurename->value - this matches in the same way as the previous specification but takes the value specified instead of the value of the feature.
- @STRING - if no annotation matches, take the original document text until some other higher priority specification matches. This can only be reasonably be used as the last entry in the list (since it will always match).
Note that trailing spaces are not ignored as they may be significant, for example for a specification like "SpaceToken-> " which specifies that for each SpaceToken annotation a literal space should be generated.
The processing option list consists of entries of the following form, in any order:
- takeAll=true|false if true, apply all matching specifications at some offset, if false only the one with the highest priority. If there are several annotations matching a single specification, only the longest match is selected, no matter what this parameter is set to.
- takeOverlapping=true|false if true, try all offsets within a range that already matched, if false, proceed to the offset behind the current match.
- separator=somestring how to separate the output from different offsets
- separatorSame=someString how to seperate the output that comes from identical offsets. If not set, use the same string as for separator.
- terminator=string how to terminate the output string.
- separatorKeyValue=someString if this is specified and not empty, the output will have the format "keyvalue" where key is the specification, someString what has been specified for this parameter and value the text added to the output.
Note that some of the annotation specifications or processing options do not make sense when used with {@link #getIterator}.
Before extracting annotations, the user can request that mappings from offsets in the original document to the output string and/or mappings from the offsets in the output string back to the original document are creating by calling {@link #setGenerateForwardOffsetMap} with parameter trueand/or {@link #setGenerateBackwardOffsetMap} with parameter true.This is implemented as a setter method and not as a processing parameter since it is something that the end user in general should not be able to influence.
When those maps have been generated, the methods {@link #addBackMappedAnnotations} and{@link #addForwardMappedAnnotations} (NOT YET IMPLEMENTED) can be used to transfer annotations from the virtual document to the original document or from the original document to the virtual document.
TODOS: Add the construct !Type and !Type.feature to the specification: if this is present and active at an offset, the processing should immediately skip to the next candidate offset and not try the remaining specs. The meaning of !Type.feature is that there is an annotation type Type present which does have a feature map that contains a non-null value for the feature. The meaning of just !Type is that there is an annotation of that type present. Maybe we should only implement !Type at the moment.
- IMPORTANT: transfer the code to actually create a virtual document into this class!
@author Johann Petrak

        processingOptions.remove("separator");
      }
    }
    try {
      annotatedDocumentTransformer =
        new AnnotatedDocumentTransformer(
        getAnnotationSpecifications(), processingOptions,
        false, true);
    } catch (InvalidNameException ex) {
      throw new ExecutionException(ex);
    }

View Full Code Here

        processingOptions.remove("separator");
      }
    }
    try {
      annotatedDocumentTransformer =
        new AnnotatedDocumentTransformer(
        getAnnotationSpecifications(), processingOptions,
        false, true);
    } catch (InvalidNameException ex) {
      throw new ExecutionException(ex);
    }

View Full Code Here

    annSpecs.add("Lookup.majorType");
    annSpecs.add("Token.string");
    FeatureMap procSpecs = Factory.newFeatureMap();
    procSpecs.put("separator"," ");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,true,true);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    assertEquals("time_modifier is a simple test document for testing the PN class . stop contains token annotations in the default annotation set from person_first and some additional manual annotations .", theString1);


    OffsetMapping bom = adt.getBackwardOffsetMap();
    OffsetMapping fom = adt.getForwardOffsetMap();
    // test some backward mappings
    assertEquals("Backward mapping of 'is' start - start offset", 5, bom.getFrom(14));
    assertEquals("Backward mapping of 'is' start - end offset", 7, bom.getTo(14));
    assertEquals("Backward mapping of 'is' end - start offset", 5, bom.getFrom(15));
    assertEquals("Backward mapping of 'is' end - end offset", 7, bom.getTo(15));
    adt.getForwardOffsetMap();
  }

View Full Code Here

    FeatureMap procSpecs = Factory.newFeatureMap();
    procSpecs.put("separator","/");
    procSpecs.put("terminator","EOL");
    //procSpecs.add("separatorKeyValue==");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,false,false);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    assertEquals("time_modifier/PN/stop/person_firstEOL", theString1);
  }

View Full Code Here

    annSpecs.add("Person->CONSTANT");
    FeatureMap procSpecs = Factory.newFeatureMap();
    procSpecs.put("separator"," ");
    procSpecs.put("separatorKeyValue","=");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,false,false);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    assertEquals("Lookup.majorType=time_modifier Unknown.kind=PN Lookup.majorType=stop Lookup.majorType=person_first", theString1);
  }

View Full Code Here

    annSpecs.add("Lookup.majorType");
    annSpecs.add("Person->CONSTANT");
    FeatureMap procSpecs = Factory.newFeatureMap();
    procSpecs.put("separator",",");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,false,false);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    assertEquals("time_modifier,PN,stop,FirstName", theString1);
  }

View Full Code Here

    procSpecs.put("separator", ",");
    procSpecs.put("separatorSame", ";");
    procSpecs.put("separatorKeyValue", "=");
    procSpecs.put("takeOverlapping","true");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,false,false);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    assertEquals("Lookup.majorType=time_modifier,WhatClass=AnnotatedDocumentTranformer class,ClassWord=class,Lookup.majorType=stop,FirstPerson.rule=FirstName", theString1);
  }

View Full Code Here

    procSpecs.put("separator", ",");
    procSpecs.put("separatorSame", ";");
    procSpecs.put("separatorKeyValue", "=");
    procSpecs.put("takeAll", "true");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,false,false);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    assertEquals("Lookup.majorType=time_modifier,WhatClass=AnnotatedDocumentTranformer class;ClassName=AnnotatedDocumentTranformer;Unknown.kind=PN,ClassWord=class,Lookup.majorType=stop,FirstPerson.rule=FirstName;Person.rule1=GazPersonFirst;Lookup.majorType=person_first;Person=CONSTANT", theString1);
  }

View Full Code Here

    annSpecs.add("SpaceToken-> ");
    annSpecs.add("@STRING");
    FeatureMap procSpecs = Factory.newFeatureMap();
    procSpecs.put("separator","");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,false,false);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    assertEquals("time_modifier is a simple test document for testing the AnnotatedDocumentTranformer class. stop contains token annotations in the default annotation set from CONSTANT and some additional manual annotations.", theString1);
  }

View Full Code Here

    annSpecs.add("Person->CONSTANT");
    annSpecs.add("Lookup.majorType");
    FeatureMap procSpecs = Factory.newFeatureMap();
    procSpecs.put("separator","");


    AnnotatedDocumentTransformer adt =
            new AnnotatedDocumentTransformer(annSpecs,procSpecs,true,true);
    String theString1 = adt.getStringForDocument(testDoc01In,"");
    System.out.println("String: >"+theString1+"<");
    System.out.println("Forward offset map: "+adt.getForwardOffsetMap());
    System.out.println("Backward offset map: "+adt.getBackwardOffsetMap());
  }

View Full Code Here

TOP

Related Classes of at.ofai.gate.virtualdocuments.AnnotatedDocumentTransformer

at.ofai.gate.extendedgazetteer.IndirectExtendedGazetteer

at.ofai.gate.regexpannotator.IndirectSimpleRegexpAnnotator

at.ofai.gate.virtualdocuments.testing.TestAnnotatedDocumentTransformer

gate.Annotation

gate.AnnotationSet

gate.util.GateRuntimeException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.