Examples of org.apache.hadoop.mapreduce.Mapper$Context

org.apache.commons.scxml2.Context
Maps input key/value pairs to a set of intermediate key/value pairs.
Maps are the individual tasks which transform input records into a intermediate records. The transformed intermediate records need not be of the same type as the input records. A given input pair may map to zero or many output pairs.

The Hadoop Map-Reduce framework spawns one map task for each {@link InputSplit} generated by the {@link InputFormat} for the job.Mapper implementations can access the {@link Configuration} for the job via the {@link JobContext#getConfiguration()}.
The framework first calls {@link #setup(org.apache.hadoop.mapreduce.Mapper.Context)}, followed by {@link #map(Object,Object,Context)} for each key/value pair in the InputSplit. Finally {@link #cleanup(Context)} is called.
All intermediate values associated with a given output key are subsequently grouped by the framework, and passed to a {@link Reducer} to determine the final output. Users can control the sorting and grouping by specifying two key {@link RawComparator} classes.
The Mapper outputs are partitioned per Reducer. Users can control which keys (and hence records) go to which Reducer by implementing a custom {@link Partitioner}.
Users can optionally specify a combiner, via {@link Job#setCombinerClass(Class)}, to perform local aggregation of the intermediate outputs, which helps to cut down the amount of data transferred from the Mapper to the Reducer.
Applications can specify if and how the intermediate outputs are to be compressed and which {@link CompressionCodec}s are to be used via the Configuration.

If the job has zero reduces then the output of the Mapper is directly written to the {@link OutputFormat} without sorting by keys.
Example:
```
 public class TokenCounterMapper  extends Mapper{ private final static IntWritable one = new IntWritable(1); private Text word = new Text(); public void map(Object key, Text value, Context context) throws IOException { StringTokenizer itr = new StringTokenizer(value.toString()); while (itr.hasMoreTokens()) { word.set(itr.nextToken()); context.collect(word, one); } } } 
```
Applications may override the {@link #run(Context)} method to exert greater control on map processing e.g. multi-threaded Mappers etc.
@see InputFormat @see JobContext @see Partitioner @see Reducer

            throw new InvokerException(ioe.getMessage(), ioe.getCause());
        } catch (XMLStreamException xse) {
            throw new InvokerException(xse.getMessage(), xse.getCause());
        }
        executor = new SCXMLExecutor(evaluator, new SimpleDispatcher(), new SimpleErrorReporter());
        Context rootCtx = evaluator.newContext(null);
        for (Map.Entry<String, Object> entry : params.entrySet()) {
            rootCtx.setLocal(entry.getKey(), entry.getValue());
        }
        executor.setRootContext(rootCtx);
        try {
            executor.setStateMachine(scxml);
        }

View Full Code Here

     * {@inheritDoc}
     */
    @Override
    public Set<Entry<String, Object>> entrySet() {
        Set<Entry<String, Object>> entrySet = new HashSet<Entry<String, Object>>();
        Context current = leaf;
        while (current != null) {
            entrySet.addAll(current.getVars().entrySet());
            current = current.getParent();
        }
        return entrySet;
    }

View Full Code Here

     * {@inheritDoc}
     */
    @Override
    public Object get(final Object key) {
        if (key != null) {
            Context current = leaf;
            while (current != null) {
                if (current.getVars().containsKey(key.toString())) {
                    return current.getVars().get(key);
                }
                current = current.getParent();
            }
        }
        return null;
    }

View Full Code Here

    /**
     * {@inheritDoc}
     */
    @Override
    public void execute(ActionExecutionContext exctx) throws ModelException, SCXMLExpressionException {
        Context ctx = exctx.getContext(getParentEnterableState());
        Evaluator eval = exctx.getEvaluator();
        ctx.setLocal(getNamespacesKey(), getNamespaces());
        exctx.getAppLog().info(label + ": " + String.valueOf(eval.eval(ctx, expr)));
        ctx.setLocal(getNamespacesKey(), null);
    }

View Full Code Here

    /**
     * {@inheritDoc}
     */
    @Override
    public void execute(ActionExecutionContext exctx) throws ModelException, SCXMLExpressionException {
        Context ctx = exctx.getContext(getParentEnterableState());
        Evaluator eval = exctx.getEvaluator();
        ctx.setLocal(getNamespacesKey(), getNamespaces());
        Object varObj = eval.eval(ctx, expr);
        ctx.setLocal(getNamespacesKey(), null);
        ctx.setLocal(name, varObj);
        if (exctx.getAppLog().isDebugEnabled()) {
            exctx.getAppLog().debug("<var>: Defined variable '" + name
                + "' with initial value '" + String.valueOf(varObj) + "'");
        }
        TriggerEvent ev = new TriggerEvent(name + ".change", TriggerEvent.CHANGE_EVENT);

View Full Code Here

    /**
     * {@inheritDoc}
     */
    @Override
    public void execute(ActionExecutionContext exctx) throws ModelException, SCXMLExpressionException {
        Context ctx = exctx.getContext(getParentEnterableState());
        Evaluator eval = exctx.getEvaluator();
        ctx.setLocal(getNamespacesKey(), getNamespaces());
        try {
            Object arrayObject = eval.eval(ctx,array);
            if (arrayObject != null && (arrayObject instanceof Iterable || arrayObject.getClass().isArray())) {
                if (arrayObject.getClass().isArray()) {
                    for (int currentIndex = 0, size = Array.getLength(arrayObject); currentIndex < size; currentIndex++) {
                        ctx.setLocal(item, Array.get(arrayObject, currentIndex));
                        ctx.setLocal(index, currentIndex);
                        // The "foreach" statement is a "container"
                        for (Action aa : actions) {
                            aa.execute(exctx);
                        }
                    }
                }
                else {
                    // Spec requires to iterate over a shallow copy of underlying array in a way that modifications to
                    // the collection during the execution of <foreach> must not affect the iteration behavior.
                    // For array objects (see above) this isn't needed, but for Iterables we don't have that guarantee
                    // so we make a copy first
                    ArrayList<Object> arrayList = new ArrayList<Object>();
                    for (Object value: (Iterable)arrayObject) {
                        arrayList.add(value);
                    }
                    int currentIndex = 0;
                    for (Object value : arrayList) {
                        ctx.setLocal(item, value);
                        if (index != null) {
                            ctx.setLocal(index, currentIndex);
                        }
                        // The "foreach" statement is a "container"
                        for (Action aa : actions) {
                            aa.execute(exctx);
                        }
                        currentIndex++;
                    }
                }
            }
            // else {} TODO: place the error 'error.execution' in the internal event queue. (section "3.12.2 Errors")
        }
        finally {
            ctx.setLocal(getNamespacesKey(), null);
        }
    }

View Full Code Here

     * {@inheritDoc}
     */
    @Override
    public void execute(ActionExecutionContext exctx) throws ModelException, SCXMLExpressionException {
        EnterableState parentState = getParentEnterableState();
        Context ctx = exctx.getContext(parentState);
        Evaluator eval = exctx.getEvaluator();
        ctx.setLocal(getNamespacesKey(), getNamespaces());
        Boolean rslt;
        try {
            rslt = eval.evalCond(ctx, cond);
            if (rslt == null) {
                if (exctx.getAppLog().isDebugEnabled()) {
                    exctx.getAppLog().debug("Treating as false because the cond expression was evaluated as null: '"
                            + cond + "'");
                }
                rslt = Boolean.FALSE;
            }
        } catch (SCXMLExpressionException e) {
            rslt = Boolean.FALSE;
            exctx.getInternalIOProcessor().addEvent(new TriggerEvent(TriggerEvent.ERROR_EXECUTION, TriggerEvent.ERROR_EVENT));
            exctx.getErrorReporter().onError(ErrorConstants.EXPRESSION_ERROR, "Treating as false due to error: "
                    + e.getMessage(), this);
        }
        execute = rslt;
        ctx.setLocal(getNamespacesKey(), null);
        // The "if" statement is a "container"
        for (Action aa : actions) {
            if (execute && !(aa instanceof ElseIf)) {
                aa.execute(exctx);
            } else if (execute && aa instanceof ElseIf) {
                break;
            } else if (aa instanceof Else) {
                execute = true;
            } else if (aa instanceof ElseIf) {
                ctx.setLocal(getNamespacesKey(), getNamespaces());
                execute = eval.evalCond(ctx, ((ElseIf) aa).getCond());
                ctx.setLocal(getNamespacesKey(), null);
            }
        }
    }

View Full Code Here

     */
    @Override
    public void execute(ActionExecutionContext exctx) throws ModelException, SCXMLExpressionException {
        // Send attributes evaluation
        EnterableState parentState = getParentEnterableState();
        Context ctx = exctx.getContext(parentState);
        ctx.setLocal(getNamespacesKey(), getNamespaces());
        Evaluator eval = exctx.getEvaluator();
        // Most attributes of <send> are expressions so need to be
        // evaluated before the EventDispatcher callback
        Object hintsValue = null;
        if (hints != null) {
            hintsValue = eval.eval(ctx, hints);
        }
        String targetValue = target;
        if (target != null) {
            targetValue = (String) eval.eval(ctx, target);
            if ((targetValue == null || targetValue.trim().length() == 0)
                    && exctx.getAppLog().isWarnEnabled()) {
                exctx.getAppLog().warn("<send>: target expression \"" + target
                    + "\" evaluated to null or empty String");
            }
        }
        String typeValue;
        if (type != null) {
            typeValue = (String) eval.eval(ctx, type);
            if ((typeValue == null || typeValue.trim().length() == 0)
                    && exctx.getAppLog().isWarnEnabled()) {
                exctx.getAppLog().warn("<send>: type expression \"" + type
                    + "\" evaluated to null or empty String");
            }
        } else {
            // must default to 'scxml' when unspecified
            typeValue = TYPE_SCXML;
        }
        Map<String, Object> params = null;
        if (namelist != null) {
            StringTokenizer tkn = new StringTokenizer(namelist);
            params = new HashMap<String, Object>(tkn.countTokens());
            while (tkn.hasMoreTokens()) {
                String varName = tkn.nextToken();
                Object varObj = ctx.get(varName);
                if (varObj == null) {
                    //considered as a warning here
                    exctx.getErrorReporter().onError(ErrorConstants.UNDEFINED_VARIABLE,
                            varName + " = null", parentState);
                }
                params.put(varName, varObj);
            }
        }
        long wait = 0L;
        if (delay != null) {
            Object delayValue = eval.eval(ctx, delay);
            if (delayValue != null) {
                String delayString = delayValue.toString();
                wait = parseDelay(delayString, exctx.getAppLog());
            }
        }
        String eventValue = event;
        if (event != null) {
            eventValue = (String) eval.eval(ctx, event);
            if ((eventValue == null || eventValue.trim().length() == 0) && exctx.getAppLog().isWarnEnabled()) {
                exctx.getAppLog().warn("<send>: event expression \"" + event
                    + "\" evaluated to null or empty String");
            }
        }
        // Lets see if we should handle it ourselves
        if (typeValue != null
              && typeValue.trim().equalsIgnoreCase(TYPE_SCXML)) {
            if (targetValue == null || targetValue.trim().length() == 0) {
                // TODO: Remove both short-circuit passes in v1.0
                if (wait == 0L) {
                    if (exctx.getAppLog().isDebugEnabled()) {
                        exctx.getAppLog().debug("<send>: Enqueued event '" + eventValue
                            + "' with no delay");
                    }
                    exctx.getInternalIOProcessor().addEvent(
                            new TriggerEvent(eventValue, TriggerEvent.SIGNAL_EVENT, params));
                    return;
                }
            } else {
                // We know of no other
                if (exctx.getAppLog().isWarnEnabled()) {
                    exctx.getAppLog().warn("<send>: Unavailable target - "
                        + targetValue);
                }
                exctx.getInternalIOProcessor().addEvent(
                        new TriggerEvent(EVENT_ERR_SEND_TARGETUNAVAILABLE, TriggerEvent.ERROR_EVENT));
                // short-circuit the EventDispatcher
                return;
            }
        }
        ctx.setLocal(getNamespacesKey(), null);
        if (exctx.getAppLog().isDebugEnabled()) {
            exctx.getAppLog().debug("<send>: Dispatching event '" + eventValue
                + "' to target '" + targetValue + "' of target type '"
                + typeValue + "' with suggested delay of " + wait
                + "ms");

View Full Code Here

     * {@inheritDoc}
     */
    @Override
    public void execute(ActionExecutionContext exctx) throws ModelException, SCXMLExpressionException {
        EnterableState parentState = getParentEnterableState();
        Context ctx = exctx.getContext(parentState);
        Evaluator eval = exctx.getEvaluator();
        ctx.setLocal(getNamespacesKey(), getNamespaces());
        // "location" gets preference over "name"
        if (location != null) {
            Node oldNode = eval.evalLocation(ctx, location);
            if (oldNode != null) {
                //// rvalue may be ...
                // a Node, if so, import it at location
                Node newNode;
                try {
                    if (src != null && src.trim().length() > 0) {
                        newNode = getSrcNode();
                    } else {
                        newNode = eval.evalLocation(ctx, expr);
                    }
                    // Remove all children
                    Node removeChild = oldNode.getFirstChild();
                    while (removeChild != null) {
                        Node nextChild = removeChild.getNextSibling();
                        oldNode.removeChild(removeChild);
                        removeChild = nextChild;
                    }
                    if (newNode != null) {
                        // Adopt new children
                        for (Node child = newNode.getFirstChild();
                                child != null;
                                child = child.getNextSibling()) {
                            Node importedNode = oldNode.getOwnerDocument().
                                importNode(child, true);
                            oldNode.appendChild(importedNode);
                        }
                    }
                } catch (SCXMLExpressionException see) {
                    // or something else, stuff toString() into lvalue
                    Object valueObject = eval.eval(ctx, expr);
                    setNodeValue(oldNode, valueObject.toString());
                }
                if (exctx.getAppLog().isDebugEnabled()) {
                    exctx.getAppLog().debug("<assign>: data node '" + oldNode.getNodeName()
                        + "' updated");
                }
                /* TODO: send to notificationRegistry instead?
                TriggerEvent ev = new TriggerEvent(name + ".change",
                    TriggerEvent.CHANGE_EVENT);
                exctx.addInternalEvent(ev);
                */
            } else {
                exctx.getAppLog().error("<assign>: location does not point to"
                    + " a <data> node");
            }
        } else {
            // lets try "name" (usage as in Sep '05 WD, useful with <var>)
            if (!ctx.has(name)) {
                exctx.getErrorReporter().onError(ErrorConstants.UNDEFINED_VARIABLE, name
                    + " = null", parentState);
            } else {
                Object varObj;
                if (src != null && src.trim().length() > 0) {
                    varObj = getSrcNode();
                } else {
                    varObj = eval.eval(ctx, expr);
                }
                ctx.set(name, varObj);
                if (exctx.getAppLog().isDebugEnabled()) {
                    exctx.getAppLog().debug("<assign>: Set variable '" + name + "' to '"
                        + String.valueOf(varObj) + "'");
                }
                TriggerEvent ev = new TriggerEvent(name + ".change", TriggerEvent.CHANGE_EVENT);
                exctx.getInternalIOProcessor().addEvent(ev);
            }
        }
        ctx.setLocal(getNamespacesKey(), null);
    }

View Full Code Here

    /**
     * {@inheritDoc}
     */
    @Override
    public void execute(ActionExecutionContext exctx) throws ModelException, SCXMLExpressionException {
        Context ctx = isGlobalScript() ? exctx.getGlobalContext() : exctx.getContext(getParentEnterableState());
        ctx.setLocal(getNamespacesKey(), getNamespaces());
        Evaluator eval = exctx.getEvaluator();
        eval.evalScript(ctx, getScript());
        ctx.setLocal(getNamespacesKey(), null);
    }

View Full Code Here

0 1 2

TOP

Related Classes of org.apache.hadoop.mapreduce.Mapper$Context

ca.eandb.jmist.framework.color.Color

ca.eandb.jmist.framework.Intersection

ca.uhn.fhir.model.dstu.composite.CodeableConceptDt

ca.uhn.fhir.model.dstu.composite.PeriodDt

com.asakusafw.utils.java.model.syntax.Expression

com.dianping.cat.message.spi.MessageTree

com.dianping.cat.message.Transaction

com.facebook.presto.hive.shaded.com.google.common.util.concurrent.ThreadFactoryBuilder

com.google.code.or.binlog.impl.event.RotateEvent

com.google.code.or.binlog.impl.event.TableMapEvent

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.