Examples of com.ngdata.sep.impl.SepConsumer

com.ngdata.sep.impl.SepConsumer
SepConsumer consumes the events for a certain SEP subscription and dispatches them to an EventListener (optionally multi-threaded). Multiple SepConsumer's can be started that take events from the same subscription: each consumer will receive a subset of the events.
On a more technical level, SepConsumer is the remote process (the "fake hbase regionserver") to which the regionservers in the hbase master cluster connect to replicate log entries.

                    mapper, htablePool, sharder, solrWriter);
            IndexingEventListener eventListener = new IndexingEventListener(
                    indexer, indexerConf.getTable(), indexerConf.tableNameIsRegex());


            int threads = hbaseConf.getInt("hbaseindexer.indexer.threads", 10);
            SepConsumer sepConsumer = new SepConsumer(indexerDef.getSubscriptionId(),
                    indexerDef.getSubscriptionTimestamp(), eventListener, threads, hostName,
                    zk, hbaseConf, null);


            handle = new IndexerHandle(indexerDef, indexer, sepConsumer, solr, connectionManager);
            handle.start();

View Full Code Here

        }


        PayloadExtractor payloadExtractor = new BasePayloadExtractor(Bytes.toBytes("sep-user-demo"), Bytes.toBytes("info"),
                Bytes.toBytes("payload"));


        SepConsumer sepConsumer = new SepConsumer(subscriptionName, 0, new EventLogger(), 1, "localhost", zk, conf,
                payloadExtractor);


        sepConsumer.start();
        System.out.println("Started");


        while (true) {
            Thread.sleep(Long.MAX_VALUE);
        }

View Full Code Here


            LinkIndex linkIndex = new LinkIndex(indexManager, /* TODO multiple repositories */ repositoryManager);


            LinkIndexUpdater linkIndexUpdater = new LinkIndexUpdater(repositoryManager, linkIndex);


            sepConsumer = new SepConsumer("LinkIndexUpdater", 0L, linkIndexUpdater, threads, hostName,
                    new ZooKeeperItfAdapter(zk), hbaseConf, new LilyPayloadExtractor());
            sepConsumer.start();
        }
    }

View Full Code Here

     */
    public void startSepEventSlave(String subscriptionId, EventListener eventListener) throws Exception {
        if (sepConsumer != null) {
            throw new IllegalStateException("There is already a SepEventSlave running, stop it first");
        }
        sepConsumer = new SepConsumer(subscriptionId, System.currentTimeMillis(), eventListener, 1, "localhost",
                new ZooKeeperItfAdapter(zk), hadoopConf, new LilyPayloadExtractor());
        sepConsumer.start();
    }

View Full Code Here

TOP

Related Classes of com.ngdata.sep.impl.SepConsumer

com.ngdata.hbaseindexer.supervisor.IndexerSupervisor

com.ngdata.sep.demo.LoggingConsumer

com.ngdata.sep.SepEvent

com.ngdata.sep.util.concurrent.WaitPolicy

org.apache.hadoop.hbase.Cell

org.apache.hadoop.hbase.CellScanner

org.apache.hadoop.hbase.ipc.FifoRpcScheduler

org.apache.hadoop.hbase.ipc.RpcServer

org.apache.hadoop.hbase.KeyValue

org.apache.hadoop.hbase.regionserver.wal.HLogKey

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.