Examples of net.joshdevins.storm.spout.KafkaSpout

net.joshdevins.storm.spout.KafkaSpout

This spout can be used to consume messages from a cluster of Kafka brokers. The consumption and further processing in the Storm topology is currently unreliable! When you create this spout in Storm, it is recommended that the parallelism hint you set be less than or equal to the number of partitions available for the topic you are consuming from. If you specify a parallelism hint that is great than the number of partitions, some spouts/consumers will sit idle and not do anything. This design is inherent to the way Kafka works. @author Josh Devins

        Properties kafkaProps = new Properties();
        kafkaProps.put("zk.connect", "localhost:2182");
        kafkaProps.put("zk.connectiontimeout.ms", "1000000");
        kafkaProps.put("groupid", "storm");


        return new KafkaSpout(kafkaProps, "test", new StringScheme());
    }

Examples of net.joshdevins.storm.spout.KafkaSpout

Related Classes of net.joshdevins.storm.spout.KafkaSpout