Examples of com.mapr.franz.catcher.Client$MessageRetry

com.mapr.franz.catcher.Client

Handles sending log messages to the array of catcher servers. In doing so, we need to be as robust as possible and recover from previous errors.

The basic idea is that when we originally connect, we give a list of as many servers as we can in the form of hostname, port. These servers are connected to and they are queried to find out about any other servers that might be in the cloud. Servers might have multiple routes, but we resolve this because each server returns a unique identifier.

When it comes time to log, we look in a topic=> server cache. If we find a preferred server there, we use it. Otherwise, we pick some server at random.

When we send a log message, the response may include a redirect to a different server. If so, we make sure that we have connected to all forms of that server's address and cache that for later. Then we retry the send to the new server. We may also cache that message for later retry.

If a log request results in an error, we try to get rid of the connection that caused us this grief. This might ultimately cause us to forget a host or even a cache entry, but we will attempt to re-open these connections later, usually due to a referral back to that host. During server cloud reorganizations, we may not re-open the same server.

        Options options = parseOptions(args);
        log.warn("Options = {}", options.toString());


        double scale = Math.log(options.ratio) / 2;


        Client client = new Client(Collections.singleton(new PeerInfo(options.host, options.port)));


        History recorder = new History(1, 10, 30).logTicks();


        Random gen = RandomUtils.getRandom();
        ChineseRestaurant topicSampler = new ChineseRestaurant(options.alpha);
        Exponential interval = new Exponential(1, gen);
        double t = 0;
        long t0 = System.nanoTime();
        long messageId = 0;
        while (messageId < options.max) {
            double rate = options.peak * Math.exp(-scale * (Math.cos(2 * Math.PI * t / options.period) + 1));
            double dt = interval.nextDouble() / rate;
            t += dt;
            double now = (System.nanoTime() - t0) / 1e9;
            if (t > now + 0.01) {
                double millis = Math.floor((t - now) * 1000);
                double nanos = Math.rint((t - now) * 1e9 - millis * 1e6);
                Thread.sleep((long) millis, (int) nanos);
            }


            String topic = "topic-" + topicSampler.sample();
            String message = "m-" + (++messageId);
            client.sendMessage(topic, message);
            recorder.message(topic);
            log.debug("Sent {} / {}", topic, message);
            if ((messageId % 10000) == 0) {
                log.warn("Sent {} messages", messageId);
            }

            }
        });




        Thread.sleep(7000);
        Client c = new Client(Lists.newArrayList(new PeerInfo("localhost", 7070)));


        c.sendMessage("foo-1", "hello there");
        c.close();


        Thread.sleep(7000);
        bg.shutdownNow();
        Thread.sleep(1000);

Examples of com.mapr.franz.catcher.Client$MessageRetry

Related Classes of com.mapr.franz.catcher.Client$MessageRetry