Examples of GiraphTransferRegulator

org.apache.giraph.graph.GiraphTransferRegulator
Utility class to manage data transfers from a local worker reading InputSplits. Currently, this class measures # of vertices and edges per outgoing Collection of graph data (destined for a particular Partition and remote worker node, preselected by the master.) TODO: implement defaults and configurable options for measuring the size of input <V> or <E> data per read vertex, and setting limits on totals per outgoing graph data Collection etc. (See GIRAPH-260)

Examples of org.apache.giraph.graph.GiraphTransferRegulator

      Mapper<?, ?, ?, ?>.Context context,
      ImmutableClassesGiraphConfiguration<I, V, E, M> configuration) {
    this.context = context;
    this.configuration = configuration;
    transferRegulator =
        new GiraphTransferRegulator(configuration);
    if (LOG.isInfoEnabled()) {
      LOG.info("SendPartitionCache: maxVerticesPerTransfer = " +
          transferRegulator.getMaxVerticesPerTransfer());
      LOG.info("SendPartitionCache: maxEdgesPerTransfer = " +
          transferRegulator.getMaxEdgesPerTransfer());

View Full Code Here

Examples of org.apache.giraph.graph.GiraphTransferRegulator

    edges.initialize(3);
    edges.add(EdgeFactory.create(new IntWritable(2), new DoubleWritable(22)));
    edges.add(EdgeFactory.create(new IntWritable(3), new DoubleWritable(33)));
    edges.add(EdgeFactory.create(new IntWritable(4), new DoubleWritable(44)));
    vertex.initialize(new IntWritable(1), new FloatWritable(1), edges);
    GiraphTransferRegulator gtr =
        new GiraphTransferRegulator(job.getConfiguration());
    PartitionOwner owner = mock(PartitionOwner.class);
    when(owner.getPartitionId()).thenReturn(57);
    assertFalse(gtr.transferThisPartition(owner));
    gtr.incrementCounters(owner, vertex);
    assertTrue(gtr.transferThisPartition(owner));
  }

View Full Code Here

Examples of org.apache.giraph.graph.GiraphTransferRegulator

    edges.initialize(3);
    edges.add(EdgeFactory.create(new IntWritable(2), new DoubleWritable(22)));
    edges.add(EdgeFactory.create(new IntWritable(3), new DoubleWritable(33)));
    edges.add(EdgeFactory.create(new IntWritable(4), new DoubleWritable(44)));
    vertex.initialize(new IntWritable(1), new FloatWritable(1), edges);
    GiraphTransferRegulator gtr =
        new GiraphTransferRegulator(job.getConfiguration());
    PartitionOwner owner = mock(PartitionOwner.class);
    when(owner.getPartitionId()).thenReturn(57);
    assertFalse(gtr.transferThisPartition(owner));
    gtr.incrementCounters(owner, vertex);
    assertTrue(gtr.transferThisPartition(owner));
  }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.