public class GiraphTransferRegulator extends Object
Modifier and Type | Field and Description |
---|---|
static String |
MAX_EDGES_PER_TRANSFER
Maximum edges to read from an InputSplit locally that are
to be routed to a remote worker, before sending them.
|
static int |
MAX_EDGES_PER_TRANSFER_DEFAULT
Default maximum number of vertices per
temp partition before sending.
|
static String |
MAX_VERTICES_PER_TRANSFER
Maximum vertices to read from an InputSplit locally that are
to be routed to a remote worker, before sending them.
|
static int |
MAX_VERTICES_PER_TRANSFER_DEFAULT
Default maximum number of vertices per
temp partition before sending.
|
Constructor and Description |
---|
GiraphTransferRegulator(org.apache.hadoop.conf.Configuration conf)
Default constructor
|
Modifier and Type | Method and Description |
---|---|
void |
clearCounters()
Clear storage to reset for reading new InputSplit
|
long |
getMaxEdgesPerTransfer()
Getter for MAX edge count to initiate a transfer
|
long |
getMaxVerticesPerTransfer()
Getter for MAX vertex count to initiate a transfer
|
long |
getTotalEdges()
Getter for total edge count for the current InputSplit
|
long |
getTotalVertices()
Getter for total vetex count for the current InputSplit
|
<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
incrementCounters(PartitionOwner partitionOwner,
Vertex<I,V,E> vertex)
Increment V & E counts for new vertex read, store values
for that outgoing _temporary_ Partition, which shares the
Partition ID for the actual remote Partition the collection
will eventually be processed in.
|
boolean |
transferThisPartition(PartitionOwner owner)
Is this outbound data Collection full,
and ready to transfer?
|
public static final String MAX_VERTICES_PER_TRANSFER
public static final int MAX_VERTICES_PER_TRANSFER_DEFAULT
public static final String MAX_EDGES_PER_TRANSFER
public static final int MAX_EDGES_PER_TRANSFER_DEFAULT
public GiraphTransferRegulator(org.apache.hadoop.conf.Configuration conf)
conf
- the Configuration for this jobpublic boolean transferThisPartition(PartitionOwner owner)
owner
- the partition owner for the outbound datapublic void clearCounters()
public <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> void incrementCounters(PartitionOwner partitionOwner, Vertex<I,V,E> vertex)
I
- the vertex id type.V
- the vertex value type.E
- the edge value type.partitionOwner
- the owner of the Partition this data
will eventually belong to.vertex
- the vertex to extract counts from.public long getMaxEdgesPerTransfer()
public long getMaxVerticesPerTransfer()
public long getTotalEdges()
public long getTotalVertices()
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.