Package | Description |
---|---|
org.apache.giraph.bsp |
Package of generic bulk synchronous processing objects.
|
org.apache.giraph.comm |
Package of communication related objects, IPC service.
|
org.apache.giraph.comm.netty |
Package for netty implementations.
|
org.apache.giraph.graph |
Package of all the graph related objects, built on the
org.apache.bsp package.
|
org.apache.giraph.partition |
Package of partitioning related objects.
|
org.apache.giraph.worker |
Package of all the worker related things
|
Modifier and Type | Method and Description |
---|---|
PartitionOwner |
CentralizedServiceWorker.getVertexPartitionOwner(I vertexId)
Every client will need to get a partition owner from a vertex id so that
they know which worker to sent the request to.
|
Modifier and Type | Method and Description |
---|---|
Iterable<? extends PartitionOwner> |
CentralizedServiceWorker.getPartitionOwners()
Get all partition owners.
|
Collection<? extends PartitionOwner> |
CentralizedServiceWorker.startSuperstep()
Take all steps prior to actually beginning the computation of a
superstep.
|
Modifier and Type | Method and Description |
---|---|
void |
CentralizedServiceWorker.exchangeVertexPartitions(Collection<? extends PartitionOwner> masterSetPartitionOwners)
If desired by the user, vertex partitions are redistributed among
workers according to the chosen WorkerGraphPartitioner.
|
Modifier and Type | Method and Description |
---|---|
PartitionOwner |
WorkerClient.getVertexPartitionOwner(I vertexId)
Lookup PartitionOwner for a vertex.
|
Modifier and Type | Method and Description |
---|---|
int |
SendPartitionCache.addVertex(PartitionOwner partitionOwner,
Vertex<I,V,E> vertex)
Add a vertex to the cache.
|
boolean |
WorkerClientRequestProcessor.sendVertexRequest(PartitionOwner partitionOwner,
Vertex<I,V,E> vertex)
Sends a vertex to the appropriate partition owner
|
Modifier and Type | Method and Description |
---|---|
PartitionOwner |
NettyWorkerClient.getVertexPartitionOwner(I vertexId) |
Modifier and Type | Method and Description |
---|---|
boolean |
NettyWorkerClientRequestProcessor.sendVertexRequest(PartitionOwner partitionOwner,
Vertex<I,V,E> vertex) |
Modifier and Type | Method and Description |
---|---|
Collection<PartitionOwner> |
AddressesAndPartitionsWritable.getPartitionOwners()
Get partition owners
|
Modifier and Type | Method and Description |
---|---|
<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
GiraphTransferRegulator.incrementCounters(PartitionOwner partitionOwner,
Vertex<I,V,E> vertex)
Increment V & E counts for new vertex read, store values
for that outgoing _temporary_ Partition, which shares the
Partition ID for the actual remote Partition the collection
will eventually be processed in.
|
boolean |
GiraphTransferRegulator.transferThisPartition(PartitionOwner owner)
Is this outbound data Collection full,
and ready to transfer?
|
Constructor and Description |
---|
AddressesAndPartitionsWritable(MasterInfo masterInfo,
List<WorkerInfo> workerInfos,
Collection<PartitionOwner> partitionOwners)
Constructor when we want to serialize object
|
Modifier and Type | Class and Description |
---|---|
class |
BasicPartitionOwner
Basic partition owner, can be subclassed for more complicated partition
owner implementations.
|
Modifier and Type | Method and Description |
---|---|
PartitionOwner |
WorkerGraphPartitionerImpl.createPartitionOwner() |
PartitionOwner |
WorkerGraphPartitioner.createPartitionOwner()
Instantiate the
PartitionOwner implementation used to read the
master assignments. |
PartitionOwner |
WorkerGraphPartitionerImpl.getPartitionOwner(I vertexId) |
PartitionOwner |
WorkerGraphPartitioner.getPartitionOwner(I vertexId)
Figure out the owner of a vertex
|
Modifier and Type | Method and Description |
---|---|
static Collection<PartitionOwner> |
PartitionBalancer.balancePartitionsAcrossWorkers(org.apache.hadoop.conf.Configuration conf,
Collection<PartitionOwner> partitionOwners,
Collection<PartitionStats> allPartitionStats,
Collection<WorkerInfo> availableWorkerInfos)
Balance the partitions with an algorithm based on a value.
|
Collection<PartitionOwner> |
MasterGraphPartitionerImpl.createInitialPartitionOwners(Collection<WorkerInfo> availableWorkerInfos,
int maxWorkers) |
Collection<PartitionOwner> |
MasterGraphPartitioner.createInitialPartitionOwners(Collection<WorkerInfo> availableWorkerInfos,
int maxWorkers)
Set some initial partition owners for the graph.
|
Collection<PartitionOwner> |
MasterGraphPartitionerImpl.generateChangedPartitionOwners(Collection<PartitionStats> allPartitionStatsList,
Collection<WorkerInfo> availableWorkers,
int maxWorkers,
long superstep) |
Collection<PartitionOwner> |
MasterGraphPartitioner.generateChangedPartitionOwners(Collection<PartitionStats> allPartitionStatsList,
Collection<WorkerInfo> availableWorkers,
int maxWorkers,
long superstep)
After the worker stats have been merged to a single list, the master can
use this information to send commands to the workers for any
Partition changes. |
Collection<PartitionOwner> |
MasterGraphPartitionerImpl.getCurrentPartitionOwners() |
Collection<PartitionOwner> |
MasterGraphPartitioner.getCurrentPartitionOwners()
Get current partition owners at this time.
|
Collection<? extends PartitionOwner> |
WorkerGraphPartitionerImpl.getPartitionOwners() |
Collection<? extends PartitionOwner> |
WorkerGraphPartitioner.getPartitionOwners()
Get a collection of the
PartitionOwner objects. |
Modifier and Type | Method and Description |
---|---|
static void |
PartitionUtils.analyzePartitionStats(Collection<PartitionOwner> partitionOwnerList,
List<PartitionStats> allPartitionStats)
Check for imbalances on a per worker basis, by calculating the
mean, high and low workers by edges and vertices.
|
static Collection<PartitionOwner> |
PartitionBalancer.balancePartitionsAcrossWorkers(org.apache.hadoop.conf.Configuration conf,
Collection<PartitionOwner> partitionOwners,
Collection<PartitionStats> allPartitionStats,
Collection<WorkerInfo> availableWorkerInfos)
Balance the partitions with an algorithm based on a value.
|
void |
MasterGraphPartitionerImpl.setPartitionOwners(Collection<PartitionOwner> partitionOwners) |
void |
MasterGraphPartitioner.setPartitionOwners(Collection<PartitionOwner> partitionOwners)
Sets partition owners for the graph.
|
static PartitionExchange |
PartitionBalancer.updatePartitionOwners(List<PartitionOwner> partitionOwnerList,
WorkerInfo myWorkerInfo,
Collection<? extends PartitionOwner> masterSetPartitionOwners)
Helper function to update partition owners and determine which
partitions need to be sent from a specific worker.
|
static PartitionExchange |
PartitionBalancer.updatePartitionOwners(List<PartitionOwner> partitionOwnerList,
WorkerInfo myWorkerInfo,
Collection<? extends PartitionOwner> masterSetPartitionOwners)
Helper function to update partition owners and determine which
partitions need to be sent from a specific worker.
|
PartitionExchange |
WorkerGraphPartitionerImpl.updatePartitionOwners(WorkerInfo myWorkerInfo,
Collection<? extends PartitionOwner> masterSetPartitionOwners) |
PartitionExchange |
WorkerGraphPartitioner.updatePartitionOwners(WorkerInfo myWorkerInfo,
Collection<? extends PartitionOwner> masterSetPartitionOwners)
Get the partitions owners and update locally.
|
Modifier and Type | Method and Description |
---|---|
PartitionOwner |
BspServiceWorker.getVertexPartitionOwner(I vertexId) |
Modifier and Type | Method and Description |
---|---|
Iterable<? extends PartitionOwner> |
BspServiceWorker.getPartitionOwners() |
Collection<? extends PartitionOwner> |
BspServiceWorker.startSuperstep() |
Modifier and Type | Method and Description |
---|---|
void |
BspServiceWorker.exchangeVertexPartitions(Collection<? extends PartitionOwner> masterSetPartitionOwners) |
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.