org.apache.giraph.worker

Class InputSplitsCallable<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.worker.InputSplitsCallable<I,V,E>

Type Parameters:

I - Vertex index value

V - Vertex value

E - Edge value

All Implemented Interfaces:

Callable<VertexEdgeCount>

Direct Known Subclasses:

EdgeInputSplitsCallable, MappingInputSplitsCallable, VertexInputSplitsCallable

public abstract class InputSplitsCallable<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends Object
implements Callable<VertexEdgeCount>

Abstract base class for loading vertex/edge input splits. Every thread will has its own instance of WorkerClientRequestProcessor to send requests.

Field Summary

Fields

Modifier and Type	Field and Description
protected ImmutableClassesGiraphConfiguration<I,V,E>	configuration Configuration
protected org.apache.hadoop.mapreduce.Mapper.Context	context Context
protected WorkerClientRequestProcessor<I,V,E>	workerClientRequestProcessor Handles IPC communication

Constructor Summary

Constructors

Constructor and Description
InputSplitsCallable(org.apache.hadoop.mapreduce.Mapper.Context context, ImmutableClassesGiraphConfiguration<I,V,E> configuration, BspServiceWorker<I,V,E> bspServiceWorker, WorkerInputSplitsHandler splitsHandler) Constructor.

Method Summary

Modifier and Type	Method and Description
VertexEdgeCount	call()
abstract GiraphInputFormat	getInputFormat() Get input format
protected org.apache.hadoop.mapreduce.InputSplit	getInputSplit(byte[] serializedInputSplit) Talk to ZooKeeper to convert the input split path to the actual InputSplit.
abstract InputType	getInputType() Get input type
static com.yammer.metrics.core.Counter	getTotalEdgesFilteredCounter() Get Counter tracking edges filtered
static com.yammer.metrics.core.Meter	getTotalEdgesLoadedMeter() Get Meter tracking edges loaded
static com.yammer.metrics.core.Counter	getTotalVerticesFilteredCounter() Get Counter tracking vertices filtered
static com.yammer.metrics.core.Meter	getTotalVerticesLoadedMeter() Get Meter tracking number of vertices loaded.
static void	initMetrics() Initialize metrics used by this class and its subclasses.
protected abstract VertexEdgeCount	readInputSplit(org.apache.hadoop.mapreduce.InputSplit inputSplit) Load vertices/edges from the given input split.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

configuration

protected final ImmutableClassesGiraphConfiguration<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> configuration

Configuration

context

protected final org.apache.hadoop.mapreduce.Mapper.Context context

Context

workerClientRequestProcessor

protected final WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> workerClientRequestProcessor

Handles IPC communication

Constructor Detail

InputSplitsCallable

public InputSplitsCallable(org.apache.hadoop.mapreduce.Mapper.Context context,
                           ImmutableClassesGiraphConfiguration<I,V,E> configuration,
                           BspServiceWorker<I,V,E> bspServiceWorker,
                           WorkerInputSplitsHandler splitsHandler)

Constructor.

Parameters:

context - Context

configuration - Configuration

bspServiceWorker - service worker

splitsHandler - Handler for input splits

Method Detail

getInputFormat

public abstract GiraphInputFormat getInputFormat()

Get input format

Returns:

Input format

getInputType

public abstract InputType getInputType()

Get input type

Returns:

Input type

getTotalEdgesLoadedMeter

public static com.yammer.metrics.core.Meter getTotalEdgesLoadedMeter()

Get Meter tracking edges loaded

Returns:

Meter tracking edges loaded

getTotalEdgesFilteredCounter

public static com.yammer.metrics.core.Counter getTotalEdgesFilteredCounter()

Get Counter tracking edges filtered

Returns:

Counter tracking edges filtered

getTotalVerticesLoadedMeter

public static com.yammer.metrics.core.Meter getTotalVerticesLoadedMeter()

Get Meter tracking number of vertices loaded.

Returns:

Meter for vertices loaded

getTotalVerticesFilteredCounter

public static com.yammer.metrics.core.Counter getTotalVerticesFilteredCounter()

Get Counter tracking vertices filtered

Returns:

Counter tracking vertices filtered

initMetrics

public static void initMetrics()

Initialize metrics used by this class and its subclasses.

readInputSplit

protected abstract VertexEdgeCount readInputSplit(org.apache.hadoop.mapreduce.InputSplit inputSplit)
                                           throws IOException,
                                                  InterruptedException

Load vertices/edges from the given input split.

Parameters:

inputSplit - Input split to load

Returns:

Count of vertices and edges loaded

Throws:

IOException

InterruptedException

call

public VertexEdgeCount call()

Specified by:

call in interface Callable<VertexEdgeCount>

getInputSplit

protected org.apache.hadoop.mapreduce.InputSplit getInputSplit(byte[] serializedInputSplit)
                                                        throws IOException,
                                                               ClassNotFoundException

Talk to ZooKeeper to convert the input split path to the actual InputSplit.

Parameters:

serializedInputSplit - Serialized input split

Returns:

instance of InputSplit

Throws:

IOException

ClassNotFoundException