GraphTaskManager (Apache Giraph Parent 1.4.0-SNAPSHOT API)

java.lang.Object
- org.apache.giraph.graph.GraphTaskManager<I,V,E>

Type Parameters:

I - Vertex id

V - Vertex data

E - Edge data

All Implemented Interfaces:

ResetSuperstepMetricsObserver
```
public class GraphTaskManager<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends Object
implements ResetSuperstepMetricsObserver
```
The Giraph-specific business logic for a single BSP compute node in whatever underlying type of cluster our Giraph job will run on. Owning object will provide the glue into the underlying cluster framework and will call this object to perform Giraph work.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static interface`	`GraphTaskManager.CheckerIfWorkerShouldFailAfterException` Interface to check if worker should fail after a thread gets an exception
`static class`	`GraphTaskManager.FailWithEveryExceptionOccurred` Class to use by default, where each exception causes job failure

Field Summary

Fields
Modifier and Type	Field and Description
`static ClassConfOption<GraphTaskManager.CheckerIfWorkerShouldFailAfterException>`	`CHECKER_IF_WORKER_SHOULD_FAIL_AFTER_EXCEPTION_CLASS` Class which checks if an exception on some thread should cause worker to fail
`static String`	`TIMER_COMMUNICATION_TIME` Name of metric for time from first message till last message flushed
`static String`	`TIMER_COMPUTE_ALL` Name of metric for compute on all vertices in msec
`static String`	`TIMER_SUPERSTEP_GC_TIME` Name of metric for time spent doing GC per superstep in msec
`static String`	`TIMER_SUPERSTEP_TIME` Name of metric for superstep time in msec
`static String`	`TIMER_TIME_TO_FIRST_MSG` Name of metric for time from begin compute to first message sent

Constructor Summary

Constructors
Constructor and Description

GraphTaskManager(org.apache.hadoop.mapreduce.Mapper.Context context)
Default constructor for GiraphTaskManager.

Constructors
Constructor and Description
`GraphTaskManager(org.apache.hadoop.mapreduce.Mapper.Context context)` Default constructor for GiraphTaskManager.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`cleanup()` Called by owner of this GraphTaskManager object on each compute node
`Thread.UncaughtExceptionHandler`	`createUncaughtExceptionHandler()` Creates exception handler that will terminate process gracefully in case of any uncaught exception.
`Thread.UncaughtExceptionHandler`	`createUncaughtExceptionHandler(GraphTaskManager.CheckerIfWorkerShouldFailAfterException checker)` Creates exception handler with the passed implementation of `GraphTaskManager.CheckerIfWorkerShouldFailAfterException`.
`void`	`execute()` Perform the work assigned to this compute node for this job run.
`ImmutableClassesGiraphConfiguration<I,V,E>`	`getConf()`
`GraphFunctions`	`getGraphFunctions()` Get the worker function enum.
`JobProgressTracker`	`getJobProgressTracker()`
`long`	`getSuperstepGCTime()`
`WorkerContext`	`getWorkerContext()`
`String`	`getZookeeperList()` Returns a list of zookeeper servers to connect to.
`static boolean`	`isConnectionResetByPeer(Throwable throwable)` Checks the message of a throwable, and checks whether it is a "connection reset by peer" type of exception.
`boolean`	`isMaster()` Get "isMaster" status flag -- we need to know if we're the master in the "finally" block of our GiraphYarnTask#execute() to commit final job output.
`void`	`newSuperstep(SuperstepMetricsRegistry superstepMetrics)` Starting a new superstep.
`void`	`notifyFinishedCommunication()` Notification of last message flushed.
`void`	`notifySentMessages()` Notification from Vertex that a message has been sent.
`void`	`sendWorkerCountersAndFinishCleanup()` Method to send the counter values from the worker to the master, after all supersteps are done, and finish cleanup
`void`	`setIsMaster(boolean im)` Sets the "isMaster" flag for final output commit to happen on master.
`void`	`setup(org.apache.hadoop.fs.Path[] zkPathList)` Called by owner of this GraphTaskManager on each compute node
`void`	`setupMapperObservers()` Setup mapper observers
`void`	`workerFailureCleanup()` Cleanup all of Giraph's framework-agnostic resources regardless of which type of cluster Giraph is running on.
`void`	`zooKeeperCleanup()` Cleanup a ZooKeeper instance managed by this GiraphWorker upon job run failure.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - CHECKER_IF_WORKER_SHOULD_FAIL_AFTER_EXCEPTION_CLASS
```
public static final ClassConfOption<GraphTaskManager.CheckerIfWorkerShouldFailAfterException> CHECKER_IF_WORKER_SHOULD_FAIL_AFTER_EXCEPTION_CLASS
```
    Class which checks if an exception on some thread should cause worker to fail
  - TIMER_SUPERSTEP_TIME
```
public static final String TIMER_SUPERSTEP_TIME
```
    Name of metric for superstep time in msec
    
    See Also:
    
    Constant Field Values
  - TIMER_COMPUTE_ALL
```
public static final String TIMER_COMPUTE_ALL
```
    Name of metric for compute on all vertices in msec
    
    See Also:
    
    Constant Field Values
  - TIMER_TIME_TO_FIRST_MSG
```
public static final String TIMER_TIME_TO_FIRST_MSG
```
    Name of metric for time from begin compute to first message sent
    
    See Also:
    
    Constant Field Values
  - TIMER_COMMUNICATION_TIME
```
public static final String TIMER_COMMUNICATION_TIME
```
    Name of metric for time from first message till last message flushed
    
    See Also:
    
    Constant Field Values
  - TIMER_SUPERSTEP_GC_TIME
```
public static final String TIMER_SUPERSTEP_GC_TIME
```
    Name of metric for time spent doing GC per superstep in msec
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - GraphTaskManager
```
public GraphTaskManager(org.apache.hadoop.mapreduce.Mapper.Context context)
```
    Default constructor for GiraphTaskManager.
    
    Parameters:
    
    context - a handle to the underlying cluster framework. For Hadoop clusters, this is a Mapper#Context.
- Method Detail
  - setup
```
public void setup(org.apache.hadoop.fs.Path[] zkPathList)
           throws IOException,
                  InterruptedException
```
    Called by owner of this GraphTaskManager on each compute node
    
    Parameters:
    
    zkPathList - the path to the ZK jars we need to run the job
    
    Throws:
    
    IOException
    
    InterruptedException
  - execute
```
public void execute()
             throws IOException,
                    InterruptedException
```
    Perform the work assigned to this compute node for this job run. 1) Run checkpoint per frequency policy. 2) For every vertex on this mapper, run the compute() function 3) Wait until all messaging is done. 4) Check if all vertices are done. If not goto 2). 5) Dump output.
    
    Throws:
    
    IOException
    
    InterruptedException
  - setIsMaster
```
public void setIsMaster(boolean im)
```
    Sets the "isMaster" flag for final output commit to happen on master.
    
    Parameters:
    
    im - the boolean input to set isMaster. Applies to "pure YARN only"
  - isMaster
```
public boolean isMaster()
```
    Get "isMaster" status flag -- we need to know if we're the master in the "finally" block of our GiraphYarnTask#execute() to commit final job output.
    
    Returns:
    
    true if this task IS the master.
  - getGraphFunctions
```
public GraphFunctions getGraphFunctions()
```
    Get the worker function enum.
    
    Returns:
    
    an enum detailing the roles assigned to this compute node for this Giraph job.
  - getWorkerContext
```
public final WorkerContext getWorkerContext()
```
  - getJobProgressTracker
```
public JobProgressTracker getJobProgressTracker()
```
  - newSuperstep
```
public void newSuperstep(SuperstepMetricsRegistry superstepMetrics)
```
    Description copied from interface: ResetSuperstepMetricsObserver
    
    Starting a new superstep. Re-initialize your metrics.
    
    Specified by:
    
    newSuperstep in interface ResetSuperstepMetricsObserver
    
    Parameters:
    
    superstepMetrics - SuperstepMetricsRegistry being used.
  - notifySentMessages
```
public void notifySentMessages()
```
    Notification from Vertex that a message has been sent.
  - notifyFinishedCommunication
```
public void notifyFinishedCommunication()
```
    Notification of last message flushed. Comes when we finish the superstep and are done waiting for all messages to send.
  - setupMapperObservers
```
public void setupMapperObservers()
```
    Setup mapper observers
  - cleanup
```
public void cleanup()
             throws IOException,
                    InterruptedException
```
    Called by owner of this GraphTaskManager object on each compute node
    
    Throws:
    
    IOException
    
    InterruptedException
  - sendWorkerCountersAndFinishCleanup
```
public void sendWorkerCountersAndFinishCleanup()
```
    Method to send the counter values from the worker to the master, after all supersteps are done, and finish cleanup
  - zooKeeperCleanup
```
public void zooKeeperCleanup()
```
    Cleanup a ZooKeeper instance managed by this GiraphWorker upon job run failure.
  - workerFailureCleanup
```
public void workerFailureCleanup()
```
    Cleanup all of Giraph's framework-agnostic resources regardless of which type of cluster Giraph is running on.
  - createUncaughtExceptionHandler
```
public Thread.UncaughtExceptionHandler createUncaughtExceptionHandler()
```
    Creates exception handler that will terminate process gracefully in case of any uncaught exception.
    
    Returns:
    
    new exception handler object.
  - createUncaughtExceptionHandler
```
public Thread.UncaughtExceptionHandler createUncaughtExceptionHandler(GraphTaskManager.CheckerIfWorkerShouldFailAfterException checker)
```
    Creates exception handler with the passed implementation of GraphTaskManager.CheckerIfWorkerShouldFailAfterException.
    
    Parameters:
    
    checker - Instance that checks whether the job should fail.
    
    Returns:
    
    Exception handler.
  - getConf
```
public ImmutableClassesGiraphConfiguration<I,V,E> getConf()
```
  - getSuperstepGCTime
```
public long getSuperstepGCTime()
```
    Returns:
    
    Time spent in GC recorder by the GC listener
  - getZookeeperList
```
public String getZookeeperList()
```
    Returns a list of zookeeper servers to connect to. If the port is set to 0 and Giraph is starting a single ZooKeeper server, then Zookeeper will pick its own port. Otherwise, the ZooKeeper port set by the user will be used.
    
    Returns:
    
    host:port,host:port for each zookeeper
  - isConnectionResetByPeer
```
public static boolean isConnectionResetByPeer(Throwable throwable)
```
    Checks the message of a throwable, and checks whether it is a "connection reset by peer" type of exception.
    
    Parameters:
    
    throwable - Throwable
    
    Returns:
    
    True if the throwable is a "connection reset by peer", false otherwise.

Class GraphTaskManager<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

CHECKER_IF_WORKER_SHOULD_FAIL_AFTER_EXCEPTION_CLASS

TIMER_SUPERSTEP_TIME

TIMER_COMPUTE_ALL

TIMER_TIME_TO_FIRST_MSG

TIMER_COMMUNICATION_TIME

TIMER_SUPERSTEP_GC_TIME

Constructor Detail

GraphTaskManager

Method Detail

setup

execute

setIsMaster

isMaster

getGraphFunctions

getWorkerContext

getJobProgressTracker

newSuperstep

notifySentMessages

notifyFinishedCommunication

setupMapperObservers

cleanup

sendWorkerCountersAndFinishCleanup

zooKeeperCleanup

workerFailureCleanup

createUncaughtExceptionHandler

createUncaughtExceptionHandler

getConf

getSuperstepGCTime

getZookeeperList

isConnectionResetByPeer