Package | Description |
---|---|
org.apache.giraph.benchmark |
Package of benchmarks for performance testing and optimization
|
org.apache.giraph.bsp.checkpoints |
Package of generic bulk synchronous processing objects.
|
org.apache.giraph.comm.netty.handler |
Package for classes which handle requests and responses.
|
org.apache.giraph.conf |
Package of Giraph configuration related things.
|
org.apache.giraph.factories |
Package of factories for creating types, for example the user's graph types.
|
org.apache.giraph.graph |
Package of all the graph related objects, built on the
org.apache.bsp package.
|
org.apache.giraph.io.formats |
Package of reusable library Giraph objects.
|
org.apache.giraph.job |
Package of all the classes involved in job launch.
|
org.apache.giraph.jython.factories |
Jython factories.
|
org.apache.giraph.metrics |
Package of all the Giraph metrics related data.
|
org.apache.giraph.utils |
Package of all generic utility classes.
|
org.apache.giraph.worker |
Package of all the worker related things
|
org.apache.giraph.yarn |
Catch all package for YARN-specific code.
|
Modifier and Type | Method and Description |
---|---|
protected void |
ShortestPathsBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected void |
WeightedPageRankBenchmark.prepareConfiguration(GiraphConfiguration configuration,
org.apache.commons.cli.CommandLine cmd)
Set vertex edges, input format, partitioner classes and related parameters
based on command-line arguments.
|
protected void |
RandomMessageBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected void |
ReducersBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected void |
PageRankBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected abstract void |
GiraphBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd)
Process options from CommandLine and prepare configuration for running
the job.
|
protected void |
AggregatorsBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
Modifier and Type | Method and Description |
---|---|
boolean |
DefaultCheckpointSupportedChecker.isCheckpointSupported(GiraphConfiguration conf,
MasterCompute masterCompute) |
boolean |
DisabledCheckpointSupportedChecker.isCheckpointSupported(GiraphConfiguration conf,
MasterCompute masterCompute) |
boolean |
CheckpointSupportedChecker.isCheckpointSupported(GiraphConfiguration conf,
MasterCompute masterCompute)
Does the job support checkpoints?
It is true by default, set it to false if your job uses some
non-checkpointable features:
- static variables for storing data between supersteps.
|
Constructor and Description |
---|
RequestEncoder(GiraphConfiguration conf)
Constructor.
|
Modifier and Type | Class and Description |
---|---|
class |
ImmutableClassesGiraphConfiguration<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
The classes set here are immutable, the remaining configuration is mutable.
|
Modifier and Type | Method and Description |
---|---|
void |
FacebookConfiguration.configure(GiraphConfiguration conf)
Expand GiraphConfiguration with default Facebook settings.
|
void |
BulkConfigurator.configure(GiraphConfiguration conf)
Modify given configuration.
|
Modifier and Type | Method and Description |
---|---|
String |
DefaultComputationFactory.computationName(GiraphConfiguration conf) |
String |
ComputationFactory.computationName(GiraphConfiguration conf)
Get name of this particular computation
|
Modifier and Type | Method and Description |
---|---|
void |
JobProgressTrackerClientNoOp.init(GiraphConfiguration conf) |
void |
RetryableJobProgressTrackerClient.init(GiraphConfiguration conf) |
void |
JobProgressTrackerClient.init(GiraphConfiguration conf)
Initialize the client.
|
Constructor and Description |
---|
RetryableJobProgressTrackerClient(GiraphConfiguration conf)
Constructor
|
Modifier and Type | Method and Description |
---|---|
static void |
InMemoryVertexOutputFormat.initializeOutputGraph(GiraphConfiguration conf)
Initialize this output format - needs to be called before running the
application.
|
Modifier and Type | Method and Description |
---|---|
GiraphConfiguration |
GiraphJob.getConfiguration()
Get the configuration from the internal job.
|
Modifier and Type | Method and Description |
---|---|
protected boolean |
DefaultJobProgressTrackerService.canFinishInTime(GiraphConfiguration conf,
org.apache.hadoop.mapreduce.Job job,
CombinedWorkerProgress progress)
Determine if the job will finish in allotted time
|
static JobProgressTrackerService |
DefaultJobProgressTrackerService.createJobProgressTrackerService(GiraphConfiguration conf,
GiraphJobObserver jobObserver)
Create job progress server on job client if enabled in configuration.
|
void |
DefaultJobProgressTrackerService.init(GiraphConfiguration conf,
GiraphJobObserver jobObserver) |
void |
JobProgressTrackerService.init(GiraphConfiguration conf,
GiraphJobObserver jobObserver)
Initialize the service
|
static void |
HaltApplicationUtils.printHaltInfo(org.apache.hadoop.mapreduce.Job submittedJob,
GiraphConfiguration conf)
Wait for halt info to become available and print instructions on how to
halt
|
Constructor and Description |
---|
ClientThriftServer(GiraphConfiguration conf,
List<?> services)
Create and start the Thrift server.
|
GiraphJob(GiraphConfiguration giraphConfiguration,
String jobName)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
String |
JythonComputationFactory.computationName(GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
static SuperstepMetricsRegistry |
SuperstepMetricsRegistry.create(GiraphConfiguration conf,
long superstep)
Create with Hadoop Configuration and superstep number.
|
static GiraphMetricsRegistry |
GiraphMetricsRegistry.create(GiraphConfiguration conf,
String groupName,
String type)
Create registry with Hadoop Configuration and group to use for metrics.
|
static void |
GiraphMetrics.init(GiraphConfiguration conf)
Initialize singleton instance of GiraphMetrics.
|
Modifier and Type | Method and Description |
---|---|
static org.apache.commons.cli.CommandLine |
ConfigurationUtils.parseArgs(GiraphConfiguration giraphConf,
String[] args)
Translate CLI arguments to GiraphRunner or 'bin/hadoop jar' into
Configuration Key-Value pairs.
|
static Iterable<String> |
InternalVertexRunner.run(GiraphConfiguration conf,
String[] vertexInputData)
Attempts to run the vertex internally in the current JVM, reading from and
writing to a temporary folder on local disk.
|
static Iterable<String> |
InternalVertexRunner.run(GiraphConfiguration conf,
String[] vertexInputData,
String[] edgeInputData)
Attempts to run the vertex internally in the current JVM, reading from and
writing to a temporary folder on local disk.
|
static Iterable<String> |
InternalVertexRunner.run(GiraphConfiguration conf,
String[] vertexInputData,
String[] edgeInputData,
String checkpointsDir,
File tmpDir)
Attempts to run the vertex internally in the current JVM, reading from and
writing to a temporary folder on local disk.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.run(GiraphConfiguration conf,
TestGraph<I,V,E> graph)
Attempts to run the vertex internally in the current JVM,
reading from an in-memory graph.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.run(GiraphConfiguration conf,
TestGraph<I,V,E> graph,
File tmpDir,
String checkpointsDir)
Attempts to run the vertex internally in the current JVM,
reading from an in-memory graph.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.runWithInMemoryOutput(GiraphConfiguration conf,
TestGraph<I,V,E> graph)
Attempts to run the vertex internally in the current JVM, reading and
writing to an in-memory graph.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.runWithInMemoryOutput(GiraphConfiguration conf,
TestGraph<I,V,E> graph,
File tmpDir,
String checkpointsDir)
Attempts to run the vertex internally in the current JVM, reading and
writing to an in-memory graph.
|
Constructor and Description |
---|
TestGraph(GiraphConfiguration conf)
Constructor requiring classes
|
Constructor and Description |
---|
MemoryObserver(ZooKeeperExt zk,
String zkPath,
GiraphConfiguration conf)
Constructor
|
Modifier and Type | Method and Description |
---|---|
static void |
YarnUtils.addFsResourcesToMap(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> map,
GiraphConfiguration giraphConf,
org.apache.hadoop.yarn.api.records.ApplicationId appId)
Populates the LocalResources list with the HDFS paths listed in
the conf under GiraphConstants.GIRAPH_YARN_LIBJARS, and the
GiraphConfiguration for this job.
|
static void |
YarnUtils.addGiraphConfToLocalResourceMap(GiraphConfiguration giraphConf,
org.apache.hadoop.yarn.api.records.ApplicationId appId,
Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResourceMap)
Populate the LocalResources list with the GiraphConf XML file's HDFS path.
|
static void |
YarnUtils.addLocalClasspathToEnv(Map<String,String> env,
GiraphConfiguration giraphConf)
Popuate the environment string map to be added to the environment vars
in a remote execution container.
|
static void |
YarnUtils.exportGiraphConfiguration(GiraphConfiguration giraphConf,
org.apache.hadoop.yarn.api.records.ApplicationId appId)
Export our populated GiraphConfiguration as an XML file to be used by the
ApplicationMaster's exec container, and register it with LocalResources.
|
Constructor and Description |
---|
GiraphYarnClient(GiraphConfiguration giraphConf,
String jobName)
Constructor.
|
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.