Package | Description |
---|---|
org.apache.giraph.benchmark |
Package of benchmarks for performance testing and optimization
|
org.apache.giraph.block_app.framework |
Block application abstraction package.
|
org.apache.giraph.block_app.framework.internal |
Implementation of execution logic, guiding internal execution of
Block Application.
|
org.apache.giraph.block_app.framework.output |
Multi-output support for Block Applications
|
org.apache.giraph.block_app.migration |
Migration utility for transforming standard Giraph applications, into
Block Applications.
|
org.apache.giraph.block_app.test_setup |
Utilities for unit tests.
|
org.apache.giraph.bsp.checkpoints |
Package of generic bulk synchronous processing objects.
|
org.apache.giraph.comm.netty.handler |
Package for classes which handle requests and responses.
|
org.apache.giraph.conf |
Package of Giraph configuration related things.
|
org.apache.giraph.examples.block_app |
Package of Giraph examples for Block Applications.
|
org.apache.giraph.factories |
Package of factories for creating types, for example the user's graph types.
|
org.apache.giraph.graph |
Package of all the graph related objects, built on the
org.apache.bsp package.
|
org.apache.giraph.io.formats |
Package of reusable library Giraph objects.
|
org.apache.giraph.job |
Package of all the classes involved in job launch.
|
org.apache.giraph.jython.factories |
Jython factories.
|
org.apache.giraph.metrics |
Package of all the Giraph metrics related data.
|
org.apache.giraph.utils |
Package of all generic utility classes.
|
org.apache.giraph.worker |
Package of all the worker related things
|
org.apache.giraph.yarn |
Catch all package for YARN-specific code.
|
Modifier and Type | Method and Description |
---|---|
protected void |
ShortestPathsBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected void |
WeightedPageRankBenchmark.prepareConfiguration(GiraphConfiguration configuration,
org.apache.commons.cli.CommandLine cmd)
Set vertex edges, input format, partitioner classes and related parameters
based on command-line arguments.
|
protected void |
RandomMessageBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected void |
ReducersBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected void |
PageRankBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
protected abstract void |
GiraphBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd)
Process options from CommandLine and prepare configuration for running
the job.
|
protected void |
AggregatorsBenchmark.prepareConfiguration(GiraphConfiguration conf,
org.apache.commons.cli.CommandLine cmd) |
Modifier and Type | Method and Description |
---|---|
protected void |
AbstractBlockFactory.additionalInitConfig(GiraphConfiguration conf)
Additional configuration initialization, other then overriding
class specification.
|
Block |
BlockFactory.createBlock(GiraphConfiguration conf)
Create a block (representing a full Giraph job), based on the given
configuration.
|
S |
BlockFactory.createExecutionStage(GiraphConfiguration conf)
Create an empty instance of execution stage object.
|
protected abstract Class<? extends org.apache.hadoop.io.Writable> |
AbstractBlockFactory.getEdgeValueClass(GiraphConfiguration conf)
Concrete edge value class application will use.
|
protected abstract Class<? extends org.apache.hadoop.io.WritableComparable> |
AbstractBlockFactory.getVertexIDClass(GiraphConfiguration conf)
Concrete vertex id class application will use.
|
protected abstract Class<? extends org.apache.hadoop.io.Writable> |
AbstractBlockFactory.getVertexValueClass(GiraphConfiguration conf)
Concrete vertex value class application will use.
|
protected Class<?> |
AbstractBlockFactory.getWorkerContextValueClass(GiraphConfiguration conf)
Concrete worker context value class application will use, if overridden.
|
static void |
BlockUtils.initAndCheckConfig(GiraphConfiguration conf)
Initializes configuration, such that running it executes block application.
|
void |
BlockFactory.initConfig(GiraphConfiguration conf)
Based on provided configuration, updates it, such that all necessary
properties are initialized.
|
void |
AbstractBlockFactory.initConfig(GiraphConfiguration conf) |
void |
BlockFactory.registerOutputs(GiraphConfiguration conf)
Register outputs to use during the application (vs output at the end of
the application), based on provided configuration.
|
void |
AbstractBlockFactory.registerOutputs(GiraphConfiguration conf) |
static void |
BlockUtils.setAndInitBlockFactoryClass(GiraphConfiguration conf,
Class<? extends BlockFactory<?>> clazz)
Set block factory, and initialize configs with it.
|
protected boolean |
AbstractBlockFactory.shouldCreateVertexOnMsgs(GiraphConfiguration conf)
Override if vertices shouldn't be created by default, if message is sent
to a vertex that doesn't exist.
|
protected boolean |
AbstractBlockFactory.shouldSendOneMessageToAll(GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
void |
BlockMasterLogic.initialize(GiraphConfiguration conf,
BlockMasterApi masterApi)
Initialize master logic to execute BlockFactory defined in
the configuration.
|
Modifier and Type | Method and Description |
---|---|
static <OD> void |
BlockOutputFormat.addOutputDesc(OD outputDesc,
String confOption,
GiraphConfiguration conf) |
void |
BlockOutputOption.register(OD outputDesc,
GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
MigrationSuperstepStage |
MigrationFullBlockFactory.createExecutionStage(GiraphConfiguration conf) |
<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,MR extends org.apache.hadoop.io.Writable,MS extends org.apache.hadoop.io.Writable> |
MigrationFullBlockFactory.createMigrationAppBlock(Class<? extends MigrationAbstractComputation.MigrationFullAbstractComputation<I,V,E,MR,MS>> computationClass,
MigrationMasterCompute.MigrationFullMasterCompute masterCompute,
Class<MS> messageClass,
Class<? extends MessageCombiner<? super I,MS>> messageCombinerClass,
GiraphConfiguration conf) |
protected Class<? extends MigrationWorkerContext> |
MigrationFullBlockFactory.getWorkerContextValueClass(GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,E extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
TestGraphUtils.runTest(TestGraphModifier<? super I,? super V,? super E> graphInitializer,
TestGraphChecker<? super I,? super V,? super E> graphChecker,
GiraphConfiguration conf)
Uses provided configuration, initializes the graph using
graphInitializer, and checks it via graphChecker.
|
Constructor and Description |
---|
NumericTestGraph(GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
boolean |
DefaultCheckpointSupportedChecker.isCheckpointSupported(GiraphConfiguration conf,
MasterCompute masterCompute) |
boolean |
DisabledCheckpointSupportedChecker.isCheckpointSupported(GiraphConfiguration conf,
MasterCompute masterCompute) |
boolean |
CheckpointSupportedChecker.isCheckpointSupported(GiraphConfiguration conf,
MasterCompute masterCompute)
Does the job support checkpoints?
It is true by default, set it to false if your job uses some
non-checkpointable features:
- static variables for storing data between supersteps.
|
Constructor and Description |
---|
RequestEncoder(GiraphConfiguration conf)
Constructor.
|
Modifier and Type | Class and Description |
---|---|
class |
ImmutableClassesGiraphConfiguration<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
The classes set here are immutable, the remaining configuration is mutable.
|
Modifier and Type | Method and Description |
---|---|
void |
FacebookConfiguration.configure(GiraphConfiguration conf)
Expand GiraphConfiguration with default Facebook settings.
|
void |
BulkConfigurator.configure(GiraphConfiguration conf)
Modify given configuration.
|
Modifier and Type | Method and Description |
---|---|
Block |
SimpleMigrationMasterBlockFactory.createBlock(GiraphConfiguration conf) |
protected Class<org.apache.hadoop.io.FloatWritable> |
SimpleMigrationMasterBlockFactory.getEdgeValueClass(GiraphConfiguration conf) |
protected Class<org.apache.hadoop.io.LongWritable> |
SimpleMigrationMasterBlockFactory.getVertexIDClass(GiraphConfiguration conf) |
protected Class<org.apache.hadoop.io.DoubleWritable> |
SimpleMigrationMasterBlockFactory.getVertexValueClass(GiraphConfiguration conf) |
protected Class<SimpleMigrationMasterBlockFactory.SimpleMigrationMasterWorkerContext> |
SimpleMigrationMasterBlockFactory.getWorkerContextValueClass(GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
String |
DefaultComputationFactory.computationName(GiraphConfiguration conf) |
String |
ComputationFactory.computationName(GiraphConfiguration conf)
Get name of this particular computation
|
Modifier and Type | Method and Description |
---|---|
void |
JobProgressTrackerClientNoOp.init(GiraphConfiguration conf) |
void |
RetryableJobProgressTrackerClient.init(GiraphConfiguration conf) |
void |
JobProgressTrackerClient.init(GiraphConfiguration conf)
Initialize the client.
|
Constructor and Description |
---|
RetryableJobProgressTrackerClient(GiraphConfiguration conf)
Constructor
|
Modifier and Type | Method and Description |
---|---|
static void |
InMemoryVertexOutputFormat.initializeOutputGraph(GiraphConfiguration conf)
Initialize this output format - needs to be called before running the
application.
|
Modifier and Type | Method and Description |
---|---|
GiraphConfiguration |
GiraphJob.getConfiguration()
Get the configuration from the internal job.
|
Modifier and Type | Method and Description |
---|---|
protected boolean |
DefaultJobProgressTrackerService.canFinishInTime(GiraphConfiguration conf,
org.apache.hadoop.mapreduce.Job job,
CombinedWorkerProgress progress)
Determine if the job will finish in allotted time
|
static JobProgressTrackerService |
DefaultJobProgressTrackerService.createJobProgressTrackerService(GiraphConfiguration conf,
GiraphJobObserver jobObserver)
Create job progress server on job client if enabled in configuration.
|
void |
DefaultJobProgressTrackerService.init(GiraphConfiguration conf,
GiraphJobObserver jobObserver) |
void |
JobProgressTrackerService.init(GiraphConfiguration conf,
GiraphJobObserver jobObserver)
Initialize the service
|
static void |
HaltApplicationUtils.printHaltInfo(org.apache.hadoop.mapreduce.Job submittedJob,
GiraphConfiguration conf)
Wait for halt info to become available and print instructions on how to
halt
|
Constructor and Description |
---|
ClientThriftServer(GiraphConfiguration conf,
List<?> services)
Create and start the Thrift server.
|
GiraphJob(GiraphConfiguration giraphConfiguration,
String jobName)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
String |
JythonComputationFactory.computationName(GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
static SuperstepMetricsRegistry |
SuperstepMetricsRegistry.create(GiraphConfiguration conf,
long superstep)
Create with Hadoop Configuration and superstep number.
|
static GiraphMetricsRegistry |
GiraphMetricsRegistry.create(GiraphConfiguration conf,
String groupName,
String type)
Create registry with Hadoop Configuration and group to use for metrics.
|
static void |
GiraphMetrics.init(GiraphConfiguration conf)
Initialize singleton instance of GiraphMetrics.
|
Modifier and Type | Method and Description |
---|---|
static org.apache.commons.cli.CommandLine |
ConfigurationUtils.parseArgs(GiraphConfiguration giraphConf,
String[] args)
Translate CLI arguments to GiraphRunner or 'bin/hadoop jar' into
Configuration Key-Value pairs.
|
static Iterable<String> |
InternalVertexRunner.run(GiraphConfiguration conf,
String[] vertexInputData)
Attempts to run the vertex internally in the current JVM, reading from and
writing to a temporary folder on local disk.
|
static Iterable<String> |
InternalVertexRunner.run(GiraphConfiguration conf,
String[] vertexInputData,
String[] edgeInputData)
Attempts to run the vertex internally in the current JVM, reading from and
writing to a temporary folder on local disk.
|
static Iterable<String> |
InternalVertexRunner.run(GiraphConfiguration conf,
String[] vertexInputData,
String[] edgeInputData,
String checkpointsDir,
File tmpDir)
Attempts to run the vertex internally in the current JVM, reading from and
writing to a temporary folder on local disk.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.run(GiraphConfiguration conf,
TestGraph<I,V,E> graph)
Attempts to run the vertex internally in the current JVM,
reading from an in-memory graph.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.run(GiraphConfiguration conf,
TestGraph<I,V,E> graph,
File tmpDir,
String checkpointsDir)
Attempts to run the vertex internally in the current JVM,
reading from an in-memory graph.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.runWithInMemoryOutput(GiraphConfiguration conf,
TestGraph<I,V,E> graph)
Attempts to run the vertex internally in the current JVM, reading and
writing to an in-memory graph.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
InternalVertexRunner.runWithInMemoryOutput(GiraphConfiguration conf,
TestGraph<I,V,E> graph,
File tmpDir,
String checkpointsDir)
Attempts to run the vertex internally in the current JVM, reading and
writing to an in-memory graph.
|
Constructor and Description |
---|
TestGraph(GiraphConfiguration conf)
Constructor requiring classes
|
Constructor and Description |
---|
MemoryObserver(ZooKeeperExt zk,
String zkPath,
GiraphConfiguration conf)
Constructor
|
Modifier and Type | Method and Description |
---|---|
static void |
YarnUtils.addFsResourcesToMap(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> map,
GiraphConfiguration giraphConf,
org.apache.hadoop.yarn.api.records.ApplicationId appId)
Populates the LocalResources list with the HDFS paths listed in
the conf under GiraphConstants.GIRAPH_YARN_LIBJARS, and the
GiraphConfiguration for this job.
|
static void |
YarnUtils.addGiraphConfToLocalResourceMap(GiraphConfiguration giraphConf,
org.apache.hadoop.yarn.api.records.ApplicationId appId,
Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResourceMap)
Populate the LocalResources list with the GiraphConf XML file's HDFS path.
|
static void |
YarnUtils.addLocalClasspathToEnv(Map<String,String> env,
GiraphConfiguration giraphConf)
Popuate the environment string map to be added to the environment vars
in a remote execution container.
|
static void |
YarnUtils.exportGiraphConfiguration(GiraphConfiguration giraphConf,
org.apache.hadoop.yarn.api.records.ApplicationId appId)
Export our populated GiraphConfiguration as an XML file to be used by the
ApplicationMaster's exec container, and register it with LocalResources.
|
Constructor and Description |
---|
GiraphYarnClient(GiraphConfiguration giraphConf,
String jobName)
Constructor.
|
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.