Package | Description |
---|---|
org.apache.giraph.block_app.library |
Core library of Pieces and Suppliers, providing most common usages.
|
org.apache.giraph.block_app.library.internal |
Internal implementation of Pieces needed for Pieces utility class.
|
org.apache.giraph.block_app.library.iteration |
Utilities for having iteration within execution stage object.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.forAllVertices(String pieceName,
Consumer<Vertex<I,V,E>> process)
For each vertex execute given process function.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.forAllVerticesOnReceive(String pieceName,
Consumer<Vertex<I,V,E>> process)
For each vertex execute given process function.
|
static Piece<org.apache.hadoop.io.WritableComparable,org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable,NoMessage,Object> |
Pieces.masterCompute(String pieceName,
Consumer<BlockMasterApi> process)
Execute given function on master.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduce(String name,
ReduceOperation<S,R> reduceOp,
SupplierFromVertex<I,V,E,S> valueSupplier,
Consumer<R> reducedValueConsumer)
Creates single reducer piece - given reduce class, supplier of values on
worker, reduces and passes the result to given consumer on master.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduceAndBroadcast(String name,
ReduceOperation<S,R> reduceOp,
SupplierFromVertex<I,V,E,S> valueSupplier,
ConsumerWithVertex<I,V,E,R> reducedValueConsumer)
Creates single reducer and broadcast piece - given reduce class, supplier
of values on worker, reduces and broadcasts the value, passing it to the
consumer on worker for each vertex.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduceAndBroadcastWithArrayOfHandles(String name,
int numHandles,
Supplier<ReduceOperation<S,R>> reduceOp,
SupplierFromVertex<I,V,E,Long> handleHashSupplier,
SupplierFromVertex<I,V,E,S> valueSupplier,
ConsumerWithVertex<I,V,E,R> reducedValueConsumer)
Like reduceAndBroadcast, but uses array of handles for reducers and
broadcasts, to make it feasible and performant when values are large.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduceWithMaster(String name,
ReduceOperation<S,R> reduceOp,
SupplierFromVertex<I,V,E,S> valueSupplier,
PairConsumer<R,BlockMasterApi> reducedValueConsumer)
Creates single reducer piece - given reduce class, supplier of values on
worker, reduces and passes the result to given consumer on master.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.removeVertices(String pieceName,
SupplierFromVertex<I,V,E,Boolean> shouldRemoveVertex)
Creates Piece which removes vertices for which supplier returns true.
|
Modifier and Type | Class and Description |
---|---|
class |
SendMessagePiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable>
Piece that sends a message provided through messageProducer to given set of
neighbors, and passes them to messagesConsumer.
|
class |
SendMessageWithCombinerPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable>
Piece that sends a message provided through messageProducer to given set of
neighbors, uses a message combiner and passes them to messagesConsumer.
|
Modifier and Type | Class and Description |
---|---|
class |
IterationCounterPiece
Piece that increments execution stage iteration.
|
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.