org.apache.giraph.comm.messages

Class ByteArrayMessagesPerVertexStore<I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.comm.messages.SimpleMessageStore<I,M,DataInputOutput>

org.apache.giraph.comm.messages.ByteArrayMessagesPerVertexStore<I,M>

Type Parameters:

I - Vertex id

M - Message data

All Implemented Interfaces:

MessageStore<I,M>

public class ByteArrayMessagesPerVertexStore<I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable>
extends SimpleMessageStore<I,M,DataInputOutput>

Implementation of SimpleMessageStore where multiple messages are stored per vertex as byte backed datastructures. Used when there is no combiner provided.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ByteArrayMessagesPerVertexStore.Factory<I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable> Factory for ByteArrayMessagesPerVertexStore

Field Summary

Fields inherited from class org.apache.giraph.comm.messages.SimpleMessageStore

config, map, messageValueFactory, partitionInfo

Constructor Summary

Constructors

Constructor and Description
ByteArrayMessagesPerVertexStore(MessageValueFactory<M> messageValueFactory, PartitionSplitInfo<I> partitionInfo, ImmutableClassesGiraphConfiguration<I,?,?> config) Constructor

Method Summary

Modifier and Type	Method and Description
void	addMessage(I vertexId, M message) Adds a message for a particular vertex The method is used by InternalMessageStore to send local messages; for the general case, use a more efficient addPartitionMessages
void	addPartitionMessages(int partitionId, VertexIdMessages<I,M> messages) Adds messages for partition
protected Iterable<M>	getMessagesAsIterable(DataInputOutput dataInputOutput) Get messages as an iterable from message storage
protected int	getNumberOfMessagesIn(ConcurrentMap<I,DataInputOutput> partitionMap) Get number of messages in partition map
boolean	isPointerListEncoding() True if this message-store encodes messages as a list of long pointers to compact serialized messages
static <I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable> MessageStoreFactory<I,M,MessageStore<I,M>>	newFactory(CentralizedServiceWorker<I,?,?> service, ImmutableClassesGiraphConfiguration<I,?,?> config) Create new factory for this message store
protected DataInputOutput	readFieldsForMessages(DataInput in) Read message storage from DataInput
protected void	writeMessages(DataInputOutput dataInputOutput, DataOutput out) Write message storage to DataOutput

Methods inherited from class org.apache.giraph.comm.messages.SimpleMessageStore

clearAll, clearPartition, clearVertexMessages, finalizeStore, getOrCreatePartitionMap, getPartitionDestinationVertices, getPartitionId, getVertexMessages, hasMessagesForPartition, hasMessagesForVertex, readFieldsForPartition, writePartition

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ByteArrayMessagesPerVertexStore

public ByteArrayMessagesPerVertexStore(MessageValueFactory<M> messageValueFactory,
                                       PartitionSplitInfo<I> partitionInfo,
                                       ImmutableClassesGiraphConfiguration<I,?,?> config)

Constructor

Parameters:

messageValueFactory - Message class held in the store

partitionInfo - Partition split info

config - Hadoop configuration

Method Detail

isPointerListEncoding

public boolean isPointerListEncoding()

Description copied from interface: MessageStore

True if this message-store encodes messages as a list of long pointers to compact serialized messages

Returns:

true if we encode messages as a list of pointers

addPartitionMessages

public void addPartitionMessages(int partitionId,
                                 VertexIdMessages<I,M> messages)

Description copied from interface: MessageStore

Adds messages for partition

Parameters:

partitionId - Id of partition

messages - Collection of vertex ids and messages we want to add

addMessage

public void addMessage(I vertexId,
                       M message)
                throws IOException

Description copied from interface: MessageStore

Adds a message for a particular vertex The method is used by InternalMessageStore to send local messages; for the general case, use a more efficient addPartitionMessages

Parameters:

vertexId - Id of target vertex

message - A message to send

Throws:

IOException

getMessagesAsIterable

protected Iterable<M> getMessagesAsIterable(DataInputOutput dataInputOutput)

Description copied from class: SimpleMessageStore

Get messages as an iterable from message storage

Specified by:

getMessagesAsIterable in class SimpleMessageStore<I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable,DataInputOutput>

Parameters:

dataInputOutput - Message storage

Returns:

Messages as an iterable

getNumberOfMessagesIn

protected int getNumberOfMessagesIn(ConcurrentMap<I,DataInputOutput> partitionMap)

Description copied from class: SimpleMessageStore

Get number of messages in partition map

Specified by:

getNumberOfMessagesIn in class SimpleMessageStore<I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable,DataInputOutput>

Parameters:

partitionMap - Partition map in which to count messages

Returns:

Number of messages in partition map

writeMessages

protected void writeMessages(DataInputOutput dataInputOutput,
                             DataOutput out)
                      throws IOException

Description copied from class: SimpleMessageStore

Write message storage to DataOutput

Specified by:

writeMessages in class SimpleMessageStore<I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable,DataInputOutput>

Parameters:

dataInputOutput - Message storage

out - Data output

Throws:

IOException

readFieldsForMessages

protected DataInputOutput readFieldsForMessages(DataInput in)
                                         throws IOException

Description copied from class: SimpleMessageStore

Read message storage from DataInput

Specified by:

readFieldsForMessages in class SimpleMessageStore<I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable,DataInputOutput>

Parameters:

in - Data input

Returns:

Message storage

Throws:

IOException

newFactory

public static <I extends org.apache.hadoop.io.WritableComparable,M extends org.apache.hadoop.io.Writable> MessageStoreFactory<I,M,MessageStore<I,M>> newFactory(CentralizedServiceWorker<I,?,?> service,
                                                                                                                                                                ImmutableClassesGiraphConfiguration<I,?,?> config)

Create new factory for this message store

Type Parameters:

I - Vertex id

M - Message data

Parameters:

service - Worker service

config - Hadoop configuration

Returns:

Factory