public class ResultPartition extends Object implements BufferPoolOwner
This class is the runtime part of a logical IntermediateResultPartition
. Essentially,
a result partition is a collection of Buffer
instances. The buffers are organized in one
or more ResultSubpartition
instances, which further partition the data depending on the
number of consuming tasks and the data DistributionPattern
.
Tasks, which consume a result partition have to request one of its subpartitions. The request
happens either remotely (see RemoteInputChannel
) or locally (see LocalInputChannel
)
Modifier and Type | Field and Description |
---|---|
int |
numTargetKeyGroups |
Constructor and Description |
---|
ResultPartition(String owningTaskName,
TaskActions taskActions,
JobID jobId,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numberOfSubpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
ResultPartitionConsumableNotifier partitionConsumableNotifier,
IOManager ioManager,
boolean sendScheduleOrUpdateConsumersMessage) |
Modifier and Type | Method and Description |
---|---|
void |
add(Buffer buffer,
int subpartitionIndex)
Adds a buffer to the subpartition with the given index.
|
ResultSubpartitionView |
createSubpartitionView(int index,
BufferAvailabilityListener availabilityListener)
Returns the requested subpartition.
|
void |
destroyBufferPool() |
void |
finish()
Finishes the result partition.
|
BufferPool |
getBufferPool() |
BufferProvider |
getBufferProvider() |
Throwable |
getFailureCause() |
JobID |
getJobId() |
int |
getNumberOfQueuedBuffers() |
int |
getNumberOfSubpartitions() |
int |
getNumTargetKeyGroups() |
ResultPartitionID |
getPartitionId() |
ResultPartitionType |
getPartitionType()
Returns the type of this result partition.
|
int |
getTotalNumberOfBuffers()
Returns the total number of processed network buffers since initialization.
|
long |
getTotalNumberOfBytes()
Returns the total size of processed network buffers since initialization.
|
void |
registerBufferPool(BufferPool bufferPool)
Registers a buffer pool with this result partition.
|
void |
release() |
void |
release(Throwable cause)
Releases the result partition.
|
void |
releaseMemory(int toRelease)
Releases buffers held by this result partition.
|
String |
toString() |
public ResultPartition(String owningTaskName, TaskActions taskActions, JobID jobId, ResultPartitionID partitionId, ResultPartitionType partitionType, int numberOfSubpartitions, int numTargetKeyGroups, ResultPartitionManager partitionManager, ResultPartitionConsumableNotifier partitionConsumableNotifier, IOManager ioManager, boolean sendScheduleOrUpdateConsumersMessage)
public void registerBufferPool(BufferPool bufferPool)
There is one pool for each result partition, which is shared by all its sub partitions.
The pool is registered with the partition *after* it as been constructed in order to conform
to the life-cycle of task registrations in the TaskManager
.
public JobID getJobId()
public ResultPartitionID getPartitionId()
public int getNumberOfSubpartitions()
public BufferProvider getBufferProvider()
public BufferPool getBufferPool()
public int getTotalNumberOfBuffers()
public long getTotalNumberOfBytes()
public int getNumberOfQueuedBuffers()
public ResultPartitionType getPartitionType()
public void add(Buffer buffer, int subpartitionIndex) throws IOException
For PIPELINED results, this will trigger the deployment of consuming tasks after the first buffer has been added.
IOException
public void finish() throws IOException
After this operation, it is not possible to add further data to the result partition.
For BLOCKING results, this will trigger the deployment of consuming tasks.
IOException
public void release()
public void release(Throwable cause)
public void destroyBufferPool()
public ResultSubpartitionView createSubpartitionView(int index, BufferAvailabilityListener availabilityListener) throws IOException
IOException
public Throwable getFailureCause()
public int getNumTargetKeyGroups()
public void releaseMemory(int toRelease) throws IOException
This is a callback from the buffer pool, which is registered for result partitions, which are back pressure-free.
releaseMemory
in interface BufferPoolOwner
IOException
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.