@NotThreadSafe public class SortMergeResultPartition extends ResultPartition
SortMergeResultPartition
appends records and events to SortBuffer
and after the
SortBuffer
is full, all data in the SortBuffer
will be copied and spilled to a
PartitionedFile
in subpartition index order sequentially. Large records that can not be
appended to an empty SortBuffer
will be spilled to the result PartitionedFile
separately.AvailabilityProvider.AvailabilityHelper
bufferCompressor, bufferPool, LOG, numBuffersOut, numBytesOut, numSubpartitions, partitionId, partitionManager, partitionType
AVAILABLE
Constructor and Description |
---|
SortMergeResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
int networkBufferSize,
ResultPartitionManager partitionManager,
String resultFileBasePath,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
Modifier and Type | Method and Description |
---|---|
void |
broadcastEvent(AbstractEvent event,
boolean isPriorityEvent)
Writes the given
AbstractEvent to all channels. |
void |
broadcastRecord(ByteBuffer record)
Writes the given serialized record to all subpartitions.
|
void |
close()
Closes the partition writer which releases the allocated resource, for example the buffer
pool.
|
ResultSubpartitionView |
createSubpartitionView(int subpartitionIndex,
BufferAvailabilityListener availabilityListener)
Returns a reader for the subpartition with the given index.
|
void |
emitRecord(ByteBuffer record,
int targetSubpartition)
Writes the given serialized record to the target subpartition.
|
void |
finish()
Finishes the result partition.
|
void |
flush(int subpartitionIndex)
Manually trigger the consumption of data from the given subpartitions.
|
void |
flushAll()
Manually trigger the consumption of data from all subpartitions.
|
CompletableFuture<?> |
getAvailableFuture() |
int |
getNumberOfQueuedBuffers()
Returns the total number of queued buffers of all subpartitions.
|
int |
getNumberOfQueuedBuffers(int targetSubpartition)
Returns the number of queued buffers of the given target subpartition.
|
protected void |
releaseInternal()
Releases all produced data including both those stored in memory and persisted on disk.
|
canBeCompressed, checkInProduceState, fail, getBufferPool, getFailureCause, getNumberOfSubpartitions, getNumTargetKeyGroups, getOwningTaskName, getPartitionId, getPartitionIndex, getPartitionManager, getPartitionType, isFinished, isReleased, release, release, setMetricGroup, setup, toString
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
and, isApproximatelyAvailable, isAvailable, or
public SortMergeResultPartition(String owningTaskName, int partitionIndex, ResultPartitionID partitionId, ResultPartitionType partitionType, int numSubpartitions, int numTargetKeyGroups, int networkBufferSize, ResultPartitionManager partitionManager, String resultFileBasePath, @Nullable BufferCompressor bufferCompressor, SupplierWithException<BufferPool,IOException> bufferPoolFactory)
protected void releaseInternal()
ResultPartition
releaseInternal
in class ResultPartition
public void emitRecord(ByteBuffer record, int targetSubpartition) throws IOException
ResultPartitionWriter
IOException
public void broadcastRecord(ByteBuffer record) throws IOException
ResultPartitionWriter
IOException
public void broadcastEvent(AbstractEvent event, boolean isPriorityEvent) throws IOException
ResultPartitionWriter
AbstractEvent
to all channels.IOException
public void finish() throws IOException
ResultPartition
After this operation, it is not possible to add further data to the result partition.
For BLOCKING results, this will trigger the deployment of consuming tasks.
finish
in interface ResultPartitionWriter
finish
in class ResultPartition
IOException
public void close()
ResultPartitionWriter
close
in interface AutoCloseable
close
in interface ResultPartitionWriter
close
in class ResultPartition
public ResultSubpartitionView createSubpartitionView(int subpartitionIndex, BufferAvailabilityListener availabilityListener) throws IOException
ResultPartitionWriter
IOException
public void flushAll()
ResultPartitionWriter
public void flush(int subpartitionIndex)
ResultPartitionWriter
public CompletableFuture<?> getAvailableFuture()
getAvailableFuture
in interface AvailabilityProvider
getAvailableFuture
in class ResultPartition
public int getNumberOfQueuedBuffers()
ResultPartition
getNumberOfQueuedBuffers
in class ResultPartition
public int getNumberOfQueuedBuffers(int targetSubpartition)
ResultPartition
getNumberOfQueuedBuffers
in class ResultPartition
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.