Modifier and Type | Method and Description |
---|---|
Collection<ResultPartitionID> |
NettyShuffleEnvironment.getPartitionsOccupyingLocalResources()
Report unreleased partitions.
|
Modifier and Type | Method and Description |
---|---|
boolean |
TaskEventDispatcher.publish(ResultPartitionID partitionId,
TaskEvent event)
Publishes the event to the registered
EventListener instances. |
boolean |
TaskEventPublisher.publish(ResultPartitionID partitionId,
TaskEvent event)
Publishes the event to the registered
EventListener instances. |
void |
TaskEventDispatcher.registerPartition(ResultPartitionID partitionId)
Registers the given partition for incoming task events allowing calls to
TaskEventDispatcher.subscribeToEvent(ResultPartitionID, EventListener, Class) . |
void |
PartitionRequestClient.requestSubpartition(ResultPartitionID partitionId,
int subpartitionIndex,
RemoteInputChannel inputChannel,
int delayMs)
Requests a remote sub partition.
|
void |
NetworkSequenceViewReader.requestSubpartitionView(ResultPartitionProvider partitionProvider,
ResultPartitionID resultPartitionId,
int subPartitionIndex) |
void |
PartitionRequestClient.sendTaskEvent(ResultPartitionID partitionId,
TaskEvent event,
RemoteInputChannel inputChannel)
Sends a task event backwards to an intermediate result partition.
|
void |
TaskEventDispatcher.subscribeToEvent(ResultPartitionID partitionId,
EventListener<TaskEvent> eventListener,
Class<? extends TaskEvent> eventType)
Subscribes a listener to this dispatcher for events on a partition.
|
void |
TaskEventDispatcher.unregisterPartition(ResultPartitionID partitionId)
Removes the given partition from listening to incoming task events, thus forbidding calls to
TaskEventDispatcher.subscribeToEvent(ResultPartitionID, EventListener, Class) . |
Modifier and Type | Method and Description |
---|---|
void |
NettyShuffleEnvironment.releasePartitionsLocally(Collection<ResultPartitionID> partitionIds) |
Modifier and Type | Method and Description |
---|---|
ResultPartitionID |
ResultPartitionWriter.getPartitionId() |
Modifier and Type | Method and Description |
---|---|
void |
NettyPartitionRequestClient.requestSubpartition(ResultPartitionID partitionId,
int subpartitionIndex,
RemoteInputChannel inputChannel,
int delayMs)
Requests a remote intermediate result partition queue.
|
void |
NettyPartitionRequestClient.sendTaskEvent(ResultPartitionID partitionId,
TaskEvent event,
RemoteInputChannel inputChannel)
Sends a task event backwards to an intermediate result partition producer.
|
Modifier and Type | Field and Description |
---|---|
protected ResultPartitionID |
ResultPartition.partitionId |
Modifier and Type | Field and Description |
---|---|
protected Map<ResultPartitionID,org.apache.flink.runtime.io.network.partition.AbstractPartitionTracker.PartitionInfo<K,M>> |
AbstractPartitionTracker.partitionInfos |
Modifier and Type | Method and Description |
---|---|
ResultPartitionID |
PartitionException.getPartitionId() |
ResultPartitionID |
ResultPartition.getPartitionId() |
ResultPartitionID |
PartitionTrackerEntry.getResultPartitionId() |
ResultPartitionID |
TaskExecutorPartitionInfo.getResultPartitionId() |
Modifier and Type | Method and Description |
---|---|
Map<ResultPartitionID,ShuffleDescriptor> |
DataSetMetaInfo.getShuffleDescriptors() |
Collection<ResultPartitionID> |
ResultPartitionManager.getUnreleasedPartitions() |
Modifier and Type | Method and Description |
---|---|
ResultPartition |
ResultPartitionFactory.create(String taskNameWithSubtaskAndId,
int partitionIndex,
ResultPartitionID id,
ResultPartitionType type,
int numberOfSubpartitions,
int maxParallelism,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
ResultSubpartitionView |
ResultPartitionProvider.createSubpartitionView(ResultPartitionID partitionId,
int index,
BufferAvailabilityListener availabilityListener)
Returns the requested intermediate result partition input view.
|
ResultSubpartitionView |
ResultPartitionManager.createSubpartitionView(ResultPartitionID partitionId,
int subpartitionIndex,
BufferAvailabilityListener availabilityListener) |
boolean |
PartitionTracker.isPartitionTracked(ResultPartitionID resultPartitionID)
Returns whether the given partition is being tracked.
|
boolean |
AbstractPartitionTracker.isPartitionTracked(ResultPartitionID resultPartitionID) |
void |
ResultPartitionManager.releasePartition(ResultPartitionID partitionId,
Throwable cause) |
void |
PartitionProducerStateProvider.requestPartitionProducerState(IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId,
java.util.function.Consumer<? super PartitionProducerStateProvider.ResponseHandle> responseConsumer)
Trigger the producer execution state request.
|
Modifier and Type | Method and Description |
---|---|
DataSetMetaInfo |
DataSetMetaInfo.addShuffleDescriptors(Map<ResultPartitionID,ShuffleDescriptor> shuffleDescriptors) |
void |
TaskExecutorPartitionTracker.promoteJobPartitions(Collection<ResultPartitionID> partitionsToPromote)
Promotes the given partitions.
|
void |
TaskExecutorPartitionTrackerImpl.promoteJobPartitions(Collection<ResultPartitionID> partitionsToPromote) |
CompletableFuture<Void> |
JobMasterPartitionTracker.stopTrackingAndPromotePartitions(Collection<ResultPartitionID> resultPartitionIds)
Promotes the given partitions, and stops the tracking of partitions that were promoted.
|
CompletableFuture<Void> |
JobMasterPartitionTrackerImpl.stopTrackingAndPromotePartitions(Collection<ResultPartitionID> resultPartitionIds) |
void |
TaskExecutorPartitionTracker.stopTrackingAndReleaseJobPartitions(Collection<ResultPartitionID> resultPartitionIds)
Releases the given partitions and stop the tracking of partitions that were released.
|
void |
TaskExecutorPartitionTrackerImpl.stopTrackingAndReleaseJobPartitions(Collection<ResultPartitionID> partitionsToRelease) |
default void |
JobMasterPartitionTracker.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> resultPartitionIds)
Releases the given partitions and stop the tracking of partitions that were released.
|
void |
JobMasterPartitionTracker.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> resultPartitionIds,
boolean releaseOnShuffleMaster)
Releases the given partitions and stop the tracking of partitions that were released.
|
void |
JobMasterPartitionTrackerImpl.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> resultPartitionIds,
boolean releaseOnShuffleMaster) |
Collection<PartitionTrackerEntry<K,M>> |
PartitionTracker.stopTrackingPartitions(Collection<ResultPartitionID> resultPartitionIds)
Stops the tracking of the given partitions.
|
Collection<PartitionTrackerEntry<K,M>> |
AbstractPartitionTracker.stopTrackingPartitions(Collection<ResultPartitionID> resultPartitionIds) |
Constructor and Description |
---|
BoundedBlockingResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
BufferWritingResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
PartitionException(String message,
ResultPartitionID partitionId) |
PartitionException(String message,
ResultPartitionID partitionId,
Throwable throwable) |
PartitionNotFoundException(ResultPartitionID partitionId) |
PipelinedResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
ResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
SortMergeResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
BatchShuffleReadBufferPool readBufferPool,
Executor readIOExecutor,
ResultPartitionManager partitionManager,
String resultFileBasePath,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
Modifier and Type | Field and Description |
---|---|
protected ResultPartitionID |
InputChannel.partitionId
The parent partition of the subpartition consumed by this channel.
|
Modifier and Type | Method and Description |
---|---|
ResultPartitionID |
InputChannel.getPartitionId() |
Constructor and Description |
---|
InputChannel(SingleInputGate inputGate,
int channelIndex,
ResultPartitionID partitionId,
int consumedSubpartitionIndex,
int initialBackoff,
int maxBackoff,
Counter numBytesIn,
Counter numBuffersIn) |
LocalInputChannel(SingleInputGate inputGate,
int channelIndex,
ResultPartitionID partitionId,
int consumedSubpartitionIndex,
ResultPartitionManager partitionManager,
TaskEventPublisher taskEventPublisher,
int initialBackoff,
int maxBackoff,
Counter numBytesIn,
Counter numBuffersIn,
ChannelStateWriter stateWriter) |
PartitionConnectionException(ResultPartitionID partitionId,
Throwable throwable) |
RemoteChannelStateChecker(ResultPartitionID resultPartitionId,
String taskNameWithSubtask) |
RemoteInputChannel(SingleInputGate inputGate,
int channelIndex,
ResultPartitionID partitionId,
int consumedSubpartitionIndex,
ConnectionID connectionId,
ConnectionManager connectionManager,
int initialBackOff,
int maxBackoff,
int networkBuffersPerChannel,
Counter numBytesIn,
Counter numBuffersIn,
ChannelStateWriter stateWriter) |
Constructor and Description |
---|
HsResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
BatchShuffleReadBufferPool readBufferPool,
ScheduledExecutorService readIOExecutor,
ResultPartitionManager partitionManager,
String dataFileBashPath,
int networkBufferSize,
HybridShuffleConfiguration hybridShuffleConfiguration,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
Constructor and Description |
---|
PartitionProducerDisposedException(ResultPartitionID resultPartitionID) |
Modifier and Type | Method and Description |
---|---|
void |
TaskManagerGateway.releasePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds)
Batch release intermediate result partitions.
|
Modifier and Type | Method and Description |
---|---|
CompletableFuture<ExecutionState> |
JobMaster.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
CompletableFuture<ExecutionState> |
JobMasterGateway.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID partitionId)
Requests the current state of the partition.
|
Modifier and Type | Method and Description |
---|---|
void |
RpcTaskManagerGateway.releasePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds) |
CompletableFuture<?> |
JobMaster.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds) |
CompletableFuture<?> |
JobMasterGateway.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds)
Notifies the
JobMasterPartitionTracker
to stop tracking the target result partitions and release the locally occupied resources on
TaskExecutor s if any. |
Modifier and Type | Method and Description |
---|---|
ExecutionState |
SchedulerNG.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
ExecutionState |
ExecutionGraphHandler.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
ExecutionState |
SchedulerBase.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
Modifier and Type | Method and Description |
---|---|
ExecutionState |
AdaptiveScheduler.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
Modifier and Type | Method and Description |
---|---|
ResultPartitionID |
ShuffleDescriptor.getResultPartitionID() |
ResultPartitionID |
NettyShuffleDescriptor.getResultPartitionID() |
ResultPartitionID |
UnknownShuffleDescriptor.getResultPartitionID() |
Modifier and Type | Method and Description |
---|---|
Collection<ResultPartitionID> |
ShuffleEnvironment.getPartitionsOccupyingLocalResources()
Report partitions which still occupy some resources locally.
|
Modifier and Type | Method and Description |
---|---|
void |
ShuffleEnvironment.releasePartitionsLocally(Collection<ResultPartitionID> partitionIds)
Release local resources occupied by the given partitions.
|
CompletableFuture<?> |
JobShuffleContextImpl.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds) |
CompletableFuture<?> |
JobShuffleContext.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds)
Notifies the job to stop tracking and release the target result partitions, which means these
partitions will be removed and will be reproduced if used afterwards.
|
Constructor and Description |
---|
NettyShuffleDescriptor(ResourceID producerLocation,
NettyShuffleDescriptor.PartitionConnectionInfo partitionConnectionInfo,
ResultPartitionID resultPartitionID) |
UnknownShuffleDescriptor(ResultPartitionID resultPartitionID) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<ExecutionState> |
PartitionProducerStateChecker.requestPartitionProducerState(JobID jobId,
IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId)
Requests the execution state of the execution producing a result partition.
|
Modifier and Type | Method and Description |
---|---|
CompletableFuture<Acknowledge> |
TaskExecutorGatewayDecoratorBase.promotePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds) |
CompletableFuture<Acknowledge> |
TaskExecutor.promotePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds) |
CompletableFuture<Acknowledge> |
TaskExecutorGateway.promotePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds)
Batch promote intermediate result partitions.
|
void |
TaskExecutorGatewayDecoratorBase.releasePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds) |
void |
TaskExecutor.releasePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds) |
void |
TaskExecutorGateway.releasePartitions(JobID jobId,
Set<ResultPartitionID> partitionIds)
Batch release intermediate result partitions.
|
Modifier and Type | Method and Description |
---|---|
Set<ResultPartitionID> |
ClusterPartitionReport.ClusterPartitionReportEntry.getHostedPartitions() |
Map<ResultPartitionID,ShuffleDescriptor> |
ClusterPartitionReport.ClusterPartitionReportEntry.getShuffleDescriptors() |
Collection<ResultPartitionID> |
PartitionTable.stopTrackingPartitions(K key)
Stops the tracking of all partition for the given key.
|
Modifier and Type | Method and Description |
---|---|
void |
PartitionTable.startTrackingPartitions(K key,
Collection<ResultPartitionID> newPartitionIds)
Starts the tracking of the given partition for the given key.
|
void |
PartitionTable.stopTrackingPartitions(K key,
Collection<ResultPartitionID> partitionIds)
Stops the tracking of the given set of partitions for the given key.
|
Constructor and Description |
---|
ClusterPartitionReportEntry(IntermediateDataSetID dataSetId,
int numTotalPartitions,
Map<ResultPartitionID,ShuffleDescriptor> shuffleDescriptors) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<ExecutionState> |
RpcPartitionStateChecker.requestPartitionProducerState(JobID jobId,
IntermediateDataSetID resultId,
ResultPartitionID partitionId) |
Modifier and Type | Method and Description |
---|---|
void |
Task.requestPartitionProducerState(IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId,
java.util.function.Consumer<? super PartitionProducerStateProvider.ResponseHandle> responseConsumer) |
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.