|Modifier and Type||Method and Description|
Asynchronously register a partition and its producer with the shuffle service.
Release any external resources occupied by the given partition.
CompletableFuture<T> registerPartitionWithProducer(PartitionDescriptor partitionDescriptor, ProducerDescriptor producerDescriptor)
IMPORTANT: the returned future must be completed due to limitations in the default scheduler.
The returned shuffle descriptor is an internal handle which identifies the partition internally within the shuffle service. The descriptor should provide enough information to read from or write data to the partition.
partitionDescriptor- general job graph information about the partition
producerDescriptor- general producer information (location, execution id, connection info)
void releasePartitionExternally(ShuffleDescriptor shuffleDescriptor)
This call triggers release of any resources which are occupied by the given partition in the external systems
outside of the producer executor. This is mostly relevant for the batch jobs and blocking result partitions.
The producer local resources are managed by
shuffleDescriptor- shuffle descriptor of the result partition to release externally.
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.