public final class UnknownShuffleDescriptor extends Object implements ShuffleDescriptor
ShuffleDescriptor
for which the producer has not been deployed yet.
When a partition consumer is being scheduled, it can happen
that the producer of the partition (consumer input channel) has not been scheduled
and its location and other relevant data is yet to be defined.
To proceed with the consumer deployment, currently unknown input channels have to be
marked with placeholders which are this special implementation of ShuffleDescriptor
.
Constructor and Description |
---|
UnknownShuffleDescriptor(ResultPartitionID resultPartitionID) |
Modifier and Type | Method and Description |
---|---|
ResultPartitionID |
getResultPartitionID() |
boolean |
isUnknown()
Returns whether the partition is known and registered with the
ShuffleMaster implementation. |
Optional<ResourceID> |
storesLocalResourcesOn()
Returns the location of the producing task executor if the partition occupies local resources there.
|
public UnknownShuffleDescriptor(ResultPartitionID resultPartitionID)
public ResultPartitionID getResultPartitionID()
getResultPartitionID
in interface ShuffleDescriptor
public boolean isUnknown()
ShuffleDescriptor
ShuffleMaster
implementation.
When a partition consumer is being scheduled, it can happen
that the producer of the partition (consumer input channel) has not been scheduled
and its location and other relevant data is yet to be defined.
To proceed with the consumer deployment, currently unknown input channels have to be
marked with placeholders. The placeholder is a special implementation of the shuffle descriptor:
UnknownShuffleDescriptor
.
Note: this method is not supposed to be overridden in concrete shuffle implementation.
The only class where it returns true
is UnknownShuffleDescriptor
.
isUnknown
in interface ShuffleDescriptor
ShuffleMaster
implementation.public Optional<ResourceID> storesLocalResourcesOn()
ShuffleDescriptor
Indicates that this partition occupies local resources in the producing task executor. Such partition requires
that the task executor is running and being connected to be able to consume the produced data. This is mostly
relevant for the batch jobs and blocking result partitions which can outlive the producer lifetime and
be released externally.
ShuffleEnvironment.releasePartitionsLocally(Collection)
can be used to release such kind of partitions locally.
storesLocalResourcesOn
in interface ShuffleDescriptor
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.