Modifier and Type | Class and Description |
---|---|
class |
AbstractIterativeTask<S extends Function,OT>
The abstract base class for all tasks able to participate in an iteration.
|
class |
IterationHeadTask<X,Y,S extends Function,OT>
The head is responsible for coordinating an iteration and can run a
Driver inside. |
class |
IterationIntermediateTask<S extends Function,OT>
An intermediate iteration task, which runs a
Driver
inside. |
class |
IterationSynchronizationSinkTask
The task responsible for synchronizing all iteration heads, implemented as an output task.
|
class |
IterationTailTask<S extends Function,OT>
An iteration tail, which runs a driver inside.
|
Modifier and Type | Method and Description |
---|---|
Class<? extends AbstractInvokable> |
JobVertex.getInvokableClass(ClassLoader cl)
Returns the invokable class which represents the task of this vertex.
|
Modifier and Type | Method and Description |
---|---|
void |
JobVertex.setInvokableClass(Class<? extends AbstractInvokable> invokable) |
Modifier and Type | Class and Description |
---|---|
class |
BatchTask<S extends Function,OT>
The base class for all batch tasks.
|
class |
DataSinkTask<IT>
DataSinkTask which is executed by a task manager.
|
class |
DataSourceTask<OT>
DataSourceTask which is executed by a task manager.
|
Modifier and Type | Method and Description |
---|---|
AbstractInvokable |
BatchTask.getContainingTask() |
AbstractInvokable |
TaskContext.getContainingTask() |
Modifier and Type | Method and Description |
---|---|
static void |
BatchTask.closeChainedTasks(List<ChainedDriver<?,?>> tasks,
AbstractInvokable parent)
Closes all chained tasks, in the order as they are stored in the array.
|
static String |
BatchTask.constructLogString(String message,
String taskName,
AbstractInvokable parent)
Utility function that composes a string for logging purposes.
|
static <T> Collector<T> |
BatchTask.getOutputCollector(AbstractInvokable task,
TaskConfig config,
ClassLoader cl,
List<RecordWriter<?>> eventualOutputs,
int outputOffset,
int numOutputs)
Creates the
Collector for the given task, as described by the given configuration. |
static <T> Collector<T> |
BatchTask.initOutputs(AbstractInvokable containingTask,
UserCodeClassLoader cl,
TaskConfig config,
List<ChainedDriver<?,?>> chainedTasksTarget,
List<RecordWriter<?>> eventualOutputs,
ExecutionConfig executionConfig,
Map<String,Accumulator<?,?>> accumulatorMap)
Creates a writer for each output.
|
static void |
BatchTask.logAndThrowException(Exception ex,
AbstractInvokable parent)
Prints an error message and throws the given exception.
|
static void |
BatchTask.openChainedTasks(List<ChainedDriver<?,?>> tasks,
AbstractInvokable parent)
Opens all chained tasks, in the order as they are stored in the array.
|
void |
NoOpChainedDriver.setup(AbstractInvokable parent) |
Constructor and Description |
---|
TempBarrier(AbstractInvokable owner,
MutableObjectIterator<T> input,
TypeSerializerFactory<T> serializerFactory,
MemoryManager memManager,
IOManager ioManager,
int numPages,
List<MemorySegment> preAllocated) |
Modifier and Type | Method and Description |
---|---|
void |
ChainedFlatMapDriver.setup(AbstractInvokable parent) |
abstract void |
ChainedDriver.setup(AbstractInvokable parent) |
void |
ChainedAllReduceDriver.setup(AbstractInvokable parent) |
void |
ChainedReduceCombineDriver.setup(AbstractInvokable parent) |
void |
ChainedMapDriver.setup(AbstractInvokable parent) |
void |
ChainedTerminationCriterionDriver.setup(AbstractInvokable parent) |
void |
SynchronousChainedCombineDriver.setup(AbstractInvokable parent) |
void |
ChainedDriver.setup(TaskConfig config,
String taskName,
Collector<OT> outputCollector,
AbstractInvokable parent,
UserCodeClassLoader userCodeClassLoader,
ExecutionConfig executionConfig,
Map<String,Accumulator<?,?>> accumulatorMap) |
Modifier and Type | Method and Description |
---|---|
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
HashJoinIteratorBase.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBloomFilters) |
Constructor and Description |
---|
NonReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
Modifier and Type | Method and Description |
---|---|
static <E> ExternalSorterBuilder<E> |
ExternalSorter.newBuilder(MemoryManager memoryManager,
AbstractInvokable parentTask,
TypeSerializer<E> serializer,
TypeComparator<E> comparator)
Creates a builder for the
ExternalSorter . |
Modifier and Type | Method and Description |
---|---|
Class<? extends AbstractInvokable> |
StreamNode.getJobVertexClass() |
Modifier and Type | Method and Description |
---|---|
protected StreamNode |
StreamGraph.addNode(Integer vertexID,
String slotSharingGroup,
String coLocationGroup,
Class<? extends AbstractInvokable> vertexClass,
StreamOperatorFactory<?> operatorFactory,
String operatorName) |
Constructor and Description |
---|
StreamNode(Integer id,
String slotSharingGroup,
String coLocationGroup,
StreamOperator<?> operator,
String operatorName,
Class<? extends AbstractInvokable> jobVertexClass) |
StreamNode(Integer id,
String slotSharingGroup,
String coLocationGroup,
StreamOperatorFactory<?> operatorFactory,
String operatorName,
Class<? extends AbstractInvokable> jobVertexClass) |
Modifier and Type | Method and Description |
---|---|
static <K> MultiInputSortingDataInput.SelectableSortingInputs |
MultiInputSortingDataInput.wrapInputs(AbstractInvokable containingTask,
StreamTaskInput<Object>[] sortingInputs,
KeySelector<Object,K>[] keySelectors,
TypeSerializer<Object>[] inputSerializers,
TypeSerializer<K> keySerializer,
StreamTaskInput<Object>[] passThroughInputs,
MemoryManager memoryManager,
IOManager ioManager,
boolean objectReuse,
double managedMemoryFraction,
Configuration jobConfiguration) |
Constructor and Description |
---|
SortingDataInput(StreamTaskInput<T> wrappedInput,
TypeSerializer<T> typeSerializer,
TypeSerializer<K> keySerializer,
KeySelector<T,K> keySelector,
MemoryManager memoryManager,
IOManager ioManager,
boolean objectReuse,
double managedMemoryFraction,
Configuration jobConfiguration,
AbstractInvokable containingTask) |
Modifier and Type | Method and Description |
---|---|
static <IN1,IN2> StreamTwoInputProcessor<IN1,IN2> |
StreamTwoInputProcessorFactory.create(AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
IOManager ioManager,
MemoryManager memoryManager,
TaskIOMetricGroup taskIOMetricGroup,
StreamStatusMaintainer streamStatusMaintainer,
TwoInputStreamOperator<IN1,IN2,?> streamOperator,
WatermarkGauge input1WatermarkGauge,
WatermarkGauge input2WatermarkGauge,
BoundedMultiInput endOfInputAware,
StreamConfig streamConfig,
Configuration taskManagerConfig,
Configuration jobConfig,
ExecutionConfig executionConfig,
ClassLoader userClassloader,
Counter numRecordsIn,
InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
TaskInfo taskInfo) |
static StreamMultipleInputProcessor |
StreamMultipleInputProcessorFactory.create(AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
StreamConfig.InputConfig[] configuredInputs,
IOManager ioManager,
MemoryManager memoryManager,
TaskIOMetricGroup ioMetricGroup,
Counter mainOperatorRecordsIn,
StreamStatusMaintainer streamStatusMaintainer,
MultipleInputStreamOperator<?> mainOperator,
WatermarkGauge[] inputWatermarkGauges,
StreamConfig streamConfig,
Configuration taskManagerConfig,
Configuration jobConfig,
ExecutionConfig executionConfig,
ClassLoader userClassloader,
OperatorChain<?,?> operatorChain,
InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
TaskInfo taskInfo) |
Modifier and Type | Method and Description |
---|---|
static SingleCheckpointBarrierHandler |
SingleCheckpointBarrierHandler.aligned(String taskName,
AbstractInvokable toNotifyOnCheckpoint,
Clock clock,
int numOpenChannels,
java.util.function.BiFunction<Callable<?>,java.time.Duration,org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.Cancellable> registerTimer,
CheckpointableInput... inputs) |
static SingleCheckpointBarrierHandler |
SingleCheckpointBarrierHandler.alternating(String taskName,
AbstractInvokable toNotifyOnCheckpoint,
SubtaskCheckpointCoordinator checkpointCoordinator,
Clock clock,
int numOpenChannels,
java.util.function.BiFunction<Callable<?>,java.time.Duration,org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.Cancellable> registerTimer,
CheckpointableInput... inputs) |
static CheckpointBarrierHandler |
InputProcessorUtil.createCheckpointBarrierHandler(AbstractInvokable toNotifyOnCheckpoint,
StreamConfig config,
SubtaskCheckpointCoordinator checkpointCoordinator,
String taskName,
List<IndexedInputGate>[] inputGates,
List<StreamTaskSourceInput<?>> sourceInputs,
MailboxExecutor mailboxExecutor,
TimerService timerService) |
static CheckpointedInputGate |
InputProcessorUtil.createCheckpointedInputGate(AbstractInvokable toNotifyOnCheckpoint,
StreamConfig config,
SubtaskCheckpointCoordinator checkpointCoordinator,
IndexedInputGate[] inputGates,
TaskIOMetricGroup taskIOMetricGroup,
String taskName,
MailboxExecutor mailboxExecutor,
TimerService timerService) |
static CheckpointedInputGate[] |
InputProcessorUtil.createCheckpointedMultipleInputGate(AbstractInvokable toNotifyOnCheckpoint,
StreamConfig config,
SubtaskCheckpointCoordinator checkpointCoordinator,
TaskIOMetricGroup taskIOMetricGroup,
String taskName,
MailboxExecutor mailboxExecutor,
List<IndexedInputGate>[] inputGates,
List<StreamTaskSourceInput<?>> sourceInputs,
TimerService timerService) |
static SingleCheckpointBarrierHandler |
SingleCheckpointBarrierHandler.createUnalignedCheckpointBarrierHandler(SubtaskCheckpointCoordinator checkpointCoordinator,
String taskName,
AbstractInvokable toNotifyOnCheckpoint,
Clock clock,
CheckpointableInput... inputs) |
static SingleCheckpointBarrierHandler |
SingleCheckpointBarrierHandler.unaligned(String taskName,
AbstractInvokable toNotifyOnCheckpoint,
SubtaskCheckpointCoordinator checkpointCoordinator,
Clock clock,
int numOpenChannels,
java.util.function.BiFunction<Callable<?>,java.time.Duration,org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.Cancellable> registerTimer,
CheckpointableInput... inputs) |
Constructor and Description |
---|
CheckpointBarrierHandler(AbstractInvokable toNotifyOnCheckpoint,
Clock clock) |
CheckpointBarrierTracker(int totalNumberOfInputChannels,
AbstractInvokable toNotifyOnCheckpoint,
Clock clock) |
Modifier and Type | Class and Description |
---|---|
class |
AbstractTwoInputStreamTask<IN1,IN2,OUT>
Abstract class for executing a
TwoInputStreamOperator . |
class |
MultipleInputStreamTask<OUT>
A
StreamTask for executing a MultipleInputStreamOperator and supporting the
MultipleInputStreamOperator to select input for reading. |
class |
OneInputStreamTask<IN,OUT>
A
StreamTask for executing a OneInputStreamOperator . |
class |
SourceOperatorStreamTask<T>
A subclass of
StreamTask for executing the SourceOperator . |
class |
SourceStreamTask<OUT,SRC extends SourceFunction<OUT>,OP extends StreamSource<OUT,SRC>>
StreamTask for executing a StreamSource . |
class |
StreamIterationHead<OUT>
A special
StreamTask that is used for executing feedback edges. |
class |
StreamIterationTail<IN>
A special
StreamTask that is used for executing feedback edges. |
class |
StreamTask<OUT,OP extends StreamOperator<OUT>>
Base class for all streaming tasks.
|
class |
TwoInputStreamTask<IN1,IN2,OUT>
A
StreamTask for executing a TwoInputStreamOperator and supporting the TwoInputStreamOperator to select input for reading. |
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.