Modifier and Type | Method and Description |
---|---|
protected static MemoryManager |
MesosTaskManager.memoryManager() |
Constructor and Description |
---|
MesosTaskManager(TaskManagerConfiguration config,
ResourceID resourceID,
TaskManagerLocation taskManagerLocation,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment network,
int numberOfSlots,
HighAvailabilityServices highAvailabilityServices,
MetricRegistry metricRegistry) |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
Environment.getMemoryManager()
Returns the current
MemoryManager . |
Constructor and Description |
---|
FileChannelInputView(BlockChannelReader<MemorySegment> reader,
MemoryManager memManager,
List<MemorySegment> memory,
int sizeOfLastBlock) |
FileChannelOutputView(BlockChannelWriter<MemorySegment> writer,
MemoryManager memManager,
List<MemorySegment> memory,
int segmentSize) |
SeekableFileChannelInputView(IOManager ioManager,
FileIOChannel.ID channelId,
MemoryManager memManager,
List<MemorySegment> memory,
int sizeOfLastBlock) |
Modifier and Type | Method and Description |
---|---|
akka.actor.Props |
LocalFlinkMiniCluster.getTaskManagerProps(Class<? extends TaskManager> taskManagerClass,
TaskManagerConfiguration taskManagerConfig,
ResourceID resourceID,
TaskManagerLocation taskManagerLocation,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment networkEnvironment,
MetricRegistry metricsRegistry) |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
TaskContext.getMemoryManager() |
MemoryManager |
BatchTask.getMemoryManager() |
Constructor and Description |
---|
TempBarrier(AbstractInvokable owner,
MutableObjectIterator<T> input,
TypeSerializerFactory<T> serializerFactory,
MemoryManager memManager,
IOManager ioManager,
int numPages) |
Modifier and Type | Method and Description |
---|---|
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
HashJoinIteratorBase.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBloomFilters) |
Constructor and Description |
---|
NonReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
Modifier and Type | Field and Description |
---|---|
protected MemoryManager |
SpillingResettableMutableObjectIterator.memoryManager |
protected MemoryManager |
SpillingResettableIterator.memoryManager |
Modifier and Type | Field and Description |
---|---|
protected MemoryManager |
UnilateralSortMerger.SpillingThread.memManager |
protected MemoryManager |
UnilateralSortMerger.memoryManager
The memory manager through which memory is allocated and released.
|
Modifier and Type | Method and Description |
---|---|
protected UnilateralSortMerger.ThreadBase<E> |
UnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxFileHandles) |
protected UnilateralSortMerger.ThreadBase<E> |
CombiningUnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxFileHandles) |
Constructor and Description |
---|
AbstractMergeInnerJoinIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
AbstractMergeIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
AbstractMergeOuterJoinIterator(OuterJoinOperatorBase.OuterJoinType outerJoinType,
MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
CombiningSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles,
boolean objectReuseEnabled) |
CombiningUnilateralSortMerger(GroupCombineFunction<E,E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
CombiningUnilateralSortMerger(GroupCombineFunction<E,E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
LargeRecordHandler(TypeSerializer<T> serializer,
TypeComparator<T> comparator,
IOManager ioManager,
MemoryManager memManager,
List<MemorySegment> memory,
AbstractInvokable memoryOwner,
int maxFilehandles) |
NonReusingMergeInnerJoinIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
NonReusingMergeOuterJoinIterator(OuterJoinOperatorBase.OuterJoinType outerJoinType,
MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
ReusingMergeInnerJoinIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
ReusingMergeOuterJoinIterator(OuterJoinOperatorBase.OuterJoinType outerJoinType,
MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
SpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles)
Creates the spilling thread.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
List<MemorySegment> memory,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
List<MemorySegment> memory,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
TaskManagerServices.getMemoryManager() |
Constructor and Description |
---|
TaskExecutor(RpcService rpcService,
TaskManagerConfiguration taskManagerConfiguration,
TaskManagerLocation taskManagerLocation,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment networkEnvironment,
HighAvailabilityServices haServices,
HeartbeatServices heartbeatServices,
MetricRegistry metricRegistry,
TaskManagerMetricGroup taskManagerMetricGroup,
BroadcastVariableManager broadcastVariableManager,
FileCache fileCache,
TaskSlotTable taskSlotTable,
JobManagerTable jobManagerTable,
JobLeaderService jobLeaderService,
FatalErrorHandler fatalErrorHandler) |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
RuntimeEnvironment.getMemoryManager() |
protected MemoryManager |
TaskManager.memoryManager() |
Modifier and Type | Method and Description |
---|---|
akka.actor.Props |
TaskManager$.getTaskManagerProps(Class<? extends TaskManager> taskManagerClass,
TaskManagerConfiguration taskManagerConfig,
ResourceID resourceID,
TaskManagerLocation taskManagerLocation,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment networkEnvironment,
HighAvailabilityServices highAvailabilityServices,
MetricRegistry metricsRegistry) |
static akka.actor.Props |
TaskManager.getTaskManagerProps(Class<? extends TaskManager> taskManagerClass,
TaskManagerConfiguration taskManagerConfig,
ResourceID resourceID,
TaskManagerLocation taskManagerLocation,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment networkEnvironment,
HighAvailabilityServices highAvailabilityServices,
MetricRegistry metricsRegistry) |
Constructor and Description |
---|
RuntimeEnvironment(JobID jobId,
JobVertexID jobVertexId,
ExecutionAttemptID executionId,
ExecutionConfig executionConfig,
TaskInfo taskInfo,
Configuration jobConfiguration,
Configuration taskConfiguration,
ClassLoader userCodeClassLoader,
MemoryManager memManager,
IOManager ioManager,
BroadcastVariableManager bcVarManager,
AccumulatorRegistry accumulatorRegistry,
TaskKvStateRegistry kvStateRegistry,
InputSplitProvider splitProvider,
Map<String,Future<Path>> distCacheEntries,
ResultPartitionWriter[] writers,
InputGate[] inputGates,
CheckpointResponder checkpointResponder,
TaskManagerRuntimeInfo taskManagerInfo,
TaskMetricGroup metrics,
Task containingTask) |
Task(JobInformation jobInformation,
TaskInformation taskInformation,
ExecutionAttemptID executionAttemptID,
AllocationID slotAllocationId,
int subtaskIndex,
int attemptNumber,
Collection<ResultPartitionDeploymentDescriptor> resultPartitionDeploymentDescriptors,
Collection<InputGateDeploymentDescriptor> inputGateDeploymentDescriptors,
int targetSlotNumber,
TaskStateHandles taskStateHandles,
MemoryManager memManager,
IOManager ioManager,
NetworkEnvironment networkEnvironment,
BroadcastVariableManager bcVarManager,
TaskManagerActions taskManagerActions,
InputSplitProvider inputSplitProvider,
CheckpointResponder checkpointResponder,
LibraryCacheManager libraryCache,
FileCache fileCache,
TaskManagerRuntimeInfo taskManagerConfig,
TaskMetricGroup metricGroup,
ResultPartitionConsumableNotifier resultPartitionConsumableNotifier,
PartitionProducerStateChecker partitionProducerStateChecker,
Executor executor)
IMPORTANT: This constructor may not start any work that would need to
be undone in the case of a failing task deployment.
|
TaskManager(TaskManagerConfiguration config,
ResourceID resourceID,
TaskManagerLocation location,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment network,
int numberOfSlots,
HighAvailabilityServices highAvailabilityServices,
MetricRegistry metricsRegistry) |
Modifier and Type | Method and Description |
---|---|
protected static MemoryManager |
YarnTaskManager.memoryManager() |
Constructor and Description |
---|
YarnTaskManager(TaskManagerConfiguration config,
ResourceID resourceID,
TaskManagerLocation taskManagerLocation,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment network,
int numberOfSlots,
HighAvailabilityServices highAvailabilityServices,
MetricRegistry metricRegistry) |
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.