public class TaskManagerServices extends Object
Modifier and Type | Method and Description |
---|---|
static long |
calculateHeapSizeMB(long totalJavaMemorySizeMB,
Configuration config)
Calculates the amount of heap memory to use (to set via -Xmx and -Xms)
based on the total memory to use and the given configuration parameters.
|
static long |
calculateNetworkBufferMemory(long totalJavaMemorySize,
Configuration config)
Calculates the amount of memory used for network buffers based on the total memory to use and
the according configuration parameters.
|
static long |
calculateNetworkBufferMemory(TaskManagerServicesConfiguration tmConfig)
Calculates the amount of memory used for network buffers inside the current JVM instance
based on the available heap or the max heap size and the according configuration parameters.
|
static TaskManagerServices |
fromConfiguration(TaskManagerServicesConfiguration taskManagerServicesConfiguration,
ResourceID resourceID)
Creates and returns the task manager services.
|
BroadcastVariableManager |
getBroadcastVariableManager() |
FileCache |
getFileCache() |
IOManager |
getIOManager() |
JobLeaderService |
getJobLeaderService() |
JobManagerTable |
getJobManagerTable() |
MemoryManager |
getMemoryManager() |
NetworkEnvironment |
getNetworkEnvironment() |
TaskManagerLocation |
getTaskManagerLocation() |
TaskSlotTable |
getTaskSlotTable() |
public MemoryManager getMemoryManager()
public IOManager getIOManager()
public NetworkEnvironment getNetworkEnvironment()
public TaskManagerLocation getTaskManagerLocation()
public BroadcastVariableManager getBroadcastVariableManager()
public FileCache getFileCache()
public TaskSlotTable getTaskSlotTable()
public JobManagerTable getJobManagerTable()
public JobLeaderService getJobLeaderService()
public static TaskManagerServices fromConfiguration(TaskManagerServicesConfiguration taskManagerServicesConfiguration, ResourceID resourceID) throws Exception
resourceID
- resource ID of the task managertaskManagerServicesConfiguration
- task manager configurationException
public static long calculateNetworkBufferMemory(long totalJavaMemorySize, Configuration config)
The following configuration parameters are involved:
TaskManagerOptions.NETWORK_BUFFERS_MEMORY_FRACTION
,TaskManagerOptions.NETWORK_BUFFERS_MEMORY_MIN
,TaskManagerOptions.NETWORK_BUFFERS_MEMORY_MAX
, andTaskManagerOptions.NETWORK_NUM_BUFFERS
(fallback if the ones above do not exist)totalJavaMemorySize
- overall available memory to use (heap and off-heap, in bytes)config
- configuration objectpublic static long calculateNetworkBufferMemory(TaskManagerServicesConfiguration tmConfig)
For containers or when started via scripts, if started with a memory limit and set to use off-heap memory, the maximum heap size for the JVM is adjusted accordingly and we are able to extract the intended values from this.
The following configuration parameters are involved:
.tmConfig
- task manager services configuration objectpublic static long calculateHeapSizeMB(long totalJavaMemorySizeMB, Configuration config)
totalJavaMemorySizeMB
- overall available memory to use (heap and off-heap)config
- configuration objectCopyright © 2014–2018 The Apache Software Foundation. All rights reserved.