Modifier and Type | Method and Description |
---|---|
MemorySize |
ResourceSpec.getManagedMemory() |
MemorySize |
ResourceSpec.getTaskHeapMemory() |
MemorySize |
ResourceSpec.getTaskOffHeapMemory() |
Modifier and Type | Method and Description |
---|---|
Optional<MemorySize> |
SlotSharingGroup.getManagedMemory() |
Optional<MemorySize> |
SlotSharingGroup.getTaskHeapMemory() |
Optional<MemorySize> |
SlotSharingGroup.getTaskOffHeapMemory() |
Modifier and Type | Method and Description |
---|---|
static ResourceSpec.Builder |
ResourceSpec.newBuilder(double cpuCores,
MemorySize taskHeapMemory) |
ResourceSpec.Builder |
ResourceSpec.Builder.setManagedMemory(MemorySize managedMemory) |
SlotSharingGroup.Builder |
SlotSharingGroup.Builder.setManagedMemory(MemorySize managedMemory)
Set the task managed memory for this SlotSharingGroup.
|
ResourceSpec.Builder |
ResourceSpec.Builder.setTaskHeapMemory(MemorySize taskHeapMemory) |
SlotSharingGroup.Builder |
SlotSharingGroup.Builder.setTaskHeapMemory(MemorySize taskHeapMemory)
Set the task heap memory for this SlotSharingGroup.
|
ResourceSpec.Builder |
ResourceSpec.Builder.setTaskOffHeapMemory(MemorySize taskOffHeapMemory) |
SlotSharingGroup.Builder |
SlotSharingGroup.Builder.setTaskOffHeapMemory(MemorySize taskOffHeapMemory)
Set the task off-heap memory for this SlotSharingGroup.
|
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
FsStateChangelogOptions.IN_FLIGHT_DATA_LIMIT |
static ConfigOption<MemorySize> |
FsStateChangelogOptions.PERSIST_SIZE_THRESHOLD |
static ConfigOption<MemorySize> |
FsStateChangelogOptions.PREEMPTIVE_PERSIST_THRESHOLD |
static ConfigOption<MemorySize> |
FsStateChangelogOptions.UPLOAD_BUFFER_SIZE |
Modifier and Type | Field and Description |
---|---|
static MemorySize |
MemorySize.MAX_VALUE |
static MemorySize |
JobManagerOptions.MIN_JVM_HEAP_SIZE |
static MemorySize |
MemorySize.ZERO |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
JobManagerOptions.ADAPTIVE_BATCH_SCHEDULER_AVG_DATA_VOLUME_PER_TASK |
static ConfigOption<MemorySize> |
TaskManagerOptions.FRAMEWORK_HEAP_MEMORY
Framework Heap Memory size for TaskExecutors.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.FRAMEWORK_OFF_HEAP_MEMORY
Framework Off-Heap Memory size for TaskExecutors.
|
static ConfigOption<MemorySize> |
CheckpointingOptions.FS_SMALL_FILE_THRESHOLD
The minimum size of state data files.
|
static ConfigOption<MemorySize> |
JobManagerOptions.JOB_MANAGER_HEAP_MEMORY
Deprecated.
use
JobManagerOptions.TOTAL_FLINK_MEMORY for standalone setups and JobManagerOptions.TOTAL_PROCESS_MEMORY for containerized setups. |
static ConfigOption<MemorySize> |
JobManagerOptions.JVM_HEAP_MEMORY
JVM Heap Memory size for the JobManager.
|
static ConfigOption<MemorySize> |
JobManagerOptions.JVM_METASPACE
JVM Metaspace Size for the JobManager.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.JVM_METASPACE
JVM Metaspace Size for the TaskExecutors.
|
static ConfigOption<MemorySize> |
JobManagerOptions.JVM_OVERHEAD_MAX
Max JVM Overhead size for the TaskExecutors.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.JVM_OVERHEAD_MAX
Max JVM Overhead size for the TaskExecutors.
|
static ConfigOption<MemorySize> |
JobManagerOptions.JVM_OVERHEAD_MIN
Min JVM Overhead size for the JobManager.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.JVM_OVERHEAD_MIN
Min JVM Overhead size for the TaskExecutors.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.MANAGED_MEMORY_SIZE
Managed Memory size for TaskExecutors.
|
static ConfigOption<MemorySize> |
ResourceManagerOptions.MAX_TOTAL_MEM |
static ConfigOption<MemorySize> |
TaskManagerOptions.MEMORY_SEGMENT_SIZE
Size of memory buffers used by the network stack and the memory manager.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.MIN_MEMORY_SEGMENT_SIZE
Minimum possible size of memory buffers used by the network stack and the memory manager.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.NETWORK_BATCH_SHUFFLE_READ_MEMORY
Size of direct memory used by blocking shuffle for shuffle data read (currently only used by
sort-shuffle).
|
static ConfigOption<MemorySize> |
TaskManagerOptions.NETWORK_MEMORY_MAX
Max Network Memory size for TaskExecutors.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.NETWORK_MEMORY_MIN
Min Network Memory size for TaskExecutors.
|
static ConfigOption<MemorySize> |
JobManagerOptions.OFF_HEAP_MEMORY
Off-heap Memory size for the JobManager.
|
static ConfigOption<MemorySize> |
ExecutionOptions.SORTED_INPUTS_MEMORY |
static ConfigOption<MemorySize> |
TaskManagerOptions.TASK_HEAP_MEMORY
Task Heap Memory size for TaskExecutors.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.TASK_MANAGER_HEAP_MEMORY
Deprecated.
use
TaskManagerOptions.TOTAL_FLINK_MEMORY for standalone setups and TaskManagerOptions.TOTAL_PROCESS_MEMORY for containerized setups. |
static ConfigOption<MemorySize> |
TaskManagerOptions.TASK_OFF_HEAP_MEMORY
Task Off-Heap Memory size for TaskExecutors.
|
static ConfigOption<MemorySize> |
JobManagerOptions.TOTAL_FLINK_MEMORY
Total Flink Memory size for the JobManager.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.TOTAL_FLINK_MEMORY
Total Flink Memory size for the TaskExecutors.
|
static ConfigOption<MemorySize> |
JobManagerOptions.TOTAL_PROCESS_MEMORY
Total Process Memory size for the JobManager.
|
static ConfigOption<MemorySize> |
TaskManagerOptions.TOTAL_PROCESS_MEMORY
Total Process Memory size for the TaskExecutors.
|
Modifier and Type | Method and Description |
---|---|
MemorySize |
MemorySize.add(MemorySize that) |
MemorySize |
MemorySize.divide(long by) |
MemorySize |
MemorySize.multiply(double multiplier) |
static MemorySize |
MemorySize.ofMebiBytes(long mebiBytes) |
static MemorySize |
MemorySize.parse(String text)
Parses the given string as as MemorySize.
|
static MemorySize |
MemorySize.parse(String text,
MemorySize.MemoryUnit defaultUnit)
Parses the given string with a default unit.
|
MemorySize |
MemorySize.subtract(MemorySize that) |
Modifier and Type | Method and Description |
---|---|
ConfigOptions.TypedConfigOptionBuilder<MemorySize> |
ConfigOptions.OptionBuilder.memoryType()
Defines that the value of the option should be of
MemorySize type. |
Modifier and Type | Method and Description |
---|---|
MemorySize |
MemorySize.add(MemorySize that) |
int |
MemorySize.compareTo(MemorySize that) |
MemorySize |
MemorySize.subtract(MemorySize that) |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
StreamFormat.FETCH_IO_SIZE
The config option to define how many bytes to be read by the I/O thread in one fetch
operation.
|
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
FileSystemConnectorOptions.COMPACTION_FILE_SIZE |
static ConfigOption<MemorySize> |
FileSystemConnectorOptions.SINK_ROLLING_POLICY_FILE_SIZE |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
HBaseConnectorOptions.SINK_BUFFER_FLUSH_MAX_SIZE |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
HiveOptions.TABLE_EXEC_HIVE_FILE_OPEN_COST |
static ConfigOption<MemorySize> |
HiveOptions.TABLE_EXEC_HIVE_SPLIT_MAX_BYTES |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.BLOCK_CACHE_SIZE |
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.BLOCK_SIZE |
static ConfigOption<MemorySize> |
RocksDBOptions.FIX_PER_SLOT_MEMORY_SIZE |
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.LOG_MAX_FILE_SIZE |
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.MAX_SIZE_LEVEL_BASE |
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.METADATA_BLOCK_SIZE |
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.TARGET_FILE_SIZE_BASE |
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.WRITE_BATCH_SIZE |
static ConfigOption<MemorySize> |
RocksDBConfigurableOptions.WRITE_BUFFER_SIZE |
Modifier and Type | Method and Description |
---|---|
MemorySize |
RocksDBMemoryConfiguration.getFixedMemoryPerSlot()
Gets the fixed amount of memory to be shared between all RocksDB instances (in all tasks and
operators) of a slot.
|
Modifier and Type | Method and Description |
---|---|
void |
RocksDBMemoryConfiguration.setFixedMemoryPerSlot(MemorySize fixedMemoryPerSlot)
Configures RocksDB to use a fixed amount of memory shared between all instances (operators)
in a slot.
|
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
GSFileSystemOptions.WRITER_CHUNK_SIZE |
Modifier and Type | Method and Description |
---|---|
Optional<MemorySize> |
GSFileSystemOptions.getWriterChunkSize()
The chunk size to use for writes on the underlying Google WriteChannel.
|
Modifier and Type | Method and Description |
---|---|
GSBlobStorage.WriteChannel |
GSBlobStorageImpl.writeBlob(GSBlobIdentifier blobIdentifier,
MemorySize chunkSize) |
GSBlobStorage.WriteChannel |
GSBlobStorage.writeBlob(GSBlobIdentifier blobIdentifier,
MemorySize chunkSize)
Creates a write channel with the specified chunk size.
|
Modifier and Type | Method and Description |
---|---|
MemorySize |
TaskExecutorProcessSpec.getManagedMemorySize() |
MemorySize |
TaskExecutorProcessSpec.getNetworkMemSize() |
MemorySize |
TaskExecutorProcessSpec.getTaskHeapSize() |
MemorySize |
TaskExecutorProcessSpec.getTaskOffHeapSize() |
Modifier and Type | Method and Description |
---|---|
TaskExecutorProcessSpecBuilder |
TaskExecutorProcessSpecBuilder.withTotalProcessMemory(MemorySize totalProcessMemory) |
Modifier and Type | Method and Description |
---|---|
static Configuration |
TaskExecutorProcessUtils.getConfigurationMapLegacyTaskManagerHeapSizeToConfigOption(Configuration configuration,
ConfigOption<MemorySize> configOption) |
Constructor and Description |
---|
TaskExecutorProcessSpec(CPUResource cpuCores,
MemorySize frameworkHeapSize,
MemorySize frameworkOffHeapSize,
MemorySize taskHeapSize,
MemorySize taskOffHeapSize,
MemorySize networkMemSize,
MemorySize managedMemorySize,
MemorySize jvmMetaspaceSize,
MemorySize jvmOverheadSize,
Collection<ExternalResource> extendedResources) |
Modifier and Type | Method and Description |
---|---|
MemorySize |
ResourceProfile.getManagedMemory()
Get the managed memory needed.
|
MemorySize |
ResourceProfile.getNetworkMemory()
Get the network memory needed.
|
MemorySize |
ResourceProfile.getOperatorsMemory()
Get the memory the operators needed.
|
MemorySize |
ResourceProfile.getTaskHeapMemory()
Get the task heap memory needed.
|
MemorySize |
ResourceProfile.getTaskOffHeapMemory()
Get the task off-heap memory needed.
|
MemorySize |
ResourceProfile.getTotalMemory()
Get the total memory needed.
|
Modifier and Type | Method and Description |
---|---|
static ResourceProfile |
ResourceProfile.fromResourceSpec(ResourceSpec resourceSpec,
MemorySize networkMemory) |
ResourceProfile.Builder |
ResourceProfile.Builder.setManagedMemory(MemorySize managedMemory) |
ResourceProfile.Builder |
ResourceProfile.Builder.setNetworkMemory(MemorySize networkMemory) |
ResourceProfile.Builder |
ResourceProfile.Builder.setTaskHeapMemory(MemorySize taskHeapMemory) |
ResourceProfile.Builder |
ResourceProfile.Builder.setTaskOffHeapMemory(MemorySize taskOffHeapMemory) |
Modifier and Type | Method and Description |
---|---|
static JobManagerProcessSpec |
JobManagerProcessUtils.processSpecFromConfigWithNewOptionToInterpretLegacyHeap(Configuration config,
ConfigOption<MemorySize> newOptionToInterpretLegacyHeap) |
Constructor and Description |
---|
JobManagerProcessSpec(MemorySize jvmHeapSize,
MemorySize offHeapSize,
MemorySize jvmMetaspaceSize,
MemorySize jvmOverheadSize) |
Modifier and Type | Method and Description |
---|---|
MemorySize |
WorkerResourceSpec.getManagedMemSize() |
MemorySize |
WorkerResourceSpec.getNetworkMemSize() |
MemorySize |
WorkerResourceSpec.getTaskHeapSize() |
MemorySize |
WorkerResourceSpec.getTaskOffHeapSize() |
MemorySize |
WorkerResourceSpec.getTotalMemSize() |
Modifier and Type | Method and Description |
---|---|
MemorySize |
SlotManagerConfiguration.getMaxTotalMem() |
Constructor and Description |
---|
SlotManagerConfiguration(Time taskManagerRequestTimeout,
Time slotRequestTimeout,
Time taskManagerTimeout,
java.time.Duration requirementCheckDelay,
boolean waitResultConsumedBeforeRelease,
SlotMatchingStrategy slotMatchingStrategy,
WorkerResourceSpec defaultWorkerResourceSpec,
int numSlotsPerWorker,
int maxSlotNum,
CPUResource maxTotalCpu,
MemorySize maxTotalMem,
int redundantTaskManagerNum) |
Modifier and Type | Method and Description |
---|---|
MemorySize |
NettyShuffleMaster.computeShuffleMemorySizeForTask(TaskInputsOutputsDescriptor desc)
JM announces network memory requirement from the calculating result of this method.
|
default MemorySize |
ShuffleMaster.computeShuffleMemorySizeForTask(TaskInputsOutputsDescriptor taskInputsOutputsDescriptor)
Compute shuffle memory size for a task with the given
TaskInputsOutputsDescriptor . |
MemorySize |
ShuffleEnvironmentContext.getNetworkMemorySize() |
Constructor and Description |
---|
ShuffleEnvironmentContext(Configuration configuration,
ResourceID taskExecutorResourceId,
MemorySize networkMemorySize,
boolean localCommunicationOnly,
InetAddress hostAddress,
int numberOfSlots,
String[] tmpDirPaths,
TaskEventPublisher eventPublisher,
MetricGroup parentMetricGroup,
Executor ioExecutor) |
Modifier and Type | Method and Description |
---|---|
MemorySize |
TaskManagerServicesConfiguration.getManagedMemorySize() |
MemorySize |
TaskExecutorResourceSpec.getManagedMemorySize() |
MemorySize |
TaskManagerServicesConfiguration.getNetworkMemorySize() |
MemorySize |
TaskExecutorResourceSpec.getNetworkMemSize() |
MemorySize |
TaskExecutorResourceSpec.getTaskHeapSize() |
MemorySize |
TaskExecutorResourceSpec.getTaskOffHeapSize() |
Constructor and Description |
---|
TaskExecutorResourceSpec(CPUResource cpuCores,
MemorySize taskHeapSize,
MemorySize taskOffHeapSize,
MemorySize networkMemSize,
MemorySize managedMemorySize,
Collection<ExternalResource> extendedResources) |
Modifier and Type | Method and Description |
---|---|
static NettyShuffleEnvironmentConfiguration |
NettyShuffleEnvironmentConfiguration.fromConfiguration(Configuration configuration,
MemorySize networkMemorySize,
boolean localTaskManagerCommunication,
InetAddress taskManagerAddress)
Utility method to extract network related parameters from the configuration and to sanity
check them.
|
Modifier and Type | Method and Description |
---|---|
FM |
FlinkMemoryUtils.deriveFromTotalFlinkMemory(Configuration config,
MemorySize totalFlinkMemorySize) |
JvmMetaspaceAndOverhead |
ProcessMemoryUtils.deriveJvmMetaspaceAndOverheadFromTotalFlinkMemory(Configuration config,
MemorySize totalFlinkMemorySize) |
static MemorySize |
ProcessMemoryUtils.deriveWithFraction(String memoryDescription,
MemorySize base,
RangeFraction rangeFraction) |
static MemorySize |
ProcessMemoryUtils.deriveWithInverseFraction(String memoryDescription,
MemorySize base,
RangeFraction rangeFraction) |
static RangeFraction |
ProcessMemoryUtils.getRangeFraction(MemorySize minSize,
MemorySize maxSize,
ConfigOption<Float> fractionOption,
Configuration config) |
Modifier and Type | Method and Description |
---|---|
Configuration |
MemoryBackwardsCompatibilityUtils.getConfWithLegacyHeapSizeMappedToNewConfigOption(Configuration configuration,
ConfigOption<MemorySize> configOption) |
static MemorySize |
ProcessMemoryUtils.getMemorySizeFromConfig(Configuration config,
ConfigOption<MemorySize> option) |
Constructor and Description |
---|
JvmMetaspaceAndOverhead(MemorySize jvmMetaspace,
MemorySize jvmOverhead) |
Modifier and Type | Method and Description |
---|---|
MemorySize |
JobManagerFlinkMemory.getJvmDirectMemorySize() |
MemorySize |
JobManagerFlinkMemory.getJvmHeapMemorySize() |
MemorySize |
JobManagerFlinkMemory.getTotalFlinkMemorySize() |
Modifier and Type | Method and Description |
---|---|
JobManagerFlinkMemory |
JobManagerFlinkMemoryUtils.deriveFromTotalFlinkMemory(Configuration config,
MemorySize totalFlinkMemorySize) |
Constructor and Description |
---|
JobManagerFlinkMemory(MemorySize jvmHeap,
MemorySize offHeapMemory) |
Modifier and Type | Method and Description |
---|---|
MemorySize |
TaskExecutorFlinkMemory.getFrameworkHeap() |
static MemorySize |
TaskExecutorFlinkMemoryUtils.getFrameworkHeapMemorySize(Configuration config) |
MemorySize |
TaskExecutorFlinkMemory.getFrameworkOffHeap() |
static MemorySize |
TaskExecutorFlinkMemoryUtils.getFrameworkOffHeapMemorySize(Configuration config) |
MemorySize |
TaskExecutorFlinkMemory.getJvmDirectMemorySize() |
MemorySize |
TaskExecutorFlinkMemory.getJvmHeapMemorySize() |
MemorySize |
TaskExecutorFlinkMemory.getManaged() |
MemorySize |
TaskExecutorFlinkMemory.getNetwork() |
MemorySize |
TaskExecutorFlinkMemory.getTaskHeap() |
MemorySize |
TaskExecutorFlinkMemory.getTaskOffHeap() |
MemorySize |
TaskExecutorFlinkMemory.getTotalFlinkMemorySize() |
Modifier and Type | Method and Description |
---|---|
TaskExecutorFlinkMemory |
TaskExecutorFlinkMemoryUtils.deriveFromTotalFlinkMemory(Configuration config,
MemorySize totalFlinkMemorySize) |
Constructor and Description |
---|
TaskExecutorFlinkMemory(MemorySize frameworkHeap,
MemorySize frameworkOffHeap,
MemorySize taskHeap,
MemorySize taskOffHeap,
MemorySize network,
MemorySize managed) |
Modifier and Type | Method and Description |
---|---|
DefaultRollingPolicy.PolicyBuilder |
DefaultRollingPolicy.PolicyBuilder.withMaxPartSize(MemorySize size)
Sets the part size above which a part file will have to roll.
|
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
CollectSinkOperatorFactory.MAX_BATCH_SIZE |
Constructor and Description |
---|
CollectSinkOperatorFactory(TypeSerializer<IN> serializer,
String accumulatorName,
MemorySize maxBatchSize,
java.time.Duration socketTimeout) |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<MemorySize> |
ExecutionConfigOptions.TABLE_EXEC_RESOURCE_EXTERNAL_BUFFER_MEMORY |
static ConfigOption<MemorySize> |
ExecutionConfigOptions.TABLE_EXEC_RESOURCE_HASH_AGG_MEMORY |
static ConfigOption<MemorySize> |
ExecutionConfigOptions.TABLE_EXEC_RESOURCE_HASH_JOIN_MEMORY |
static ConfigOption<MemorySize> |
ExecutionConfigOptions.TABLE_EXEC_RESOURCE_SORT_MEMORY |
static ConfigOption<MemorySize> |
ExecutionConfigOptions.TABLE_EXEC_SPILL_COMPRESSION_BLOCK_SIZE |
Modifier and Type | Method and Description |
---|---|
MemorySize |
DescriptorProperties.getMemorySize(String key)
Deprecated.
Returns a Flink
MemorySize under the given existing key. |
Modifier and Type | Method and Description |
---|---|
Optional<MemorySize> |
DescriptorProperties.getOptionalMemorySize(String key)
Deprecated.
Returns a Flink
MemorySize under the given key if it exists. |
Modifier and Type | Method and Description |
---|---|
void |
DescriptorProperties.putMemorySize(String key,
MemorySize size)
Deprecated.
Adds a Flink
MemorySize under the given key. |
Modifier and Type | Method and Description |
---|---|
TestProcessBuilder |
TestProcessBuilder.setJvmMemory(MemorySize jvmMemory) |
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.