@PublicEvolving public class NettyShuffleEnvironmentOptions extends Object
Modifier and Type | Field and Description |
---|---|
static ConfigOption<Boolean> |
BLOCKING_SHUFFLE_COMPRESSION_ENABLED
Boolean flag indicating whether the shuffle data will be compressed for blocking shuffle mode.
|
static ConfigOption<Integer> |
CLIENT_CONNECT_TIMEOUT_SECONDS |
static ConfigOption<Integer> |
CONNECT_BACKLOG |
static ConfigOption<Integer> |
DATA_PORT
The default network port the task manager expects to receive transfer envelopes on.
|
static ConfigOption<Boolean> |
DATA_SSL_ENABLED
Config parameter to override SSL support for taskmanager's data transport.
|
static ConfigOption<Boolean> |
FORCE_PARTITION_RELEASE_ON_CONSUMPTION |
static ConfigOption<String> |
NETWORK_BLOCKING_SHUFFLE_TYPE |
static ConfigOption<Float> |
NETWORK_BUFFERS_MEMORY_FRACTION
Deprecated.
use
TaskManagerOptions.NETWORK_MEMORY_FRACTION instead |
static ConfigOption<String> |
NETWORK_BUFFERS_MEMORY_MAX
Deprecated.
use
TaskManagerOptions.NETWORK_MEMORY_MAX instead |
static ConfigOption<String> |
NETWORK_BUFFERS_MEMORY_MIN
Deprecated.
use
TaskManagerOptions.NETWORK_MEMORY_MIN instead |
static ConfigOption<Integer> |
NETWORK_BUFFERS_PER_CHANNEL
Number of network buffers to use for each outgoing/incoming channel (subpartition/input channel).
|
static ConfigOption<Boolean> |
NETWORK_DETAILED_METRICS
Boolean flag to enable/disable more detailed metrics about inbound/outbound network queue
lengths.
|
static ConfigOption<Long> |
NETWORK_EXCLUSIVE_BUFFERS_REQUEST_TIMEOUT_MILLISECONDS
The timeout for requesting exclusive buffers for each channel.
|
static ConfigOption<Integer> |
NETWORK_EXTRA_BUFFERS_PER_GATE
Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate).
|
static ConfigOption<Integer> |
NETWORK_NUM_BUFFERS
Deprecated.
|
static ConfigOption<Integer> |
NETWORK_REQUEST_BACKOFF_INITIAL
Minimum backoff for partition requests of input channels.
|
static ConfigOption<Integer> |
NETWORK_REQUEST_BACKOFF_MAX
Maximum backoff for partition requests of input channels.
|
static ConfigOption<Integer> |
NUM_ARENAS |
static ConfigOption<Integer> |
NUM_THREADS_CLIENT |
static ConfigOption<Integer> |
NUM_THREADS_SERVER |
static ConfigOption<Integer> |
SEND_RECEIVE_BUFFER_SIZE |
static ConfigOption<String> |
SHUFFLE_COMPRESSION_CODEC
The codec to be used when compressing shuffle data.
|
static ConfigOption<String> |
TRANSPORT_TYPE |
public static final ConfigOption<Integer> DATA_PORT
0
means that
the TaskManager searches for a free port.public static final ConfigOption<Boolean> DATA_SSL_ENABLED
public static final ConfigOption<Boolean> BLOCKING_SHUFFLE_COMPRESSION_ENABLED
Note: Data is compressed per buffer and compression can incur extra CPU overhead so it is more effective for IO bounded scenario when data compression ratio is high. Currently, shuffle data compression is an experimental feature and the config option can be changed in the future.
public static final ConfigOption<String> SHUFFLE_COMPRESSION_CODEC
public static final ConfigOption<Boolean> NETWORK_DETAILED_METRICS
@Deprecated public static final ConfigOption<Integer> NETWORK_NUM_BUFFERS
TaskManagerOptions.NETWORK_MEMORY_FRACTION
, TaskManagerOptions.NETWORK_MEMORY_MIN
,
and TaskManagerOptions.NETWORK_MEMORY_MAX
instead@Deprecated public static final ConfigOption<Float> NETWORK_BUFFERS_MEMORY_FRACTION
TaskManagerOptions.NETWORK_MEMORY_FRACTION
instead@Deprecated public static final ConfigOption<String> NETWORK_BUFFERS_MEMORY_MIN
TaskManagerOptions.NETWORK_MEMORY_MIN
instead@Deprecated public static final ConfigOption<String> NETWORK_BUFFERS_MEMORY_MAX
TaskManagerOptions.NETWORK_MEMORY_MAX
insteadpublic static final ConfigOption<Integer> NETWORK_BUFFERS_PER_CHANNEL
Reasoning: 1 buffer for in-flight data in the subpartition + 1 buffer for parallel serialization.
public static final ConfigOption<Integer> NETWORK_EXTRA_BUFFERS_PER_GATE
public static final ConfigOption<Long> NETWORK_EXCLUSIVE_BUFFERS_REQUEST_TIMEOUT_MILLISECONDS
public static final ConfigOption<String> NETWORK_BLOCKING_SHUFFLE_TYPE
public static final ConfigOption<Integer> NUM_ARENAS
public static final ConfigOption<Integer> NUM_THREADS_SERVER
public static final ConfigOption<Integer> NUM_THREADS_CLIENT
public static final ConfigOption<Integer> CONNECT_BACKLOG
public static final ConfigOption<Integer> CLIENT_CONNECT_TIMEOUT_SECONDS
public static final ConfigOption<Integer> SEND_RECEIVE_BUFFER_SIZE
public static final ConfigOption<String> TRANSPORT_TYPE
public static final ConfigOption<Integer> NETWORK_REQUEST_BACKOFF_INITIAL
public static final ConfigOption<Integer> NETWORK_REQUEST_BACKOFF_MAX
public static final ConfigOption<Boolean> FORCE_PARTITION_RELEASE_ON_CONSUMPTION
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.