public class Elasticsearch extends ConnectorDescriptor
Constructor and Description |
---|
Elasticsearch()
Connector descriptor for the Elasticsearch search engine.
|
Modifier and Type | Method and Description |
---|---|
Elasticsearch |
bulkFlushBackoffConstant()
Configures how to buffer elements before sending them in bulk to the cluster for efficiency.
|
Elasticsearch |
bulkFlushBackoffDelay(long delay)
Configures how to buffer elements before sending them in bulk to the cluster for efficiency.
|
Elasticsearch |
bulkFlushBackoffExponential()
Configures how to buffer elements before sending them in bulk to the cluster for efficiency.
|
Elasticsearch |
bulkFlushBackoffMaxRetries(int maxRetries)
Configures how to buffer elements before sending them in bulk to the cluster for efficiency.
|
Elasticsearch |
bulkFlushInterval(long interval)
Configures how to buffer elements before sending them in bulk to the cluster for efficiency.
|
Elasticsearch |
bulkFlushMaxActions(int maxActions)
Configures how to buffer elements before sending them in bulk to the cluster for efficiency.
|
Elasticsearch |
bulkFlushMaxSize(String maxSize)
Configures how to buffer elements before sending them in bulk to the cluster for efficiency.
|
Elasticsearch |
connectionMaxRetryTimeout(int maxRetryTimeout)
Sets connection properties to be used during REST communication to Elasticsearch.
|
Elasticsearch |
connectionPathPrefix(String pathPrefix)
Sets connection properties to be used during REST communication to Elasticsearch.
|
Elasticsearch |
disableFlushOnCheckpoint()
Disables flushing on checkpoint.
|
Elasticsearch |
documentType(String documentType)
Declares the Elasticsearch document type for every record.
|
Elasticsearch |
failureHandlerCustom(Class<? extends ActionRequestFailureHandler> failureHandlerClass)
Configures a failure handling strategy in case a request to Elasticsearch fails.
|
Elasticsearch |
failureHandlerFail()
Configures a failure handling strategy in case a request to Elasticsearch fails.
|
Elasticsearch |
failureHandlerIgnore()
Configures a failure handling strategy in case a request to Elasticsearch fails.
|
Elasticsearch |
failureHandlerRetryRejected()
Configures a failure handling strategy in case a request to Elasticsearch fails.
|
Elasticsearch |
host(String hostname,
int port,
String protocol)
Adds an Elasticsearch host to connect to.
|
Elasticsearch |
index(String index)
Declares the Elasticsearch index for every record.
|
Elasticsearch |
keyDelimiter(String keyDelimiter)
Sets a custom key delimiter in case the Elasticsearch ID needs to be constructed from
multiple fields.
|
Elasticsearch |
keyNullLiteral(String keyNullLiteral)
Sets a custom representation for null fields in keys.
|
protected Map<String,String> |
toConnectorProperties()
Converts this descriptor into a set of connector properties.
|
Elasticsearch |
version(String version)
Sets the Elasticsearch version to be used.
|
isFormatNeeded, toProperties
toString
public Elasticsearch()
public Elasticsearch version(String version)
version
- Elasticsearch version. E.g., "6".public Elasticsearch host(String hostname, int port, String protocol)
Multiple hosts can be declared by calling this method multiple times.
hostname
- connection hostnameport
- connection portprotocol
- connection protocol; e.g. "http"public Elasticsearch index(String index)
index
- Elasticsearch indexpublic Elasticsearch documentType(String documentType)
documentType
- Elasticsearch document typepublic Elasticsearch keyDelimiter(String keyDelimiter)
keyDelimiter
- key delimiter; e.g., "$" would result in IDs "KEY1$KEY2$KEY3"public Elasticsearch keyNullLiteral(String keyNullLiteral)
keyNullLiteral
- key null literal string; e.g. "N/A" would result in IDs "KEY1_N/A_KEY3"public Elasticsearch failureHandlerFail()
This strategy throws an exception if a request fails and thus causes a job failure.
public Elasticsearch failureHandlerIgnore()
This strategy ignores failures and drops the request.
public Elasticsearch failureHandlerRetryRejected()
This strategy re-adds requests that have failed due to queue capacity saturation.
public Elasticsearch failureHandlerCustom(Class<? extends ActionRequestFailureHandler> failureHandlerClass)
This strategy allows for custom failure handling using a ActionRequestFailureHandler
.
public Elasticsearch disableFlushOnCheckpoint()
Note: If flushing on checkpoint is disabled, a Elasticsearch sink does NOT provide any strong guarantees for at-least-once delivery of action requests.
public Elasticsearch bulkFlushMaxActions(int maxActions)
Sets the maximum number of actions to buffer for each bulk request.
maxActions
- the maximum number of actions to buffer per bulk request.public Elasticsearch bulkFlushMaxSize(String maxSize)
Sets the maximum size of buffered actions per bulk request (using the syntax of MemorySize
).
public Elasticsearch bulkFlushInterval(long interval)
Sets the bulk flush interval (in milliseconds).
interval
- bulk flush interval (in milliseconds).public Elasticsearch bulkFlushBackoffConstant()
Sets a constant backoff type to use when flushing bulk requests.
public Elasticsearch bulkFlushBackoffExponential()
Sets an exponential backoff type to use when flushing bulk requests.
public Elasticsearch bulkFlushBackoffMaxRetries(int maxRetries)
Sets the maximum number of retries for a backoff attempt when flushing bulk requests.
Make sure to enable backoff by selecting a strategy (bulkFlushBackoffConstant()
or
bulkFlushBackoffExponential()
).
maxRetries
- the maximum number of retries.public Elasticsearch bulkFlushBackoffDelay(long delay)
Sets the amount of delay between each backoff attempt when flushing bulk requests (in milliseconds).
Make sure to enable backoff by selecting a strategy (bulkFlushBackoffConstant()
or
bulkFlushBackoffExponential()
).
delay
- delay between each backoff attempt (in milliseconds).public Elasticsearch connectionMaxRetryTimeout(int maxRetryTimeout)
Sets the maximum timeout (in milliseconds) in case of multiple retries of the same request.
maxRetryTimeout
- maximum timeout (in milliseconds)public Elasticsearch connectionPathPrefix(String pathPrefix)
Adds a path prefix to every REST communication.
pathPrefix
- prefix string to be added to every REST communicationprotected Map<String,String> toConnectorProperties()
ConnectorDescriptor
ConnectorDescriptorValidator.CONNECTOR
.toConnectorProperties
in class ConnectorDescriptor
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.