T
- The type of elements produced by the fetcher.@Internal public class Kafka010Fetcher<T> extends AbstractFetcher<T,org.apache.kafka.common.TopicPartition>
This fetcher re-uses basically all functionality of the 0.9 fetcher. It only additionally takes the KafkaRecord-attached timestamp and attaches it to the Flink records.
checkpointLock, sourceContext, unassignedPartitionsQueue, watermarkOutput
Constructor and Description |
---|
Kafka010Fetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<WatermarkStrategy<T>> watermarkStrategy,
ProcessingTimeService processingTimeProvider,
long autoWatermarkInterval,
ClassLoader userCodeClassLoader,
String taskNameWithSubtasks,
KafkaDeserializationSchema<T> deserializer,
Properties kafkaProperties,
long pollTimeout,
MetricGroup subtaskMetricGroup,
MetricGroup consumerMetricGroup,
boolean useMetrics,
FlinkConnectorRateLimiter rateLimiter) |
Modifier and Type | Method and Description |
---|---|
void |
cancel() |
org.apache.kafka.common.TopicPartition |
createKafkaPartitionHandle(KafkaTopicPartition partition)
Creates the Kafka version specific representation of the given topic partition.
|
protected void |
doCommitInternalOffsetsToKafka(Map<KafkaTopicPartition,Long> offsets,
KafkaCommitCallback commitCallback) |
void |
runFetchLoop() |
addDiscoveredPartitions, commitInternalOffsetsToKafka, emitRecordsWithTimestamps, snapshotCurrentState, subscribedPartitionStates
public Kafka010Fetcher(SourceFunction.SourceContext<T> sourceContext, Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets, SerializedValue<WatermarkStrategy<T>> watermarkStrategy, ProcessingTimeService processingTimeProvider, long autoWatermarkInterval, ClassLoader userCodeClassLoader, String taskNameWithSubtasks, KafkaDeserializationSchema<T> deserializer, Properties kafkaProperties, long pollTimeout, MetricGroup subtaskMetricGroup, MetricGroup consumerMetricGroup, boolean useMetrics, FlinkConnectorRateLimiter rateLimiter) throws Exception
Exception
public void runFetchLoop() throws Exception
runFetchLoop
in class AbstractFetcher<T,org.apache.kafka.common.TopicPartition>
Exception
public void cancel()
cancel
in class AbstractFetcher<T,org.apache.kafka.common.TopicPartition>
public org.apache.kafka.common.TopicPartition createKafkaPartitionHandle(KafkaTopicPartition partition)
AbstractFetcher
createKafkaPartitionHandle
in class AbstractFetcher<T,org.apache.kafka.common.TopicPartition>
partition
- The Flink representation of the Kafka topic partition.protected void doCommitInternalOffsetsToKafka(Map<KafkaTopicPartition,Long> offsets, @Nonnull KafkaCommitCallback commitCallback) throws Exception
doCommitInternalOffsetsToKafka
in class AbstractFetcher<T,org.apache.kafka.common.TopicPartition>
Exception
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.