@Internal public class KafkaSourceEnumerator extends Object implements SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
Modifier and Type | Class and Description |
---|---|
static class |
KafkaSourceEnumerator.PartitionOffsetsRetrieverImpl
The implementation for offsets retriever with a consumer and an admin client.
|
Constructor and Description |
---|
KafkaSourceEnumerator(KafkaSubscriber subscriber,
OffsetsInitializer startingOffsetInitializer,
OffsetsInitializer stoppingOffsetInitializer,
Properties properties,
SplitEnumeratorContext<KafkaPartitionSplit> context) |
KafkaSourceEnumerator(KafkaSubscriber subscriber,
OffsetsInitializer startingOffsetInitializer,
OffsetsInitializer stoppingOffsetInitializer,
Properties properties,
SplitEnumeratorContext<KafkaPartitionSplit> context,
Set<org.apache.kafka.common.TopicPartition> assignedPartitions) |
Modifier and Type | Method and Description |
---|---|
void |
addReader(int subtaskId)
Add a new source reader with the given subtask ID.
|
void |
addSplitsBack(List<KafkaPartitionSplit> splits,
int subtaskId)
Add a split back to the split enumerator.
|
void |
close()
Called to close the enumerator, in case it holds on to any resources, like threads or network
connections.
|
void |
handleSplitRequest(int subtaskId,
String requesterHostname)
Handles the request for a split.
|
KafkaSourceEnumState |
snapshotState()
Checkpoints the state of this split enumerator.
|
void |
start()
Start the split enumerator.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
handleSourceEvent, notifyCheckpointComplete
notifyCheckpointAborted
public KafkaSourceEnumerator(KafkaSubscriber subscriber, OffsetsInitializer startingOffsetInitializer, OffsetsInitializer stoppingOffsetInitializer, Properties properties, SplitEnumeratorContext<KafkaPartitionSplit> context)
public KafkaSourceEnumerator(KafkaSubscriber subscriber, OffsetsInitializer startingOffsetInitializer, OffsetsInitializer stoppingOffsetInitializer, Properties properties, SplitEnumeratorContext<KafkaPartitionSplit> context, Set<org.apache.kafka.common.TopicPartition> assignedPartitions)
public void start()
SplitEnumerator
The default behavior does nothing.
start
in interface SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
public void handleSplitRequest(int subtaskId, @Nullable String requesterHostname)
SplitEnumerator
SourceReaderContext.sendSplitRequest()
method.handleSplitRequest
in interface SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
subtaskId
- the subtask id of the source reader who sent the source event.requesterHostname
- Optional, the hostname where the requesting task is running. This
can be used to make split assignments locality-aware.public void addSplitsBack(List<KafkaPartitionSplit> splits, int subtaskId)
SplitEnumerator
SourceReader
fails and there are splits assigned to it after the last successful checkpoint.addSplitsBack
in interface SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
splits
- The split to add back to the enumerator for reassignment.subtaskId
- The id of the subtask to which the returned splits belong.public void addReader(int subtaskId)
SplitEnumerator
addReader
in interface SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
subtaskId
- the subtask ID of the new source reader.public KafkaSourceEnumState snapshotState() throws Exception
SplitEnumerator
snapshotState
in interface SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
Exception
- when the snapshot cannot be taken.public void close()
SplitEnumerator
close
in interface AutoCloseable
close
in interface SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.