@Internal public class PulsarOrderedSourceReader<OUT> extends SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
Modifier and Type | Field and Description |
---|---|
protected org.apache.pulsar.client.admin.PulsarAdmin |
pulsarAdmin |
protected org.apache.pulsar.client.api.PulsarClient |
pulsarClient |
protected SourceConfiguration |
sourceConfiguration |
config, context, options, recordEmitter, splitFetcherManager
Constructor and Description |
---|
PulsarOrderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>>> elementsQueue,
java.util.function.Supplier<PulsarOrderedPartitionSplitReader> splitReaderSupplier,
PulsarRecordEmitter<OUT> recordEmitter,
SourceReaderContext context,
SourceConfiguration sourceConfiguration,
org.apache.pulsar.client.api.PulsarClient pulsarClient,
org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
protected void |
closeFinishedSplits(Set<String> finishedSplitIds) |
protected PulsarPartitionSplitState |
initializedState(PulsarPartitionSplit split)
When new splits are added to the reader.
|
void |
notifyCheckpointComplete(long checkpointId)
We have an empty default implementation here because most source readers do not have to
implement the method.
|
protected void |
onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)
Handles the finished splits to clean the state if needed.
|
void |
pauseOrResumeSplits(Collection<String> splitsToPause,
Collection<String> splitsToResume)
Pauses or resumes reading of individual source splits.
|
InputStatus |
pollNext(ReaderOutput<OUT> output)
Poll the next available record into the
ReaderOutput . |
List<PulsarPartitionSplit> |
snapshotState(long checkpointId)
Checkpoint on the state of the source.
|
void |
start()
Start the reader.
|
protected PulsarPartitionSplit |
toSplitType(String splitId,
PulsarPartitionSplitState splitState)
Convert a mutable SplitStateT to immutable SplitT.
|
addSplits, getNumberOfCurrentlyAssignedSplits, handleSourceEvents, isAvailable, notifyNoMoreSplits
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
notifyCheckpointAborted
protected final SourceConfiguration sourceConfiguration
protected final org.apache.pulsar.client.api.PulsarClient pulsarClient
protected final org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin
public PulsarOrderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>>> elementsQueue, java.util.function.Supplier<PulsarOrderedPartitionSplitReader> splitReaderSupplier, PulsarRecordEmitter<OUT> recordEmitter, SourceReaderContext context, SourceConfiguration sourceConfiguration, org.apache.pulsar.client.api.PulsarClient pulsarClient, org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin)
public void start()
SourceReader
start
in interface SourceReader<OUT,PulsarPartitionSplit>
start
in class SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public InputStatus pollNext(ReaderOutput<OUT> output) throws Exception
SourceReader
ReaderOutput
.
The implementation must make sure this method is non-blocking.
Although the implementation can emit multiple records into the given ReaderOutput, it is
recommended not doing so. Instead, emit one record into the ReaderOutput and return a InputStatus.MORE_AVAILABLE
to let the caller thread know there are more records available.
pollNext
in interface SourceReader<OUT,PulsarPartitionSplit>
pollNext
in class SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
Exception
protected void onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)
SourceReaderBase
onSplitFinished
in class SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public List<PulsarPartitionSplit> snapshotState(long checkpointId)
SourceReader
snapshotState
in interface SourceReader<OUT,PulsarPartitionSplit>
snapshotState
in class SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public void notifyCheckpointComplete(long checkpointId)
SourceReader
checkpointId
- The ID of the checkpoint that has been completed.CheckpointListener.notifyCheckpointComplete(long)
public void close() throws Exception
close
in interface AutoCloseable
Exception
protected PulsarPartitionSplitState initializedState(PulsarPartitionSplit split)
SourceReaderBase
initializedState
in class SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
split
- a newly added split.protected PulsarPartitionSplit toSplitType(String splitId, PulsarPartitionSplitState splitState)
SourceReaderBase
toSplitType
in class SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
splitState
- splitState.public void pauseOrResumeSplits(Collection<String> splitsToPause, Collection<String> splitsToResume)
SourceReader
Note that no other methods can be called in parallel, so updating subscriptions can be done atomically. This method is simply providing connectors with more expressive APIs the opportunity to update all subscriptions at once.
This is currently used to align the watermarks of splits, if watermark alignment is used and the source reads from more than one split.
The default implementation throws an UnsupportedOperationException
where the
default implementation will be removed in future releases. To be compatible with future
releases, it is recommended to implement this method and override the default implementation.
pauseOrResumeSplits
in interface SourceReader<OUT,PulsarPartitionSplit>
pauseOrResumeSplits
in class SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
splitsToPause
- the splits to pausesplitsToResume
- the splits to resumeCopyright © 2014–2022 The Apache Software Foundation. All rights reserved.