SplitT
- The type of the splits used by the source.public class IteratorSourceEnumerator<SplitT extends IteratorSourceSplit<?,?>> extends Object implements SplitEnumerator<SplitT,Collection<SplitT>>
SplitEnumerator
for iterator sources. Simply takes the pre-split set of splits and
assigns it first-come-first-serve.Constructor and Description |
---|
IteratorSourceEnumerator(SplitEnumeratorContext<SplitT> context,
Collection<SplitT> splits) |
Modifier and Type | Method and Description |
---|---|
void |
addReader(int subtaskId)
Add a new source reader with the given subtask ID.
|
void |
addSplitsBack(List<SplitT> splits,
int subtaskId)
Add a split back to the split enumerator.
|
void |
close()
Called to close the enumerator, in case it holds on to any resources, like threads or network
connections.
|
void |
handleSplitRequest(int subtaskId,
String requesterHostname)
Handles the request for a split.
|
Collection<SplitT> |
snapshotState()
Checkpoints the state of this split enumerator.
|
void |
start()
Start the split enumerator.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
handleSourceEvent, notifyCheckpointComplete
notifyCheckpointAborted
public IteratorSourceEnumerator(SplitEnumeratorContext<SplitT> context, Collection<SplitT> splits)
public void start()
SplitEnumerator
The default behavior does nothing.
start
in interface SplitEnumerator<SplitT extends IteratorSourceSplit<?,?>,Collection<SplitT extends IteratorSourceSplit<?,?>>>
public void close()
SplitEnumerator
close
in interface AutoCloseable
close
in interface SplitEnumerator<SplitT extends IteratorSourceSplit<?,?>,Collection<SplitT extends IteratorSourceSplit<?,?>>>
public void handleSplitRequest(int subtaskId, @Nullable String requesterHostname)
SplitEnumerator
SourceReaderContext.sendSplitRequest()
method.handleSplitRequest
in interface SplitEnumerator<SplitT extends IteratorSourceSplit<?,?>,Collection<SplitT extends IteratorSourceSplit<?,?>>>
subtaskId
- the subtask id of the source reader who sent the source event.requesterHostname
- Optional, the hostname where the requesting task is running. This
can be used to make split assignments locality-aware.public void addSplitsBack(List<SplitT> splits, int subtaskId)
SplitEnumerator
SourceReader
fails and there are splits assigned to it after the last successful checkpoint.addSplitsBack
in interface SplitEnumerator<SplitT extends IteratorSourceSplit<?,?>,Collection<SplitT extends IteratorSourceSplit<?,?>>>
splits
- The split to add back to the enumerator for reassignment.subtaskId
- The id of the subtask to which the returned splits belong.public Collection<SplitT> snapshotState() throws Exception
SplitEnumerator
snapshotState
in interface SplitEnumerator<SplitT extends IteratorSourceSplit<?,?>,Collection<SplitT extends IteratorSourceSplit<?,?>>>
Exception
- when the snapshot cannot be taken.public void addReader(int subtaskId)
SplitEnumerator
addReader
in interface SplitEnumerator<SplitT extends IteratorSourceSplit<?,?>,Collection<SplitT extends IteratorSourceSplit<?,?>>>
subtaskId
- the subtask ID of the new source reader.Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.