@Internal public class MapPartitionIterator<IN> extends Object implements Iterator<IN>
MapPartitionIterator
is an iterator used in the MapPartitionOperator
.The task
main thread will add records to it. It will set itself as the input parameter of MapPartitionFunction
and execute the function.Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_MAX_CACHE_NUM
Max number of caches.
|
Constructor and Description |
---|
MapPartitionIterator(java.util.function.Consumer<Iterator<IN>> udf) |
Modifier and Type | Method and Description |
---|---|
void |
addRecord(IN record) |
void |
close() |
boolean |
hasNext() |
IN |
next() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
forEachRemaining, remove
public static final int DEFAULT_MAX_CACHE_NUM
The constant defines the maximum number of caches that can be created. Its value is set to 100, which is considered sufficient for most parallel jobs. Each cache is a record and occupies a minimal amount of memory so the value is not excessively large.
public MapPartitionIterator(java.util.function.Consumer<Iterator<IN>> udf)
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.