public class HiveTableInputFormat extends HadoopInputFormatCommonBase<BaseRow,HiveTableInputSplit>
Modifier and Type | Field and Description |
---|---|
protected SplitReader |
reader |
credentials
Constructor and Description |
---|
HiveTableInputFormat(org.apache.hadoop.mapred.JobConf jobConf,
CatalogTable catalogTable,
List<HiveTablePartition> partitions,
int[] projectedFields,
long limit,
String hiveVersion,
boolean useMapRedReader) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Method that marks the end of the life-cycle of an input split.
|
void |
configure(Configuration parameters)
Configures this input format.
|
HiveTableInputSplit[] |
createInputSplits(int minNumSplits)
Creates the different splits of the input that can be processed in parallel.
|
InputSplitAssigner |
getInputSplitAssigner(HiveTableInputSplit[] inputSplits)
Gets the type of the input splits that are processed by this input format.
|
BaseStatistics |
getStatistics(BaseStatistics cachedStats)
Gets the basic statistics from the input described by this format.
|
BaseRow |
nextRecord(BaseRow reuse)
Reads the next record from the input.
|
void |
open(HiveTableInputSplit split)
Opens a parallel instance of the input format to work on a split.
|
boolean |
reachedEnd()
Method used to check if the end of the input is reached.
|
getCredentialsFromUGI, read, write
closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext
@VisibleForTesting protected transient SplitReader reader
public HiveTableInputFormat(org.apache.hadoop.mapred.JobConf jobConf, CatalogTable catalogTable, List<HiveTablePartition> partitions, int[] projectedFields, long limit, String hiveVersion, boolean useMapRedReader)
public void configure(Configuration parameters)
InputFormat
This method is always called first on a newly instantiated input format.
parameters
- The configuration with all parameters (note: not the Flink config but the TaskConfig).public void open(HiveTableInputSplit split) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be configured.
split
- The split to be opened.IOException
- Thrown, if the spit could not be opened due to an I/O problem.public boolean reachedEnd() throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
IOException
- Thrown, if an I/O error occurred.public BaseRow nextRecord(BaseRow reuse) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
reuse
- Object that may be reused.IOException
- Thrown, if an I/O error occurred.public void close() throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
IOException
- Thrown, if the input could not be closed properly.public HiveTableInputSplit[] createInputSplits(int minNumSplits) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be configured.
minNumSplits
- The minimum desired number of splits. If fewer are created, some parallel
instances may remain idle.IOException
- Thrown, when the creation of the splits was erroneous.public BaseStatistics getStatistics(BaseStatistics cachedStats)
InputFormat
When this method is called, the input format it guaranteed to be configured.
cachedStats
- The statistics that were cached. May be null.public InputSplitAssigner getInputSplitAssigner(HiveTableInputSplit[] inputSplits)
InputFormat
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.