public class HiveTableSource extends Object implements StreamTableSource<BaseRow>, PartitionableTableSource, ProjectableTableSource<BaseRow>, LimitableTableSource<BaseRow>
Constructor and Description |
---|
HiveTableSource(org.apache.hadoop.mapred.JobConf jobConf,
ObjectPath tablePath,
CatalogTable catalogTable) |
Modifier and Type | Method and Description |
---|---|
TableSource<BaseRow> |
applyLimit(long limit)
Check and push down the limit to the table source.
|
TableSource<BaseRow> |
applyPartitionPruning(List<Map<String,String>> remainingPartitions)
Applies the remaining partitions to the table source.
|
String |
explainSource()
Describes the table source.
|
DataStream<BaseRow> |
getDataStream(StreamExecutionEnvironment execEnv)
Returns the data of the table as a
DataStream . |
List<Map<String,String>> |
getPartitions()
Returns all the partitions of this
PartitionableTableSource . |
DataType |
getProducedDataType()
Returns the
DataType for the produced data of the TableSource . |
TableSchema |
getTableSchema()
Returns the schema of the produced table.
|
boolean |
isBounded()
Returns true if this is a bounded source, false if this is an unbounded source.
|
boolean |
isLimitPushedDown()
Return the flag to indicate whether limit push down has been tried.
|
TableSource<BaseRow> |
projectFields(int[] fields)
Creates a copy of the
TableSource that projects its output to the given field indexes. |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getReturnType
public HiveTableSource(org.apache.hadoop.mapred.JobConf jobConf, ObjectPath tablePath, CatalogTable catalogTable)
public boolean isBounded()
StreamTableSource
isBounded
in interface StreamTableSource<BaseRow>
public DataStream<BaseRow> getDataStream(StreamExecutionEnvironment execEnv)
StreamTableSource
DataStream
.
NOTE: This method is for internal use only for defining a TableSource
.
Do not use it in Table API programs.
getDataStream
in interface StreamTableSource<BaseRow>
public TableSchema getTableSchema()
TableSource
getTableSchema
in interface TableSource<BaseRow>
TableSchema
of the produced table.public DataType getProducedDataType()
TableSource
DataType
for the produced data of the TableSource
.getProducedDataType
in interface TableSource<BaseRow>
DataSet
or DataStream
.public boolean isLimitPushedDown()
LimitableTableSource
LimitableTableSource.applyLimit(long)
.isLimitPushedDown
in interface LimitableTableSource<BaseRow>
public TableSource<BaseRow> applyLimit(long limit)
LimitableTableSource
applyLimit
in interface LimitableTableSource<BaseRow>
limit
- the value which limit the number of records.TableSource
.public List<Map<String,String>> getPartitions()
PartitionableTableSource
PartitionableTableSource
.getPartitions
in interface PartitionableTableSource
public TableSource<BaseRow> applyPartitionPruning(List<Map<String,String>> remainingPartitions)
PartitionableTableSource
remainingPartitions
is
the remaining partitions of PartitionableTableSource.getPartitions()
after partition pruning applied.
After trying to apply partition pruning, we should return a new TableSource
instance which holds all pruned-partitions.
applyPartitionPruning
in interface PartitionableTableSource
remainingPartitions
- Remaining partitions after partition pruning applied.TableSource
holds all pruned-partitions.public String explainSource()
TableSource
explainSource
in interface TableSource<BaseRow>
TableSource
.public TableSource<BaseRow> projectFields(int[] fields)
ProjectableTableSource
TableSource
that projects its output to the given field indexes.
The field indexes relate to the physical poduced data type (TableSource.getProducedDataType()
)
and not to the table schema (TableSource.getTableSchema()
of the TableSource
.
The table schema (TableSource.getTableSchema()
of the TableSource
copy must not be
modified by this method, but only the produced data type (TableSource.getProducedDataType()
)
and the produced DataSet
(BatchTableSource#getDataSet(
) or DataStream
(StreamTableSource#getDataStream
).
If the TableSource
implements the DefinedFieldMapping
interface, it might
be necessary to adjust the mapping as well.
IMPORTANT: This method must return a true copy and must not modify the original table source object.
projectFields
in interface ProjectableTableSource<BaseRow>
fields
- The indexes of the fields to return.TableSource
that projects its output.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.