public class FileSystemTableSource extends Object implements ScanTableSource, SupportsProjectionPushDown, SupportsLimitPushDown, SupportsPartitionPushDown, SupportsFilterPushDown
ScanTableSource.ScanContext, ScanTableSource.ScanRuntimeProvider
DynamicTableSource.Context, DynamicTableSource.DataStructureConverter
SupportsFilterPushDown.Result
Constructor and Description |
---|
FileSystemTableSource(DynamicTableFactory.Context context,
DecodingFormat<BulkFormat<RowData,FileSourceSplit>> bulkReaderFormat,
DecodingFormat<DeserializationSchema<RowData>> deserializationFormat,
FileSystemFormatFactory formatFactory) |
Modifier and Type | Method and Description |
---|---|
SupportsFilterPushDown.Result |
applyFilters(List<ResolvedExpression> filters)
Provides a list of filters in conjunctive form.
|
void |
applyLimit(long limit)
Provides the expected maximum number of produced records for limiting on a best-effort basis.
|
void |
applyPartitions(List<Map<String,String>> remainingPartitions)
Provides a list of remaining partitions.
|
void |
applyProjection(int[][] projectedFields)
Provides the field index paths that should be used for a projection.
|
String |
asSummaryString()
Returns a string that summarizes this source for printing to a console or log.
|
FileSystemTableSource |
copy()
Creates a copy of this instance during planning.
|
ChangelogMode |
getChangelogMode()
Returns the set of changes that the planner can expect during runtime.
|
ScanTableSource.ScanRuntimeProvider |
getScanRuntimeProvider(ScanTableSource.ScanContext scanContext)
Returns a provider of runtime implementation for reading the data.
|
Optional<List<Map<String,String>>> |
listPartitions()
Returns a list of all partitions that a source can read if available.
|
boolean |
supportsNestedProjection()
Returns whether this source supports nested projection.
|
public FileSystemTableSource(DynamicTableFactory.Context context, @Nullable DecodingFormat<BulkFormat<RowData,FileSourceSplit>> bulkReaderFormat, @Nullable DecodingFormat<DeserializationSchema<RowData>> deserializationFormat, @Nullable FileSystemFormatFactory formatFactory)
public ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(ScanTableSource.ScanContext scanContext)
ScanTableSource
There might exist different interfaces for runtime implementation which is why ScanTableSource.ScanRuntimeProvider
serves as the base interface. Concrete ScanTableSource.ScanRuntimeProvider
interfaces might be located in other Flink modules.
Independent of the provider interface, the table runtime expects that a source
implementation emits internal data structures (see RowData
for more information).
The given ScanTableSource.ScanContext
offers utilities by the planner for creating runtime
implementation with minimal dependencies to internal data structures.
See org.apache.flink.table.connector.source.SourceFunctionProvider
in flink-table-api-java-bridge
.
getScanRuntimeProvider
in interface ScanTableSource
public ChangelogMode getChangelogMode()
ScanTableSource
getChangelogMode
in interface ScanTableSource
RowKind
public SupportsFilterPushDown.Result applyFilters(List<ResolvedExpression> filters)
SupportsFilterPushDown
See the documentation of SupportsFilterPushDown
for more information.
applyFilters
in interface SupportsFilterPushDown
public void applyLimit(long limit)
SupportsLimitPushDown
applyLimit
in interface SupportsLimitPushDown
public Optional<List<Map<String,String>>> listPartitions()
SupportsPartitionPushDown
A single partition maps each partition key to a partition value.
If Optional.empty()
is returned, the list of partitions is queried from the
catalog.
listPartitions
in interface SupportsPartitionPushDown
public void applyPartitions(List<Map<String,String>> remainingPartitions)
SupportsPartitionPushDown
See the documentation of SupportsPartitionPushDown
for more information.
applyPartitions
in interface SupportsPartitionPushDown
public boolean supportsNestedProjection()
SupportsProjectionPushDown
supportsNestedProjection
in interface SupportsProjectionPushDown
public void applyProjection(int[][] projectedFields)
SupportsProjectionPushDown
SupportsProjectionPushDown.supportsNestedProjection()
.
In the example mentioned in SupportsProjectionPushDown
, this method would receive:
[[2], [1]]
which is equivalent to [["s"], ["r"]]
if SupportsProjectionPushDown.supportsNestedProjection()
returns false.
[[2], [1, 0]]
which is equivalent to [["s"], ["r", "d"]]]
if SupportsProjectionPushDown.supportsNestedProjection()
returns true.
applyProjection
in interface SupportsProjectionPushDown
projectedFields
- field index paths of all fields that must be present in the physically
produced datapublic FileSystemTableSource copy()
DynamicTableSource
copy
in interface DynamicTableSource
public String asSummaryString()
DynamicTableSource
asSummaryString
in interface DynamicTableSource
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.