public class Table extends Object
Table
.
Use the methods of Table
to transform data. Use TableEnvironment
to convert a Table
back to a DataSet or DataStream.
When using Scala a Table
can also be converted using implicit conversions.
Example:
val env = ExecutionEnvironment.getExecutionEnvironment
val tEnv = TableEnvironment.getTableEnvironment(env)
val set: DataSet[(String, Int)] = ...
val table = set.toTable(tEnv, 'a, 'b)
...
val table2 = ...
val set2: DataSet[MyType] = table2.toDataSet[MyType]
Operations such as join
, select
, where
and groupBy
either take arguments
in a Scala DSL or as an expression String. Please refer to the documentation for the expression
syntax.
param: tableEnv The TableEnvironment
to which the table is bound.
param: logicalPlan logical representation
Constructor and Description |
---|
Table(TableEnvironment tableEnv,
LogicalNode logicalPlan) |
Table(TableEnvironment tableEnv,
String udtfCall)
Creates a
Table for a TableFunction call from a String expression. |
Modifier and Type | Method and Description |
---|---|
Table |
as(scala.collection.Seq<Expression> fields)
Renames the fields of the expression result.
|
Table |
as(String fields)
Renames the fields of the expression result.
|
Table |
distinct()
Removes duplicate values and returns only distinct (different) values.
|
Table |
filter(Expression predicate)
Filters out elements that don't pass the filter predicate.
|
Table |
filter(String predicate)
Filters out elements that don't pass the filter predicate.
|
Table |
fullOuterJoin(Table right,
Expression joinPredicate)
Joins two
Table s. |
Table |
fullOuterJoin(Table right,
String joinPredicate)
Joins two
Table s. |
org.apache.calcite.rel.RelNode |
getRelNode() |
TableSchema |
getSchema()
Returns the schema of this table.
|
GroupedTable |
groupBy(scala.collection.Seq<Expression> fields)
Groups the elements on some grouping keys.
|
GroupedTable |
groupBy(String fields)
Groups the elements on some grouping keys.
|
Table |
intersect(Table right)
Intersects two
Table s with duplicate records removed. |
Table |
intersectAll(Table right)
Intersects two
Table s. |
Table |
join(Table right)
Joins two
Table s. |
Table |
join(Table right,
Expression joinPredicate)
Joins two
Table s. |
Table |
join(Table right,
String joinPredicate)
Joins two
Table s. |
Table |
leftOuterJoin(Table right)
Joins this
Table with an user-defined TableFunction . |
Table |
leftOuterJoin(Table right,
Expression joinPredicate)
Joins two
Table s. |
Table |
leftOuterJoin(Table right,
String joinPredicate)
Joins two
Table s. |
Table |
limit(int offset)
Limits a sorted result from an offset position.
|
Table |
limit(int offset,
int fetch)
Limits a sorted result to a specified number of records from an offset position.
|
LogicalNode |
logicalPlan() |
Table |
minus(Table right)
Minus of two
Table s with duplicate records removed. |
Table |
minusAll(Table right)
Minus of two
Table s. |
Table |
orderBy(scala.collection.Seq<Expression> fields)
Sorts the given
Table . |
Table |
orderBy(String fields)
Sorts the given
Table . |
void |
printSchema()
Prints the schema of this table to the console in a tree format.
|
FlinkRelBuilder |
relBuilder() |
Table |
rightOuterJoin(Table right,
Expression joinPredicate)
Joins two
Table s. |
Table |
rightOuterJoin(Table right,
String joinPredicate)
Joins two
Table s. |
Table |
select(scala.collection.Seq<Expression> fields)
Performs a selection operation.
|
Table |
select(String fields)
Performs a selection operation.
|
TableEnvironment |
tableEnv() |
String |
tableName() |
String |
toString()
Registers an unique table name under the table environment
and return the registered table name.
|
Table |
union(Table right)
Unions two
Table s with duplicate records removed. |
Table |
unionAll(Table right)
Unions two
Table s. |
Table |
where(Expression predicate)
Filters out elements that don't pass the filter predicate.
|
Table |
where(String predicate)
Filters out elements that don't pass the filter predicate.
|
OverWindowedTable |
window(OverWindow... overWindows)
Defines over-windows on the records of a table.
|
OverWindowedTable |
window(scala.collection.Seq<OverWindow> overWindows)
Defines over-windows on the records of a table.
|
WindowedTable |
window(Window window)
Groups the records of a table by assigning them to windows defined by a time or row interval.
|
<T> void |
writeToSink(TableSink<T> sink)
Writes the
Table to a TableSink . |
<T> void |
writeToSink(TableSink<T> sink,
QueryConfig conf)
Writes the
Table to a TableSink . |
public Table(TableEnvironment tableEnv, LogicalNode logicalPlan)
public Table(TableEnvironment tableEnv, String udtfCall)
Table
for a TableFunction call from a String expression.
tableEnv
- The TableEnvironment in which the call is created.udtfCall
- A String expression of the TableFunction call.public OverWindowedTable window(OverWindow... overWindows)
An over-window defines for each record an interval of records over which aggregation functions can be computed.
Example:
table
.window(Over partitionBy 'c orderBy 'rowTime preceding 10.seconds as 'ow)
.select('c, 'b.count over 'ow, 'e.sum over 'ow)
__Note__: Computing over window aggregates on a streaming table is only a parallel operation if the window is partitioned. Otherwise, the whole stream will be processed by a single task, i.e., with parallelism 1.
__Note__: Over-windows for batch tables are currently not supported.
overWindows
- windows that specify the record interval over which aggregations are
computed.public TableEnvironment tableEnv()
public LogicalNode logicalPlan()
public FlinkRelBuilder relBuilder()
public org.apache.calcite.rel.RelNode getRelNode()
public TableSchema getSchema()
public void printSchema()
public Table select(scala.collection.Seq<Expression> fields)
Example:
tab.select('key, 'value.avg + " The average" as 'average)
fields
- (undocumented)public Table select(String fields)
Example:
tab.select("key, value.avg + ' The average' as average")
fields
- (undocumented)public Table as(scala.collection.Seq<Expression> fields)
Example:
tab.as('a, 'b)
fields
- (undocumented)public Table as(String fields)
Example:
tab.as("a, b")
fields
- (undocumented)public Table filter(Expression predicate)
Example:
tab.filter('name === "Fred")
predicate
- (undocumented)public Table filter(String predicate)
Example:
tab.filter("name = 'Fred'")
predicate
- (undocumented)public Table where(Expression predicate)
Example:
tab.where('name === "Fred")
predicate
- (undocumented)public Table where(String predicate)
Example:
tab.where("name = 'Fred'")
predicate
- (undocumented)public GroupedTable groupBy(scala.collection.Seq<Expression> fields)
Example:
tab.groupBy('key).select('key, 'value.avg)
fields
- (undocumented)public GroupedTable groupBy(String fields)
Example:
tab.groupBy("key").select("key, value.avg")
fields
- (undocumented)public Table distinct()
Example:
tab.select("key, value").distinct()
public Table join(Table right)
Table
s. Similar to an SQL join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary. You can use
where and select clauses after a join to further specify the behaviour of the join.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.join(right).where('a === 'b && 'c > 3).select('a, 'b, 'd)
right
- (undocumented)public Table join(Table right, String joinPredicate)
Table
s. Similar to an SQL join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.join(right, "a = b")
right
- (undocumented)joinPredicate
- (undocumented)public Table join(Table right, Expression joinPredicate)
Table
s. Similar to an SQL join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.join(right, 'a === 'b).select('a, 'b, 'd)
right
- (undocumented)joinPredicate
- (undocumented)public Table leftOuterJoin(Table right)
Table
with an user-defined TableFunction
.
This join is similar to a SQL left outer join with ON TRUE predicate, but it works with a
table function. Each row of the outer table is joined with all rows produced by the table
function. If the table function does not produce any row, the outer row is padded with nulls.
Scala Example:
class MySplitUDTF extends TableFunction[String] {
def eval(str: String): Unit = {
str.split("#").foreach(collect)
}
}
val split = new MySplitUDTF()
table.leftOuterJoin(split('c) as ('s)).select('a,'b,'c,'s)
Java Example:
class MySplitUDTF extends TableFunction<String> {
public void eval(String str) {
str.split("#").forEach(this::collect);
}
}
TableFunction<String> split = new MySplitUDTF();
tableEnv.registerFunction("split", split);
table.leftOuterJoin(new Table(tableEnv, "split(c)").as("s"))).select("a, b, c, s");
right
- (undocumented)public Table leftOuterJoin(Table right, String joinPredicate)
Table
s. Similar to an SQL left outer join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
and its TableConfig
must
have nullCheck enabled.
Example:
left.leftOuterJoin(right, "a = b").select('a, 'b, 'd)
right
- (undocumented)joinPredicate
- (undocumented)public Table leftOuterJoin(Table right, Expression joinPredicate)
Table
s. Similar to an SQL left outer join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
and its TableConfig
must
have nullCheck enabled.
Example:
left.leftOuterJoin(right, 'a === 'b).select('a, 'b, 'd)
right
- (undocumented)joinPredicate
- (undocumented)public Table rightOuterJoin(Table right, String joinPredicate)
Table
s. Similar to an SQL right outer join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
and its TableConfig
must
have nullCheck enabled.
Example:
left.rightOuterJoin(right, "a = b").select('a, 'b, 'd)
right
- (undocumented)joinPredicate
- (undocumented)public Table rightOuterJoin(Table right, Expression joinPredicate)
Table
s. Similar to an SQL right outer join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
and its TableConfig
must
have nullCheck enabled.
Example:
left.rightOuterJoin(right, 'a === 'b).select('a, 'b, 'd)
right
- (undocumented)joinPredicate
- (undocumented)public Table fullOuterJoin(Table right, String joinPredicate)
Table
s. Similar to an SQL full outer join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
and its TableConfig
must
have nullCheck enabled.
Example:
left.fullOuterJoin(right, "a = b").select('a, 'b, 'd)
right
- (undocumented)joinPredicate
- (undocumented)public Table fullOuterJoin(Table right, Expression joinPredicate)
Table
s. Similar to an SQL full outer join. The fields of the two joined
operations must not overlap, use as
to rename fields if necessary.
Note: Both tables must be bound to the same TableEnvironment
and its TableConfig
must
have nullCheck enabled.
Example:
left.fullOuterJoin(right, 'a === 'b).select('a, 'b, 'd)
right
- (undocumented)joinPredicate
- (undocumented)public Table minus(Table right)
Table
s with duplicate records removed.
Similar to a SQL EXCEPT clause. Minus returns records from the left table that do not
exist in the right table. Duplicate records in the left table are returned
exactly once, i.e., duplicates are removed. Both tables must have identical field types.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.minus(right)
right
- (undocumented)public Table minusAll(Table right)
Table
s. Similar to an SQL EXCEPT ALL.
Similar to a SQL EXCEPT ALL clause. MinusAll returns the records that do not exist in
the right table. A record that is present n times in the left table and m times
in the right table is returned (n - m) times, i.e., as many duplicates as are present
in the right table are removed. Both tables must have identical field types.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.minusAll(right)
right
- (undocumented)public Table union(Table right)
Table
s with duplicate records removed.
Similar to an SQL UNION. The fields of the two union operations must fully overlap.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.union(right)
right
- (undocumented)public Table unionAll(Table right)
Table
s. Similar to an SQL UNION ALL. The fields of the two union operations
must fully overlap.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.unionAll(right)
right
- (undocumented)public Table intersect(Table right)
Table
s with duplicate records removed. Intersect returns records that
exist in both tables. If a record is present in one or both tables more than once, it is
returned just once, i.e., the resulting table has no duplicate records. Similar to an
SQL INTERSECT. The fields of the two intersect operations must fully overlap.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.intersect(right)
right
- (undocumented)public Table intersectAll(Table right)
Table
s. IntersectAll returns records that exist in both tables.
If a record is present in both tables more than once, it is returned as many times as it
is present in both tables, i.e., the resulting table might have duplicate records. Similar
to an SQL INTERSECT ALL. The fields of the two intersect operations must fully overlap.
Note: Both tables must be bound to the same TableEnvironment
.
Example:
left.intersectAll(right)
right
- (undocumented)public Table orderBy(scala.collection.Seq<Expression> fields)
Table
. Similar to SQL ORDER BY.
The resulting Table is globally sorted across all parallel partitions.
Example:
tab.orderBy('name.desc)
fields
- (undocumented)public Table orderBy(String fields)
Table
. Similar to SQL ORDER BY.
The resulting Table is sorted globally sorted across all parallel partitions.
Example:
tab.orderBy("name.desc")
fields
- (undocumented)public Table limit(int offset)
Example:
// returns unlimited number of records beginning with the 4th record
tab.orderBy('name.desc).limit(3)
offset
- number of records to skippublic Table limit(int offset, int fetch)
Example:
// returns 5 records beginning with the 4th record
tab.orderBy('name.desc).limit(3, 5)
offset
- number of records to skipfetch
- number of records to be returnedpublic <T> void writeToSink(TableSink<T> sink)
Table
to a TableSink
. A TableSink
defines an external storage location.
A batch Table
can only be written to a
BatchTableSink
, a streaming Table
requires a
AppendStreamTableSink
, a
RetractStreamTableSink
, or an
UpsertStreamTableSink
.
sink
- The TableSink
to which the Table
is written.public <T> void writeToSink(TableSink<T> sink, QueryConfig conf)
Table
to a TableSink
. A TableSink
defines an external storage location.
A batch Table
can only be written to a
BatchTableSink
, a streaming Table
requires a
AppendStreamTableSink
, a
RetractStreamTableSink
, or an
UpsertStreamTableSink
.
sink
- The TableSink
to which the Table
is written.conf
- The configuration for the query that writes to the sink.public WindowedTable window(Window window)
For streaming tables of infinite size, grouping into windows is required to define finite groups on which group-based aggregates can be computed.
For batch tables of finite size, windowing essentially provides shortcuts for time-based groupBy.
__Note__: Computing windowed aggregates on a streaming table is only a parallel operation
if additional grouping attributes are added to the groupBy(...)
clause.
If the groupBy(...)
only references a window alias, the streamed table will be processed
by a single task, i.e., with parallelism 1.
window
- window that specifies how elements are grouped.public OverWindowedTable window(scala.collection.Seq<OverWindow> overWindows)
An over-window defines for each record an interval of records over which aggregation functions can be computed.
Example:
table
.window(Over partitionBy 'c orderBy 'rowTime preceding 10.seconds as 'ow)
.select('c, 'b.count over 'ow, 'e.sum over 'ow)
__Note__: Computing over window aggregates on a streaming table is only a parallel operation if the window is partitioned. Otherwise, the whole stream will be processed by a single task, i.e., with parallelism 1.
__Note__: Over-windows for batch tables are currently not supported.
overWindows
- windows that specify the record interval over which aggregations are
computed.public String tableName()
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.