Modifier and Type | Method and Description |
---|---|
<X> DataSet<X> |
DataSet.runOperation(CustomUnaryOperation<T,X> operation)
Runs a
CustomUnaryOperation on the data set. |
Modifier and Type | Method and Description |
---|---|
protected static void |
DataSet.checkSameExecutionContext(DataSet<?> set1,
DataSet<?> set2) |
protected static void |
DataSet.checkSameExecutionContext(DataSet<?> set1,
DataSet<?> set2) |
<R> CoGroupOperator.CoGroupOperatorSets<T,R> |
DataSet.coGroup(DataSet<R> other)
Initiates a CoGroup transformation.
|
<R> CrossOperator.DefaultCross<T,R> |
DataSet.cross(DataSet<R> other)
Initiates a Cross transformation.
|
<R> CrossOperator.DefaultCross<T,R> |
DataSet.crossWithHuge(DataSet<R> other)
Initiates a Cross transformation.
|
<R> CrossOperator.DefaultCross<T,R> |
DataSet.crossWithTiny(DataSet<R> other)
Initiates a Cross transformation.
|
<R> JoinOperatorSetsBase<T,R> |
DataSet.fullOuterJoin(DataSet<R> other)
Initiates a Full Outer Join transformation.
|
<R> JoinOperatorSetsBase<T,R> |
DataSet.fullOuterJoin(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Full Outer Join transformation.
|
<R> DeltaIteration<T,R> |
DataSet.iterateDelta(DataSet<R> workset,
int maxIterations,
int... keyPositions)
Initiates a delta iteration.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.join(DataSet<R> other)
Initiates a Join transformation.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.join(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Join transformation.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.joinWithHuge(DataSet<R> other)
Initiates a Join transformation.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.joinWithTiny(DataSet<R> other)
Initiates a Join transformation.
|
<R> JoinOperatorSetsBase<T,R> |
DataSet.leftOuterJoin(DataSet<R> other)
Initiates a Left Outer Join transformation.
|
<R> JoinOperatorSetsBase<T,R> |
DataSet.leftOuterJoin(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Left Outer Join transformation.
|
<R> JoinOperatorSetsBase<T,R> |
DataSet.rightOuterJoin(DataSet<R> other)
Initiates a Right Outer Join transformation.
|
<R> JoinOperatorSetsBase<T,R> |
DataSet.rightOuterJoin(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Right Outer Join transformation.
|
UnionOperator<T> |
DataSet.union(DataSet<T> other)
Creates a union of this DataSet with an other DataSet.
|
Modifier and Type | Class and Description |
---|---|
class |
AggregateOperator<IN>
This operator represents the application of a "aggregate" operation on a data set, and the result
data set produced by the function.
|
class |
BulkIterationResultSet<T>
Resulting
DataSet of bulk iterations. |
class |
CoGroupOperator<I1,I2,OUT>
A
DataSet that is the result of a CoGroup transformation. |
class |
CoGroupRawOperator<I1,I2,OUT>
A
DataSet that is the result of a CoGroup transformation. |
class |
CrossOperator<I1,I2,OUT>
A
DataSet that is the result of a Cross transformation. |
static class |
CrossOperator.DefaultCross<I1,I2>
A Cross transformation that wraps pairs of crossed elements into
Tuple2 . |
static class |
CrossOperator.ProjectCross<I1,I2,OUT extends Tuple>
|
class |
DataSource<OUT>
An operation that creates a new data set (data source).
|
static class |
DeltaIteration.SolutionSetPlaceHolder<ST>
A
DataSet that acts as a placeholder for the solution set during the iteration. |
static class |
DeltaIteration.WorksetPlaceHolder<WT>
A
DataSet that acts as a placeholder for the workset during the iteration. |
class |
DeltaIterationResultSet<ST,WT>
Resulting
DataSet of a delta iteration operation. |
class |
DistinctOperator<T>
This operator represents the application of a "distinct" function on a data set, and the result
data set produced by the function.
|
class |
FilterOperator<T>
This operator represents the application of a "filter" function on a data set, and the result
data set produced by the function.
|
class |
FlatMapOperator<IN,OUT>
This operator represents the application of a "flatMap" function on a data set, and the result
data set produced by the function.
|
class |
GroupCombineOperator<IN,OUT>
This operator behaves like the GroupReduceOperator with Combine but only runs the Combine part
which reduces all data locally in their partitions.
|
class |
GroupReduceOperator<IN,OUT>
This operator represents the application of a "reduceGroup" function on a data set, and the
result data set produced by the function.
|
class |
IterativeDataSet<T>
The IterativeDataSet represents the start of an iteration.
|
class |
JoinOperator<I1,I2,OUT>
A
DataSet that is the result of a Join transformation. |
static class |
JoinOperator.DefaultJoin<I1,I2>
A Join transformation that wraps pairs of joining elements into
Tuple2 . |
static class |
JoinOperator.EquiJoin<I1,I2,OUT>
A Join transformation that applies a
JoinFunction on each pair of joining elements. |
static class |
JoinOperator.ProjectJoin<I1,I2,OUT extends Tuple>
|
class |
MapOperator<IN,OUT>
This operator represents the application of a "map" function on a data set, and the result data
set produced by the function.
|
class |
MapPartitionOperator<IN,OUT>
This operator represents the application of a "mapPartition" function on a data set, and the
result data set produced by the function.
|
class |
NoOpOperator<IN>
This operator will be ignored during translation.
|
class |
Operator<OUT,O extends Operator<OUT,O>>
Base class of all operators in the Java API.
|
class |
PartitionOperator<T>
This operator represents a partitioning.
|
class |
ProjectOperator<IN,OUT extends Tuple>
This operator represents the application of a projection operation on a data set, and the result
data set produced by the function.
|
class |
ReduceOperator<IN>
This operator represents the application of a "reduce" function on a data set, and the result
data set produced by the function.
|
class |
SingleInputOperator<IN,OUT,O extends SingleInputOperator<IN,OUT,O>>
Base class for operations that operates on a single input data set.
|
class |
SingleInputUdfOperator<IN,OUT,O extends SingleInputUdfOperator<IN,OUT,O>>
The SingleInputUdfOperator is the base class of all unary operators that execute
user-defined functions (UDFs).
|
class |
SortPartitionOperator<T>
This operator represents a DataSet with locally sorted partitions.
|
class |
TwoInputOperator<IN1,IN2,OUT,O extends TwoInputOperator<IN1,IN2,OUT,O>>
Base class for operations that operates on two input data sets.
|
class |
TwoInputUdfOperator<IN1,IN2,OUT,O extends TwoInputUdfOperator<IN1,IN2,OUT,O>>
The TwoInputUdfOperator is the base class of all binary operators that execute
user-defined functions (UDFs).
|
class |
UnionOperator<T>
Java API operator for union of two data sets.
|
Modifier and Type | Field and Description |
---|---|
protected DataSet<T> |
Grouping.inputDataSet |
Modifier and Type | Method and Description |
---|---|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult)
Closes the iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
DataSet<OUT> |
CustomUnaryOperation.createResult() |
DataSet<T> |
DataSink.getDataSet() |
DataSet<ST> |
DeltaIteration.getInitialSolutionSet()
Gets the initial solution set.
|
DataSet<WT> |
DeltaIteration.getInitialWorkset()
Gets the initial workset.
|
DataSet<IN> |
NoOpOperator.getInput() |
DataSet<IN> |
SingleInputOperator.getInput()
Gets the data set that this operation uses as its input.
|
DataSet<IN1> |
TwoInputOperator.getInput1()
Gets the data set that this operation uses as its first input.
|
DataSet<IN2> |
TwoInputOperator.getInput2()
Gets the data set that this operation uses as its second input.
|
DataSet<T> |
Grouping.getInputDataSet()
Returns the input DataSet of a grouping operation, that is the one before the grouping.
|
DataSet<T> |
BulkIterationResultSet.getNextPartialSolution() |
DataSet<ST> |
DeltaIterationResultSet.getNextSolutionSet() |
DataSet<WT> |
DeltaIterationResultSet.getNextWorkset() |
DataSet<?> |
BulkIterationResultSet.getTerminationCriterion() |
Modifier and Type | Method and Description |
---|---|
Map<String,DataSet<?>> |
TwoInputUdfOperator.getBroadcastSets() |
Map<String,DataSet<?>> |
SingleInputUdfOperator.getBroadcastSets() |
Map<String,DataSet<?>> |
UdfOperator.getBroadcastSets()
Gets the broadcast sets (name and data set) that have been added to context of the UDF.
|
Modifier and Type | Method and Description |
---|---|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult)
Closes the iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
void |
NoOpOperator.setInput(DataSet<IN> input) |
void |
CustomUnaryOperation.setInput(DataSet<IN> inputData) |
O |
TwoInputUdfOperator.withBroadcastSet(DataSet<?> data,
String name) |
O |
SingleInputUdfOperator.withBroadcastSet(DataSet<?> data,
String name) |
O |
UdfOperator.withBroadcastSet(DataSet<?> data,
String name)
Adds a certain data set as a broadcast set to this operator.
|
Constructor and Description |
---|
AggregateOperator(DataSet<IN> input,
Aggregations function,
int field,
String aggregateLocationName)
Non grouped aggregation.
|
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
CoGroupOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
CoGroupRawOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
String defaultName) |
CoGroupRawOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
String defaultName) |
CrossOperator(DataSet<I1> input1,
DataSet<I2> input2,
CrossFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
CrossOperatorBase.CrossHint hint,
String defaultName) |
CrossOperator(DataSet<I1> input1,
DataSet<I2> input2,
CrossFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
CrossOperatorBase.CrossHint hint,
String defaultName) |
CrossProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
int[] firstFieldIndexes,
int[] secondFieldIndexes,
CrossOperatorBase.CrossHint hint) |
CrossProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
int[] firstFieldIndexes,
int[] secondFieldIndexes,
CrossOperatorBase.CrossHint hint) |
DataSink(DataSet<T> data,
OutputFormat<T> format,
TypeInformation<T> type) |
DefaultCross(DataSet<I1> input1,
DataSet<I2> input2,
CrossOperatorBase.CrossHint hint,
String defaultName) |
DefaultCross(DataSet<I1> input1,
DataSet<I2> input2,
CrossOperatorBase.CrossHint hint,
String defaultName) |
DefaultJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
DefaultJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
DeltaIteration(ExecutionEnvironment context,
TypeInformation<ST> type,
DataSet<ST> solutionSet,
DataSet<WT> workset,
Keys<ST> keys,
int maxIterations) |
DeltaIteration(ExecutionEnvironment context,
TypeInformation<ST> type,
DataSet<ST> solutionSet,
DataSet<WT> workset,
Keys<ST> keys,
int maxIterations) |
DistinctOperator(DataSet<T> input,
Keys<T> keys,
String distinctLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
FilterOperator(DataSet<T> input,
FilterFunction<T> function,
String defaultName) |
FlatMapOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
FlatMapFunction<IN,OUT> function,
String defaultName) |
GroupCombineOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
GroupCombineFunction<IN,OUT> function,
String defaultName)
Constructor for a non-grouped reduce (all reduce).
|
Grouping(DataSet<T> set,
Keys<T> keys) |
GroupReduceOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
GroupReduceFunction<IN,OUT> function,
String defaultName)
Constructor for a non-grouped reduce (all reduce).
|
IterativeDataSet(ExecutionEnvironment context,
TypeInformation<T> type,
DataSet<T> input,
int maxIterations) |
JoinOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
JoinType type) |
JoinOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
JoinType type) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
MapOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
MapFunction<IN,OUT> function,
String defaultName) |
MapPartitionOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
MapPartitionFunction<IN,OUT> function,
String defaultName) |
NoOpOperator(DataSet<IN> input,
TypeInformation<IN> resultType) |
PartitionOperator(DataSet<T> input,
Keys<T> pKeys,
Partitioner<?> customPartitioner,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
Keys<T> pKeys,
Partitioner<P> customPartitioner,
TypeInformation<P> partitionerTypeInfo,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
Keys<T> pKeys,
DataDistribution distribution,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
Keys<T> pKeys,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
String partitionLocationName) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperator.CrossProjection<I1,I2> crossProjection,
CrossOperatorBase.CrossHint hint) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperator.CrossProjection<I1,I2> crossProjection,
CrossOperatorBase.CrossHint hint) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperatorBase.CrossHint hint) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperatorBase.CrossHint hint) |
Projection(DataSet<T> ds,
int[] fieldIndexes) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
org.apache.flink.api.java.operators.JoinOperator.JoinProjection<I1,I2> joinProj) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
org.apache.flink.api.java.operators.JoinOperator.JoinProjection<I1,I2> joinProj) |
ProjectOperator(DataSet<IN> input,
int[] fields,
TupleTypeInfo<OUT> returnType) |
ReduceOperator(DataSet<IN> input,
ReduceFunction<IN> function,
String defaultName)
This is the case for a reduce-all case (in contrast to the reduce-per-group case).
|
SingleInputOperator(DataSet<IN> input,
TypeInformation<OUT> resultType) |
SingleInputUdfOperator(DataSet<IN> input,
TypeInformation<OUT> resultType)
Creates a new operators with the given data set as input.
|
SortedGrouping(DataSet<T> set,
Keys<T> keys,
int field,
Order order) |
SortedGrouping(DataSet<T> set,
Keys<T> keys,
Keys.SelectorFunctionKeys<T,K> keySelector,
Order order) |
SortedGrouping(DataSet<T> set,
Keys<T> keys,
String field,
Order order) |
SortPartitionOperator(DataSet<T> dataSet,
int sortField,
Order sortOrder,
String sortLocationName) |
SortPartitionOperator(DataSet<T> dataSet,
Keys.SelectorFunctionKeys<T,K> sortKey,
Order sortOrder,
String sortLocationName) |
SortPartitionOperator(DataSet<T> dataSet,
String sortField,
Order sortOrder,
String sortLocationName) |
TwoInputOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType) |
TwoInputOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType) |
TwoInputUdfOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType)
Creates a new operators with the two given data sets as inputs.
|
TwoInputUdfOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType)
Creates a new operators with the two given data sets as inputs.
|
UnionOperator(DataSet<T> input1,
DataSet<T> input2,
String unionLocationName)
Create an operator that produces the union of the two given data sets.
|
UnionOperator(DataSet<T> input1,
DataSet<T> input2,
String unionLocationName)
Create an operator that produces the union of the two given data sets.
|
UnsortedGrouping(DataSet<T> set,
Keys<T> keys) |
Modifier and Type | Field and Description |
---|---|
protected DataSet<I1> |
JoinOperatorSetsBase.input1 |
protected DataSet<I2> |
JoinOperatorSetsBase.input2 |
Constructor and Description |
---|
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint,
JoinType type) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint,
JoinType type) |
Modifier and Type | Method and Description |
---|---|
static <T> DataSet<Tuple2<Integer,Long>> |
DataSetUtils.countElementsPerPartition(DataSet<T> input)
Method that goes over all the elements in each partition in order to retrieve the total
number of elements.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples)
Generate a sample of DataSet which contains fixed size elements.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples,
long seed)
Generate a sample of DataSet which contains fixed size elements.
|
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithIndex(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set. |
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithUniqueId(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set as
described below. |
Modifier and Type | Method and Description |
---|---|
static <T> Utils.ChecksumHashCode |
DataSetUtils.checksumHashCode(DataSet<T> input)
Deprecated.
This method will be removed at some point.
|
static <T> DataSet<Tuple2<Integer,Long>> |
DataSetUtils.countElementsPerPartition(DataSet<T> input)
Method that goes over all the elements in each partition in order to retrieve the total
number of elements.
|
static <T> PartitionOperator<T> |
DataSetUtils.partitionByRange(DataSet<T> input,
DataDistribution distribution,
int... fields)
Range-partitions a DataSet on the specified tuple field positions.
|
static <T,K extends Comparable<K>> |
DataSetUtils.partitionByRange(DataSet<T> input,
DataDistribution distribution,
KeySelector<T,K> keyExtractor)
Range-partitions a DataSet using the specified key selector function.
|
static <T> PartitionOperator<T> |
DataSetUtils.partitionByRange(DataSet<T> input,
DataDistribution distribution,
String... fields)
Range-partitions a DataSet on the specified fields.
|
static <T> MapPartitionOperator<T,T> |
DataSetUtils.sample(DataSet<T> input,
boolean withReplacement,
double fraction)
Generate a sample of DataSet by the probability fraction of each element.
|
static <T> MapPartitionOperator<T,T> |
DataSetUtils.sample(DataSet<T> input,
boolean withReplacement,
double fraction,
long seed)
Generate a sample of DataSet by the probability fraction of each element.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples)
Generate a sample of DataSet which contains fixed size elements.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples,
long seed)
Generate a sample of DataSet which contains fixed size elements.
|
static <R extends Tuple,T extends Tuple> |
DataSetUtils.summarize(DataSet<T> input)
Summarize a DataSet of Tuples by collecting single pass statistics for all columns.
|
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithIndex(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set. |
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithUniqueId(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set as
described below. |
Modifier and Type | Class and Description |
---|---|
class |
ScalaAggregateOperator<IN>
This operator represents the application of a "aggregate" operation on a data set, and the result
data set produced by the function.
|
Constructor and Description |
---|
ScalaAggregateOperator(DataSet<IN> input,
Aggregations function,
int field)
Non grouped aggregation.
|
Modifier and Type | Method and Description |
---|---|
static DataSet<KMeans.Centroid> |
KMeansData.getDefaultCentroidDataSet(ExecutionEnvironment env) |
static DataSet<KMeans.Point> |
KMeansData.getDefaultPointDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
static DataSet<Tuple2<Long,Long>> |
PageRankData.getDefaultEdgeDataSet(ExecutionEnvironment env) |
static DataSet<EnumTrianglesDataTypes.Edge> |
EnumTrianglesData.getDefaultEdgeDataSet(ExecutionEnvironment env) |
static DataSet<Tuple2<Long,Long>> |
ConnectedComponentsData.getDefaultEdgeDataSet(ExecutionEnvironment env) |
static DataSet<Long> |
PageRankData.getDefaultPagesDataSet(ExecutionEnvironment env) |
static DataSet<Long> |
ConnectedComponentsData.getDefaultVertexDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
static DataSet<Tuple2<String,String>> |
WebLogData.getDocumentDataSet(ExecutionEnvironment env) |
static DataSet<Tuple3<Integer,String,Integer>> |
WebLogData.getRankDataSet(ExecutionEnvironment env) |
static DataSet<Tuple2<String,String>> |
WebLogData.getVisitDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
static DataSet<String> |
WordCountData.getDefaultTextLineDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
<K,T,ACC,R> |
EvictingWindowReader.aggregate(String uid,
AggregateFunction<T,ACC,R> aggregateFunction,
TypeInformation<K> keyType,
TypeInformation<T> inputType,
TypeInformation<R> outputType)
Deprecated.
Reads window state generated using an
AggregateFunction . |
<K,T,ACC,R,OUT> |
EvictingWindowReader.aggregate(String uid,
AggregateFunction<T,ACC,R> aggregateFunction,
WindowReaderFunction<R,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> inputType,
TypeInformation<OUT> outputType)
Deprecated.
Reads window state generated using an
AggregateFunction . |
<K,T,OUT> DataSet<OUT> |
EvictingWindowReader.process(String uid,
WindowReaderFunction<T,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> stateType,
TypeInformation<OUT> outputType)
Deprecated.
Reads window state generated without any preaggregation such as
WindowedStream#apply
and WindowedStream#process . |
<T,K> DataSet<T> |
EvictingWindowReader.reduce(String uid,
ReduceFunction<T> function,
TypeInformation<K> keyType,
TypeInformation<T> reduceType)
Deprecated.
Reads window state generated using a
ReduceFunction . |
<K,T,OUT> DataSet<OUT> |
EvictingWindowReader.reduce(String uid,
ReduceFunction<T> function,
WindowReaderFunction<T,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> reduceType,
TypeInformation<OUT> outputType)
Deprecated.
Reads window state generated using a
ReduceFunction . |
Modifier and Type | Method and Description |
---|---|
static <T> OneInputOperatorTransformation<T> |
OperatorTransformation.bootstrapWith(DataSet<T> dataSet)
Deprecated.
use
OperatorTransformation.bootstrapWith(DataStream) to bootstrap a savepoint using the data
stream api under batch execution. |
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.