Flink : Scala 1.20-SNAPSHOT API < Back

Packages

package root

Definition Classes
root
package org

Definition Classes
root
package apache

Definition Classes
org
package flink

Definition Classes
apache
package api

Definition Classes
flink
package scala
The Flink Scala API.
The Flink Scala API. org.apache.flink.api.scala.ExecutionEnvironment is the starting-point of any Flink program. It can be used to read from local files, HDFS, or other sources. org.apache.flink.api.scala.DataSet is the main abstraction of data in Flink. It provides operations that create new DataSets via transformations. org.apache.flink.api.scala.GroupedDataSet provides operations on grouped data that results from org.apache.flink.api.scala.DataSet.groupBy().
Use org.apache.flink.api.scala.ExecutionEnvironment.getExecutionEnvironment to obtain an execution environment. This will either create a local environment or a remote environment, depending on the context where your program is executing.

Definition Classes
api
Deprecated
All Flink Scala APIs are deprecated and will be removed in a future Flink version. You can still build your application in Scala, but you should move to the Java version of either the DataStream and/or Table API.
See also
FLIP-265 Deprecate and remove Scala API support
package extensions
acceptPartialFunctions extends the original DataSet with methods with unique names that delegate to core higher-order functions (e.g.
acceptPartialFunctions extends the original DataSet with methods with unique names that delegate to core higher-order functions (e.g. map) so that we can work around the fact that overloaded methods taking functions as parameters can't accept partial functions as well. This enables the possibility to directly apply pattern matching to decompose inputs such as tuples, case classes and collections.
The following is a small example that showcases how this extensions would work on a Flink data set:
```
object Main {
  import org.apache.flink.api.scala.extensions._
  case class Point(x: Double, y: Double)
  def main(args: Array[String]): Unit = {
    val env = ExecutionEnvironment.getExecutionEnvironment
    val ds = env.fromElements(Point(1, 2), Point(3, 4), Point(5, 6))
    ds.filterWith {
      case Point(x, _) => x > 1
    }.reduceWith {
      case (Point(x1, y1), (Point(x2, y2))) => Point(x1 + y1, x2 + y2)
    }.mapWith {
      case Point(x, y) => (x, y)
    }.flatMapWith {
      case (x, y) => Seq('x' -> x, 'y' -> y)
    }.groupingBy {
      case (id, value) => id
    }
  }
}
```
The extension consists of several implicit conversions over all the data set representations that could gain from this feature. To use this set of extensions methods the user has to explicitly opt-in by importing org.apache.flink.api.scala.extensions.acceptPartialFunctions.
For more information and usage examples please consult the Apache Flink official documentation.
Definition Classes
scala
Deprecated
All Flink Scala APIs are deprecated and will be removed in a future Flink major version. You can still build your application in Scala, but you should move to the Java version of either the DataStream and/or Table API.
See also
FLIP-265 Deprecate and remove Scala API support
package metrics

Definition Classes
scala
package operators

Definition Classes
scala
package typeutils

Definition Classes
scala
package utils

Definition Classes
scala
AggregateDataSet
ClosureCleaner
CoGroupDataSet
CrossDataSet
DataSet
ExecutionEnvironment
GroupedDataSet
JoinDataSet
JoinFunctionAssigner
PartitionSortedDataSet
SelectByMaxFunction
SelectByMinFunction
Tuple2CaseClassSerializer
UnfinishedCoGroupOperation
UnfinishedJoinOperation
UnfinishedOuterJoinOperation

org.apache.flink.api.scala

UnfinishedJoinOperation

class UnfinishedJoinOperation[L, R] extends UnfinishedJoinOperationBase[L, R, JoinDataSet[L, R]]

An unfinished inner join operation that results from calling DataSet.join(). The keys for the left and right side must be specified using first where and then equalTo.

For example:

val left = ...
val right = ...
val joinResult = left.join(right).where(...).equalTo(...)

L: The type of the left input of the join.
R: The type of the right input of the join.

Annotations: @deprecated @Public()
Deprecated: (Since version 1.18.0)

Linear Supertypes

UnfinishedJoinOperationBase[L, R, JoinDataSet[L, R]], UnfinishedKeyPairOperation[L, R, JoinDataSet[L, R]], AnyRef, Any

Ordering

Alphabetic
By Inheritance

Inherited

UnfinishedJoinOperation
UnfinishedJoinOperationBase
UnfinishedKeyPairOperation
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Instance Constructors

new UnfinishedJoinOperation(leftSet: DataSet[L], rightSet: DataSet[R], joinHint: JoinHint)

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@native() @throws( ... )
def createJoinFunctionAssigner(leftKey: Keys[L], rightKey: Keys[R]): JoinDataSet[L, R]

Definition Classes
UnfinishedJoinOperation → UnfinishedJoinOperationBase
Annotations
@Internal()
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val joinHint: JoinHint

Definition Classes
UnfinishedJoinOperationBase
val joinType: JoinType

Definition Classes
UnfinishedJoinOperationBase
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@native() @throws( ... )
def where[K](fun: KeySelector[L, K])(implicit arg0: TypeInformation[K]): HalfUnfinishedKeyPairOperation[L, R, JoinDataSet[L, R]]
Specify the key selector function for the left side of the key based operation.
Specify the key selector function for the left side of the key based operation. This returns a HalfUnfinishedKeyPairOperation on which equalTo must be called to specify the key for the right side. The result after specifying the right side key is the finished operation.

Definition Classes
UnfinishedKeyPairOperation
def where[K](fun: (L) ⇒ K)(implicit arg0: TypeInformation[K]): HalfUnfinishedKeyPairOperation[L, R, JoinDataSet[L, R]]
Specify the key selector function for the left side of the key based operation.
Specify the key selector function for the left side of the key based operation. This returns a HalfUnfinishedKeyPairOperation on which equalTo must be called to specify the key for the right side. The result after specifying the right side key is the finished operation.

Definition Classes
UnfinishedKeyPairOperation
def where(firstLeftField: String, otherLeftFields: String*): HalfUnfinishedKeyPairOperation[L, R, JoinDataSet[L, R]]
Specify the key fields for the left side of the key based operation.
Specify the key fields for the left side of the key based operation. This returns a HalfUnfinishedKeyPairOperation on which equalTo must be called to specify the key for the right side. The result after specifying the right side key is the finished operation.

Definition Classes
UnfinishedKeyPairOperation
def where(leftKeys: Int*): HalfUnfinishedKeyPairOperation[L, R, JoinDataSet[L, R]]
Specify the key fields for the left side of the key based operation.
Specify the key fields for the left side of the key based operation. This returns a HalfUnfinishedKeyPairOperation on which equalTo must be called to specify the key for the right side. The result after specifying the right side key is the finished operation.
This only works on Tuple DataSet.

Definition Classes
UnfinishedKeyPairOperation

Packages

UnfinishedJoinOperation

class UnfinishedJoinOperation[L, R] extends UnfinishedJoinOperationBase[L, R, JoinDataSet[L, R]]

Instance Constructors

Value Members

Inherited from UnfinishedJoinOperationBase[L, R, JoinDataSet[L, R]]

Inherited from UnfinishedKeyPairOperation[L, R, JoinDataSet[L, R]]

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

UnfinishedJoinOperation 

class UnfinishedJoinOperation[L, R] extends UnfinishedJoinOperationBase[L, R, JoinDataSet[L, R]]

Instance Constructors

Value Members

Inherited from UnfinishedJoinOperationBase[L, R, JoinDataSet[L, R]]

Inherited from UnfinishedKeyPairOperation[L, R, JoinDataSet[L, R]]

Inherited from AnyRef

Inherited from Any

Ungrouped

UnfinishedJoinOperation