Package

org.apache.flink.api.scala

extensions

Permalink

package extensions

acceptPartialFunctions extends the original DataSet with methods with unique names that delegate to core higher-order functions (e.g. map) so that we can work around the fact that overloaded methods taking functions as parameters can't accept partial functions as well. This enables the possibility to directly apply pattern matching to decompose inputs such as tuples, case classes and collections.

The following is a small example that showcases how this extensions would work on a Flink data set:

object Main {
  import org.apache.flink.api.scala.extensions._
  case class Point(x: Double, y: Double)
  def main(args: Array[String]): Unit = {
    val env = ExecutionEnvironment.getExecutionEnvironment
    val ds = env.fromElements(Point(1, 2), Point(3, 4), Point(5, 6))
    ds.filterWith {
      case Point(x, _) => x > 1
    }.reduceWith {
      case (Point(x1, y1), (Point(x2, y2))) => Point(x1 + y1, x2 + y2)
    }.mapWith {
      case Point(x, y) => (x, y)
    }.flatMapWith {
      case (x, y) => Seq('x' -> x, 'y' -> y)
    }.groupingBy {
      case (id, value) => id
    }
  }
}

The extension consists of several implicit conversions over all the data set representations that could gain from this feature. To use this set of extensions methods the user has to explicitly opt-in by importing org.apache.flink.api.scala.extensions.acceptPartialFunctions.

For more information and usage examples please consult the Apache Flink official documentation.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. extensions
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. implicit def acceptPartialFunctions[L, R, O](ds: UnfinishedKeyPairOperation[L, R, O]): OnUnfinishedKeyPairOperation[L, R, O]

    Permalink
    Annotations
    @PublicEvolving()
  2. implicit def acceptPartialFunctions[L, R, O](ds: HalfUnfinishedKeyPairOperation[L, R, O]): OnHalfUnfinishedKeyPairOperation[L, R, O]

    Permalink
    Annotations
    @PublicEvolving()
  3. implicit def acceptPartialFunctions[L, R](ds: CoGroupDataSet[L, R]): OnCoGroupDataSet[L, R]

    Permalink
    Annotations
    @PublicEvolving()
  4. implicit def acceptPartialFunctions[T](ds: GroupedDataSet[T]): OnGroupedDataSet[T]

    Permalink
    Annotations
    @PublicEvolving()
  5. implicit def acceptPartialFunctions[L, R](ds: CrossDataSet[L, R]): OnCrossDataSet[L, R]

    Permalink
    Annotations
    @PublicEvolving()
  6. implicit def acceptPartialFunctions[L, R](ds: JoinFunctionAssigner[L, R]): OnJoinFunctionAssigner[L, R]

    Permalink
    Annotations
    @PublicEvolving()
  7. implicit def acceptPartialFunctions[T](ds: DataSet[T]): OnDataSet[T]

    Permalink
    Annotations
    @PublicEvolving()
  8. package impl

    Permalink

Inherited from AnyRef

Inherited from Any

Ungrouped