This documentation is for an unreleased version of Apache Flink Stateful Functions. We recommend you use the latest stable version.
I/O Module #
Stateful Functions' I/O modules allow functions to receive and send messages to external systems. Based on the concept of Ingress (input) and Egress (output) points, and built on top of the Apache Flink® connector ecosystem, I/O modules enable functions to interact with the outside world through the style of message passing.
Commonly used I/O modules are bundled into the runtime by default and can be configured direclty via the applications module configuration. Additionally, custom connectors for other systems can be plugged in to the runtime.
Keep in mind that to use one of these connectors in an application, additional third party components are usually required, e.g. servers for the data stores or message queues.
An Ingress is an input point where data is consumed from an external system and forwarded to zero or more functions.
It is defined via an
IngressIdentifier and an
An ingress identifier, similar to a function type, uniquely identifies an ingress by specifying its input type, a namespace, and a name.
The spec defines the details of how to connect to the external system, which is specific to each individual I/O module. Each identifier-spec pair is bound to the system inside an stateful function module.
version: "3.0" module: meta: type: remote spec: ingresses: - ingress: meta: id: example/user-ingress type: # ingress type spec: # ingress specific configurations
A router is a stateless operator that takes each record from an ingress and routes it to zero or more functions. Routers are bound to the system via a stateful function module, and unlike other components, an ingress may have any number of routers.
Routers are defined by a list of function types.
id component of the address is pulled from the key associated with each record in its underlying source implementation.
targets: - example-namespace/my-function-1 - example-namespace/my-function-2
Egress is the opposite of ingress; it is a point that takes messages and writes them to external systems.
Each egress is defined using two components, an
EgressIdentifier and an
An egress identifier uniquely identifies an egress based on a namespace, name, and producing type. An egress spec defines the details of how to connect to the external system, the details are specific to each individual I/O module. Each identifier-spec pair is bound to the system inside a Stateful Functions module.
version: "3.0" module: meta: type: remote spec: egresses: - egress: meta: id: example/user-egress type: # egress type spec: # egress specific configurations