StreamGraphfrom a graph of
This traverses the tree of
StreamTransformations starting from the sinks. At each
transformation we recursively transform the inputs, then create a node in the
and add edges from the input Nodes to our newly created node. The transformation methods
return the IDs of the nodes in the StreamGraph that represent the input transformation. Several
IDs can be returned to be able to deal with feedback transformations and unions.
Partitioning, split/select and union don't create actual nodes in the
these, we create a virtual node in the
StreamGraph that holds the specific property, i.e.
partitioning, selector and so on. When an edge is created from a virtual node to a downstream
StreamGraph resolved the id of the original node and creates an edge
in the graph with the desired property. For example, if you have this graph:
Map-1 -> HashPartition-2 -> Map-3
where the numbers represent transformation IDs. We first recurse all the way down.
is transformed, i.e. we create a
StreamNode with ID 1. Then we transform the
HashPartition, for this, we create virtual node of ID 4 that holds the property
HashPartition. This transformation returns the ID 4. Then we transform the
We add the edge
4 -> 3. The
StreamGraph resolved the actual node with ID 1 and
creates and edge
1 -> 3 with the property HashPartition.
|Modifier and Type||Field and Description|
|Modifier and Type||Method and Description|
public static final int DEFAULT_LOWER_BOUND_MAX_PARALLELISM
public static final int UPPER_BOUND_MAX_PARALLELISM
protected static Integer iterationIdCounter
public static int getNewIterationNodeId()
public static StreamGraph generate(StreamExecutionEnvironment env, List<StreamTransformation<?>> transformations)
StreamGraphby traversing the graph of
StreamTransformationsstarting from the given transformations.
StreamExecutionEnvironmentthat is used to set some parameters of the job
transformations- The transformations starting from which to transform the graph
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.