IN
- Type of the input elements.OUT
- Type of the execution results.@Internal public abstract class AbstractPythonFunctionRunner<IN,OUT> extends Object implements PythonFunctionRunner<IN>
PythonFunctionRunner
.Modifier and Type | Field and Description |
---|---|
protected static org.slf4j.Logger |
LOG
The logger used by the runner class and its subclasses.
|
Constructor and Description |
---|
AbstractPythonFunctionRunner(String taskName,
org.apache.beam.sdk.fn.data.FnDataReceiver<OUT> resultReceiver,
PythonEnvironmentManager environmentManager,
org.apache.beam.runners.fnexecution.state.StateRequestHandler stateRequestHandler) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Tear-down the Python function runner.
|
abstract org.apache.beam.runners.core.construction.graph.ExecutableStage |
createExecutableStage()
Creates a
ExecutableStage which contains the Python user-defined functions to be executed
and all the other information needed to execute them, such as the execution environment, the input
and output coder, etc. |
org.apache.beam.runners.fnexecution.control.JobBundleFactory |
createJobBundleFactory(org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.Struct pipelineOptions) |
protected org.apache.beam.model.pipeline.v1.RunnerApi.Environment |
createPythonExecutionEnvironment()
Creates a specification which specifies the portability Python execution environment.
|
void |
finishBundle()
Forces to finish the processing of the current bundle of elements.
|
abstract TypeSerializer<IN> |
getInputTypeSerializer()
Returns the TypeSerializer for input elements.
|
abstract TypeSerializer<OUT> |
getOutputTypeSerializer()
Returns the TypeSerializer for execution results.
|
void |
open()
Prepares the Python function runner, such as preparing the Python execution environment, etc.
|
void |
processElement(IN element)
Executes the Python function with the input element.
|
void |
startBundle()
Prepares to process the next bundle of elements.
|
protected static final org.slf4j.Logger LOG
public AbstractPythonFunctionRunner(String taskName, org.apache.beam.sdk.fn.data.FnDataReceiver<OUT> resultReceiver, PythonEnvironmentManager environmentManager, org.apache.beam.runners.fnexecution.state.StateRequestHandler stateRequestHandler)
public void open() throws Exception
PythonFunctionRunner
open
in interface PythonFunctionRunner<IN>
Exception
public void close() throws Exception
PythonFunctionRunner
close
in interface PythonFunctionRunner<IN>
Exception
public void startBundle()
PythonFunctionRunner
startBundle
in interface PythonFunctionRunner<IN>
public void finishBundle()
PythonFunctionRunner
finishBundle
in interface PythonFunctionRunner<IN>
public void processElement(IN element)
PythonFunctionRunner
processElement
in interface PythonFunctionRunner<IN>
@VisibleForTesting public org.apache.beam.runners.fnexecution.control.JobBundleFactory createJobBundleFactory(org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.Struct pipelineOptions) throws Exception
Exception
protected org.apache.beam.model.pipeline.v1.RunnerApi.Environment createPythonExecutionEnvironment() throws Exception
Exception
public abstract org.apache.beam.runners.core.construction.graph.ExecutableStage createExecutableStage() throws Exception
ExecutableStage
which contains the Python user-defined functions to be executed
and all the other information needed to execute them, such as the execution environment, the input
and output coder, etc.Exception
public abstract TypeSerializer<IN> getInputTypeSerializer()
public abstract TypeSerializer<OUT> getOutputTypeSerializer()
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.