IN
- Type of the input elements.@Internal public abstract class AbstractPythonFunctionRunner<IN> extends Object implements PythonFunctionRunner<IN>
PythonFunctionRunner
.Modifier and Type | Field and Description |
---|---|
protected ByteArrayOutputStreamWithPos |
baos
Reusable OutputStream used to holding the serialized input elements.
|
protected DataOutputViewStreamWrapper |
baosWrapper
OutputStream Wrapper.
|
protected FlinkMetricContainer |
flinkMetricContainer
The flinkMetricContainer will be set to null if metric is configured to be turned off.
|
protected org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<byte[]>> |
mainInputReceiver
The receiver which forwards the input elements to a remote environment for processing.
|
protected org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> |
resultReceiver
The Python function execution result receiver.
|
Constructor and Description |
---|
AbstractPythonFunctionRunner(String taskName,
org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> resultReceiver,
PythonEnvironmentManager environmentManager,
org.apache.beam.runners.fnexecution.state.StateRequestHandler stateRequestHandler,
Map<String,String> jobOptions,
FlinkMetricContainer flinkMetricContainer) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Tear-down the Python function runner.
|
abstract org.apache.beam.runners.core.construction.graph.ExecutableStage |
createExecutableStage()
Creates a
ExecutableStage which contains the Python user-defined functions to be
executed and all the other information needed to execute them, such as the execution
environment, the input and output coder, etc. |
org.apache.beam.runners.fnexecution.control.JobBundleFactory |
createJobBundleFactory(org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.Struct pipelineOptions) |
abstract org.apache.beam.runners.fnexecution.control.OutputReceiverFactory |
createOutputReceiverFactory() |
protected org.apache.beam.model.pipeline.v1.RunnerApi.Environment |
createPythonExecutionEnvironment()
Creates a specification which specifies the portability Python execution environment.
|
void |
finishBundle()
Forces to finish the processing of the current bundle of elements.
|
void |
open()
Prepares the Python function runner, such as preparing the Python execution environment, etc.
|
void |
startBundle()
Prepares to process the next bundle of elements.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
processElement
protected final org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> resultReceiver
protected transient org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<byte[]>> mainInputReceiver
protected transient ByteArrayOutputStreamWithPos baos
protected transient DataOutputViewStreamWrapper baosWrapper
@Nullable protected FlinkMetricContainer flinkMetricContainer
public AbstractPythonFunctionRunner(String taskName, org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> resultReceiver, PythonEnvironmentManager environmentManager, org.apache.beam.runners.fnexecution.state.StateRequestHandler stateRequestHandler, Map<String,String> jobOptions, @Nullable FlinkMetricContainer flinkMetricContainer)
public void open() throws Exception
PythonFunctionRunner
open
in interface PythonFunctionRunner<IN>
Exception
public void close() throws Exception
PythonFunctionRunner
close
in interface PythonFunctionRunner<IN>
Exception
public void startBundle()
PythonFunctionRunner
startBundle
in interface PythonFunctionRunner<IN>
public void finishBundle() throws Exception
PythonFunctionRunner
finishBundle
in interface PythonFunctionRunner<IN>
Exception
@VisibleForTesting public org.apache.beam.runners.fnexecution.control.JobBundleFactory createJobBundleFactory(org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.Struct pipelineOptions) throws Exception
Exception
protected org.apache.beam.model.pipeline.v1.RunnerApi.Environment createPythonExecutionEnvironment() throws Exception
Exception
public abstract org.apache.beam.runners.core.construction.graph.ExecutableStage createExecutableStage() throws Exception
ExecutableStage
which contains the Python user-defined functions to be
executed and all the other information needed to execute them, such as the execution
environment, the input and output coder, etc.Exception
public abstract org.apache.beam.runners.fnexecution.control.OutputReceiverFactory createOutputReceiverFactory()
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.