@Internal public class EmbeddedPythonScalarFunctionOperator extends AbstractEmbeddedPythonFunctionOperator<RowData> implements OneInputStreamOperator<RowData,RowData>, BoundedOneInput
ScalarFunction
operator in embedded Python environment.Modifier and Type | Field and Description |
---|---|
protected RowType |
inputType
The input logical type.
|
protected RowType |
udfInputType
The user-defined function input logical type.
|
protected RowType |
udfOutputType
The user-defined function output logical type.
|
interpreter
bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled
chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService
Constructor and Description |
---|
EmbeddedPythonScalarFunctionOperator(Configuration config,
PythonFunctionInfo[] scalarFunctions,
RowType inputType,
RowType udfInputType,
RowType udfOutputType,
int[] udfInputOffsets) |
EmbeddedPythonScalarFunctionOperator(Configuration config,
PythonFunctionInfo[] scalarFunctions,
RowType inputType,
RowType udfInputType,
RowType udfOutputType,
int[] udfInputOffsets,
GeneratedProjection forwardedFieldGeneratedProjection) |
Modifier and Type | Method and Description |
---|---|
void |
endInput()
It is notified that no more data will arrive from the input.
|
PythonEnv |
getPythonEnv()
Returns the
PythonEnv used to create PythonEnvironmentManager. |
FlinkFnApi.UserDefinedFunctions |
getUserDefinedFunctionsProto()
Gets the proto representation of the Python user-defined functions to be executed.
|
protected void |
invokeFinishBundle() |
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
openPythonInterpreter(String pythonExecutable,
Map<String,String> env)
Setup method for Python Interpreter.
|
void |
processElement(StreamRecord<RowData> element)
Processes one element that arrived on this input of the
MultipleInputStreamOperator . |
close, createPythonEnvironmentManager
checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, registerCounterOnOutput, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
setKeyContextElement
close, finish, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
getCurrentKey, setCurrentKey
processLatencyMarker, processWatermark, processWatermarkStatus
protected final RowType inputType
protected final RowType udfInputType
protected final RowType udfOutputType
public EmbeddedPythonScalarFunctionOperator(Configuration config, PythonFunctionInfo[] scalarFunctions, RowType inputType, RowType udfInputType, RowType udfOutputType, int[] udfInputOffsets)
public EmbeddedPythonScalarFunctionOperator(Configuration config, PythonFunctionInfo[] scalarFunctions, RowType inputType, RowType udfInputType, RowType udfOutputType, int[] udfInputOffsets, GeneratedProjection forwardedFieldGeneratedProjection)
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<RowData>
open
in class AbstractEmbeddedPythonFunctionOperator<RowData>
Exception
- An exception in this method causes the operator to fail.public void openPythonInterpreter(String pythonExecutable, Map<String,String> env)
AbstractEmbeddedPythonFunctionOperator
openPythonInterpreter
in class AbstractEmbeddedPythonFunctionOperator<RowData>
public void endInput()
BoundedOneInput
WARNING: It is not safe to use this method to commit any transactions or other side
effects! You can use this method to flush any buffered data that can later on be committed
e.g. in a CheckpointListener.notifyCheckpointComplete(long)
.
NOTE: Given it is semantically very similar to the StreamOperator.finish()
method. It might be dropped in favour of the other method at some point in time.
endInput
in interface BoundedOneInput
public PythonEnv getPythonEnv()
AbstractEmbeddedPythonFunctionOperator
PythonEnv
used to create PythonEnvironmentManager.getPythonEnv
in class AbstractEmbeddedPythonFunctionOperator<RowData>
public void processElement(StreamRecord<RowData> element)
Input
MultipleInputStreamOperator
.
This method is guaranteed to not be called concurrently with other methods of the operator.processElement
in interface Input<RowData>
protected void invokeFinishBundle() throws Exception
invokeFinishBundle
in class AbstractPythonFunctionOperator<RowData>
Exception
public FlinkFnApi.UserDefinedFunctions getUserDefinedFunctionsProto()
AbstractEmbeddedPythonFunctionOperator
getUserDefinedFunctionsProto
in class AbstractEmbeddedPythonFunctionOperator<RowData>
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.