@Internal public class PythonProcessOperator<IN,OUT> extends OneInputPythonFunctionOperator<IN,OUT,Row,OUT>
PythonProcessOperator
is responsible for launching beam runner which will start a python
harness to execute user defined python ProcessFunction.bais, baisWrapper, baos, baosWrapper, collector, DATA_STREAM_STATELESS_FUNCTION_URN, jobOptions
elementCount, maxBundleSize, pythonFunctionRunner
chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService
Constructor and Description |
---|
PythonProcessOperator(Configuration config,
TypeInformation<IN> inputTypeInfo,
TypeInformation<OUT> outputTypeInfo,
DataStreamPythonFunctionInfo pythonFunctionInfo) |
Modifier and Type | Method and Description |
---|---|
void |
emitResult(Tuple2<byte[],Integer> resultTuple)
Sends the execution result to the downstream operator.
|
String |
getCoderUrn() |
String |
getFunctionUrn() |
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElement(StreamRecord<IN> element)
Processes one element that arrived on this input of the
MultipleInputStreamOperator . |
void |
processWatermark(Watermark mark)
Processes a
Watermark that arrived on the first input of this two-input operator. |
createPythonFunctionRunner, getInternalParameters, getPythonEnv
endInput
checkInvokeFinishBundleByCount, close, createPythonEnvironmentManager, dispose, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, isBundleFinished, prepareSnapshotPreBarrier, setPythonConfig
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
setKeyContextElement
close, dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
getCurrentKey, setCurrentKey
processLatencyMarker
public PythonProcessOperator(Configuration config, TypeInformation<IN> inputTypeInfo, TypeInformation<OUT> outputTypeInfo, DataStreamPythonFunctionInfo pythonFunctionInfo)
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<OUT>
open
in class OneInputPythonFunctionOperator<IN,OUT,Row,OUT>
Exception
- An exception in this method causes the operator to fail.public void emitResult(Tuple2<byte[],Integer> resultTuple) throws Exception
AbstractPythonFunctionOperator
emitResult
in class AbstractPythonFunctionOperator<OUT>
Exception
public void processElement(StreamRecord<IN> element) throws Exception
Input
MultipleInputStreamOperator
.
This method is guaranteed to not be called concurrently with other methods of the operator.processElement
in interface Input<IN>
processElement
in class OneInputPythonFunctionOperator<IN,OUT,Row,OUT>
Exception
public void processWatermark(Watermark mark) throws Exception
Input
Watermark
that arrived on the first input of this two-input operator.
This method is guaranteed to not be called concurrently with other methods of the operator.processWatermark
in interface Input<IN>
processWatermark
in class AbstractPythonFunctionOperator<OUT>
Exception
Watermark
public String getFunctionUrn()
getFunctionUrn
in class OneInputPythonFunctionOperator<IN,OUT,Row,OUT>
public String getCoderUrn()
getCoderUrn
in class OneInputPythonFunctionOperator<IN,OUT,Row,OUT>
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.