public class PythonKeyedCoProcessOperator<OUT> extends TwoInputPythonFunctionOperator<Row,Row,Row,OUT> implements ResultTypeQueryable<OUT>, Triggerable<Row,VoidNamespace>
bais, baisWrapper, baos, baosWrapper, collector, reuseRow
elementCount, maxBundleSize, pythonFunctionRunner
chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService
Constructor and Description |
---|
PythonKeyedCoProcessOperator(Configuration config,
TypeInformation<Row> inputTypeInfo1,
TypeInformation<Row> inputTypeInfo2,
TypeInformation<OUT> outputTypeInfo,
DataStreamPythonFunctionInfo pythonFunctionInfo) |
Modifier and Type | Method and Description |
---|---|
PythonFunctionRunner |
createPythonFunctionRunner()
Creates the
PythonFunctionRunner which is responsible for Python user-defined
function execution. |
void |
emitResult(Tuple2<byte[],Integer> resultTuple)
Sends the execution result to the downstream operator.
|
Object |
getCurrentKey() |
TypeInformation<OUT> |
getProducedType()
Gets the data type (as a
TypeInformation ) produced by this function or input format. |
void |
onEventTime(InternalTimer<Row,VoidNamespace> timer)
Invoked when an event-time timer fires.
|
void |
onProcessingTime(InternalTimer<Row,VoidNamespace> timer)
Invoked when a processing-time timer fires.
|
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElement1(StreamRecord<Row> element)
Processes one element that arrived on the first input of this two-input operator.
|
void |
processElement2(StreamRecord<Row> element)
Processes one element that arrived on the second input of this two-input operator.
|
void |
setCurrentKey(Object key)
As the beam state gRPC service will access the KeyedStateBackend in parallel with this
operator, we must override this method to prevent changing the current key of the
KeyedStateBackend while the beam service is handling requests.
|
getCoderUrn, getJobOptions, getPythonEnv, getPythonFunctionInfo, getRunnerInputTypeInfo, getRunnerInputTypeSerializer, getRunnerOutputTypeInfo, getRunnerOutputTypeSerializer
endInput
checkInvokeFinishBundleByCount, close, createPythonEnvironmentManager, dispose, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setPythonConfig
getChainingStrategy, getContainingTask, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2
close, dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
public PythonKeyedCoProcessOperator(Configuration config, TypeInformation<Row> inputTypeInfo1, TypeInformation<Row> inputTypeInfo2, TypeInformation<OUT> outputTypeInfo, DataStreamPythonFunctionInfo pythonFunctionInfo)
public PythonFunctionRunner createPythonFunctionRunner() throws Exception
AbstractPythonFunctionOperator
PythonFunctionRunner
which is responsible for Python user-defined
function execution.createPythonFunctionRunner
in class TwoInputPythonFunctionOperator<Row,Row,Row,OUT>
Exception
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<OUT>
open
in class TwoInputPythonFunctionOperator<Row,Row,Row,OUT>
Exception
- An exception in this method causes the operator to fail.public void processElement1(StreamRecord<Row> element) throws Exception
TwoInputStreamOperator
processElement1
in interface TwoInputStreamOperator<Row,Row,OUT>
processElement1
in class TwoInputPythonFunctionOperator<Row,Row,Row,OUT>
Exception
public void processElement2(StreamRecord<Row> element) throws Exception
TwoInputStreamOperator
processElement2
in interface TwoInputStreamOperator<Row,Row,OUT>
processElement2
in class TwoInputPythonFunctionOperator<Row,Row,Row,OUT>
Exception
public void emitResult(Tuple2<byte[],Integer> resultTuple) throws Exception
AbstractPythonFunctionOperator
emitResult
in class AbstractPythonFunctionOperator<OUT>
Exception
public TypeInformation<OUT> getProducedType()
ResultTypeQueryable
TypeInformation
) produced by this function or input format.getProducedType
in interface ResultTypeQueryable<OUT>
public void onEventTime(InternalTimer<Row,VoidNamespace> timer) throws Exception
Triggerable
onEventTime
in interface Triggerable<Row,VoidNamespace>
Exception
public void onProcessingTime(InternalTimer<Row,VoidNamespace> timer) throws Exception
Triggerable
onProcessingTime
in interface Triggerable<Row,VoidNamespace>
Exception
public void setCurrentKey(Object key)
setCurrentKey
in interface KeyContext
setCurrentKey
in class AbstractPythonFunctionOperator<OUT>
public Object getCurrentKey()
getCurrentKey
in interface KeyContext
getCurrentKey
in class AbstractStreamOperator<OUT>
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.