K
- The type of key returned by the KeySelector
.W
- The type of Window
that the WindowAssigner
assigns.public abstract class WindowOperator<K,W extends Window> extends AbstractStreamOperator<BaseRow> implements OneInputStreamOperator<BaseRow,BaseRow>, Triggerable<K,W>
WindowAssigner
and
Trigger
.
This is the base class for AggregateWindowOperator
and
TableAggregateWindowOperator
. The big difference between AggregateWindowOperator
and TableAggregateWindowOperator
is AggregateWindowOperator
emits only one
result for each aggregate group, while TableAggregateWindowOperator
can emit multi
results for each aggregate group.
When an element arrives it gets assigned a key using a KeySelector
and it gets
assigned to zero or more windows using a WindowAssigner
. Based on this, the element
is put into panes. A pane is the bucket of elements that have the same key and same
Window
. An element can be in multiple panes if it was assigned to multiple windows by
the WindowAssigner
.
Each pane gets its own instance of the provided Trigger
. This trigger determines when
the contents of the pane should be processed to emit results. When a trigger fires,
the given NamespaceAggsHandleFunctionBase
is invoked to produce the results that are emitted for the pane to which the Trigger
belongs.
The parameter types:
<IN>
: BaseRow
<OUT>
: JoinedRow(KEY, AGG_RESULT)
<KEY>
: GenericRow
<AGG_RESULT>
: GenericRow
<ACC>
: GenericRow
AbstractStreamOperator.CountingOutput<OUT>
Modifier and Type | Field and Description |
---|---|
protected TimestampedCollector<BaseRow> |
collector
This is used for emitting elements with a given timestamp.
|
protected InternalValueState<K,W,BaseRow> |
previousState |
protected boolean |
sendRetraction |
protected NamespaceAggsHandleFunctionBase<W> |
windowAggregator |
protected InternalWindowProcessFunction<K,W> |
windowFunction |
chainingStrategy, config, latencyStats, LOG, metrics, output, timeServiceManager
Modifier and Type | Method and Description |
---|---|
void |
close()
This method is called after all records have been added to the operators via the methods
OneInputStreamOperator.processElement(StreamRecord) , or
TwoInputStreamOperator.processElement1(StreamRecord) and
TwoInputStreamOperator.processElement2(StreamRecord) . |
protected abstract void |
compileGeneratedCode() |
void |
dispose()
This method is called at the very end of the operator's life, both in the case of a successful
completion of the operation, and in the case of a failure and canceling.
|
protected abstract void |
emitWindowResult(W window)
Emits the window result of the given window.
|
protected Counter |
getNumLateRecordsDropped() |
protected Gauge<Long> |
getWatermarkLatency() |
void |
onEventTime(InternalTimer<K,W> timer)
Invoked when an event-time timer fires.
|
void |
onProcessingTime(InternalTimer<K,W> timer)
Invoked when a processing-time timer fires.
|
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElement(StreamRecord<BaseRow> record)
Processes one element that arrived at this operator.
|
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getUserCodeClassloader, initializeState, initializeState, notifyCheckpointComplete, numEventTimeTimers, numProcessingTimeTimers, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
processLatencyMarker, processWatermark
getChainingStrategy, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointComplete
getCurrentKey, setCurrentKey
protected final boolean sendRetraction
protected NamespaceAggsHandleFunctionBase<W extends Window> windowAggregator
protected transient InternalWindowProcessFunction<K,W extends Window> windowFunction
protected transient TimestampedCollector<BaseRow> collector
protected transient InternalValueState<K,W extends Window,BaseRow> previousState
protected abstract void compileGeneratedCode()
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<BaseRow>
open
in class AbstractStreamOperator<BaseRow>
Exception
- An exception in this method causes the operator to fail.public void close() throws Exception
AbstractStreamOperator
OneInputStreamOperator.processElement(StreamRecord)
, or
TwoInputStreamOperator.processElement1(StreamRecord)
and
TwoInputStreamOperator.processElement2(StreamRecord)
.
The method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered should be propagated, in order to cause the operation to be recognized asa failed, because the last data items are not processed properly.
close
in interface StreamOperator<BaseRow>
close
in class AbstractStreamOperator<BaseRow>
Exception
- An exception in this method causes the operator to fail.public void dispose() throws Exception
AbstractStreamOperator
This method is expected to make a thorough effort to release all resources that the operator has acquired.
dispose
in interface StreamOperator<BaseRow>
dispose
in interface Disposable
dispose
in class AbstractStreamOperator<BaseRow>
Exception
- if something goes wrong during disposal.public void processElement(StreamRecord<BaseRow> record) throws Exception
OneInputStreamOperator
processElement
in interface OneInputStreamOperator<BaseRow,BaseRow>
Exception
public void onEventTime(InternalTimer<K,W> timer) throws Exception
Triggerable
onEventTime
in interface Triggerable<K,W extends Window>
Exception
public void onProcessingTime(InternalTimer<K,W> timer) throws Exception
Triggerable
onProcessingTime
in interface Triggerable<K,W extends Window>
Exception
protected abstract void emitWindowResult(W window) throws Exception
Exception
protected Counter getNumLateRecordsDropped()
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.