public class DataSetSessionWindowAggReduceGroupFunction extends RichGroupReduceFunction<Row,Row> implements Compiler<GeneratedAggregations>, Logging
GroupReduceOperator
. It is used for Session time-window
on batch.
Note:
This can handle two input types (depending if input is combined or not):
1. when partial aggregate is not supported, the input data structure of reduce is |groupKey1|groupKey2|sum1|count1|sum2|count2|rowTime| 2. when partial aggregate is supported, the input data structure of reduce is |groupKey1|groupKey2|sum1|count1|sum2|count2|windowStart|windowEnd|
param: genAggregations Code-generated GeneratedAggregations
param: keysAndAggregatesArity The total arity of keys and aggregates
param: finalRowWindowStartPos The relative window-start field position.
param: finalRowWindowEndPos The relative window-end field position.
param: gap Session time window gap.
Constructor and Description |
---|
DataSetSessionWindowAggReduceGroupFunction(GeneratedAggregationsFunction genAggregations,
int keysAndAggregatesArity,
scala.Option<Object> finalRowWindowStartPos,
scala.Option<Object> finalRowWindowEndPos,
long gap,
boolean isInputCombined) |
Modifier and Type | Method and Description |
---|---|
void |
doEvaluateAndCollect(Collector<Row> out,
long windowStart,
long windowEnd)
Evaluate and emit the data of the current window.
|
void |
open(Configuration config)
Initialization method for the function.
|
void |
reduce(Iterable<Row> records,
Collector<Row> out)
For grouped intermediate aggregate Rows, divide window according to the window-start
and window-end, merge data (within a unified window) into an aggregate buffer, calculate
aggregated values output from aggregate buffer, and then set them into output
Row based on the mapping relationship between intermediate aggregate data and output data.
|
close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
public DataSetSessionWindowAggReduceGroupFunction(GeneratedAggregationsFunction genAggregations, int keysAndAggregatesArity, scala.Option<Object> finalRowWindowStartPos, scala.Option<Object> finalRowWindowEndPos, long gap, boolean isInputCombined)
public void open(Configuration config)
RichFunction
The configuration object passed to the function can be used for configuration and initialization. The configuration contains all parameters that were configured on the function in the program composition.
public class MyMapper extends FilterFunction<String> {
private String searchString;
public void open(Configuration parameters) {
this.searchString = parameters.getString("foo");
}
public boolean filter(String value) {
return value.equals(searchString);
}
}
By default, this method does nothing.
open
in interface RichFunction
open
in class AbstractRichFunction
config
- The configuration containing the parameters attached to the contract.Configuration
public void reduce(Iterable<Row> records, Collector<Row> out)
reduce
in interface GroupReduceFunction<Row,Row>
reduce
in class RichGroupReduceFunction<Row,Row>
records
- Grouped intermediate aggregate Rows iterator.out
- The collector to hand results to.
public void doEvaluateAndCollect(Collector<Row> out, long windowStart, long windowEnd)
out
- the collection of the aggregate resultswindowStart
- the window's start attribute value is the min (rowtime) of all rows
in the window.windowEnd
- the window's end property value is max (rowtime) + gap for all rows
in the window.Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.