public class PythonApplyFunction<W extends Window> extends AbstractPythonUDF<WindowFunction<org.python.core.PyObject,Object,Object,W>> implements WindowFunction<org.python.core.PyObject,org.python.core.PyObject,PyKey,W>
PythonApplyFunction
is a thin wrapper layer over a Python UDF WindowFunction
.
It receives an WindowFunction
as an input and keeps it internally in a serialized form.
It is then delivered, as part of job graph, up to the TaskManager, then it is opened and becomes
a sort of mediator to the Python UDF WindowFunction
.
This function is used internally by the Python thin wrapper layer over the streaming data functionality
fun, log
Constructor and Description |
---|
PythonApplyFunction(WindowFunction<org.python.core.PyObject,Object,Object,W> fun) |
Modifier and Type | Method and Description |
---|---|
void |
apply(PyKey key,
W window,
Iterable<org.python.core.PyObject> values,
Collector<org.python.core.PyObject> out)
Evaluates the window and outputs none or several elements.
|
void |
open(Configuration config)
Initialization method for the function.
|
close
getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
public PythonApplyFunction(WindowFunction<org.python.core.PyObject,Object,Object,W> fun) throws IOException
IOException
public void open(Configuration config) throws Exception
RichFunction
The configuration object passed to the function can be used for configuration and initialization. The configuration contains all parameters that were configured on the function in the program composition.
public class MyMapper extends FilterFunction<String> {
private String searchString;
public void open(Configuration parameters) {
this.searchString = parameters.getString("foo");
}
public boolean filter(String value) {
return value.equals(searchString);
}
}
By default, this method does nothing.
open
in interface RichFunction
open
in class AbstractPythonUDF<WindowFunction<org.python.core.PyObject,Object,Object,W extends Window>>
config
- The configuration containing the parameters attached to the contract.Exception
- Implementations may forward exceptions, which are caught by the runtime. When the
runtime catches an exception, it aborts the task and lets the fail-over logic
decide whether to retry the task execution.Configuration
public void apply(PyKey key, W window, Iterable<org.python.core.PyObject> values, Collector<org.python.core.PyObject> out) throws Exception
WindowFunction
apply
in interface WindowFunction<org.python.core.PyObject,org.python.core.PyObject,PyKey,W extends Window>
key
- The key for which this window is evaluated.window
- The window that is being evaluated.values
- The elements in the window being evaluated.out
- A collector for emitting elements.Exception
- The function may throw exceptions to fail the program and trigger recovery.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.