PyFlink exposes a metric system that allows gathering and exposing metrics to external systems.
You can access the metric system from a User-defined Function by calling
function_context.get_metric_group() in the
get_metric_group() method returns a
MetricGroup object on which you can create and register new metrics.
Counter is used to count something. The current value can be in- or decremented using
inc()/inc(n: int) or
You can create and register a
Counter by calling
counter(name: str) on a
Gauge provides a value on demand. You can register a gauge by calling
gauge(name: str, obj: Callable[, int]) on a MetricGroup. The Callable object will be used to report the values. Gauge metrics are restricted to integer-only values.
A metric that reports information(sum, count, min, max and mean) about the distribution of reported values. The value can be updated using
update(n: int). You can register a distribution by calling
distribution(name: str) on a MetricGroup. Distribution metrics are restricted to integer-only distributions.
A Meter measures an average throughput. An occurrence of an event can be registered with the
mark_event() method. The occurrence of multiple events at the same time can be registered with mark_event(n: int) method. You can register a meter by calling
meter(self, name: str, time_span_in_seconds: int = 60) on a MetricGroup. The default value of time_span_in_seconds is 60.
You can refer to the Java metric document for more details on Scope definition.
You can define a user scope by calling
MetricGroup.add_group(key: str, value: str = None). If extra is not None, creates a new key-value MetricGroup pair. The key group is added to this group’s sub-groups, while the value group is added to the key group’s sub-groups. In this case, the value group will be returned and a user variable will be defined.
You can refer to the Java metric document for more details on System Scope.
You can refer to the Java metric document for more details on List of all Variables.
You can define a user variable by calling
MetricGroup.addGroup(key: str, value: str = None) and specifying the value parameter.
Important: User variables cannot be used in scope formats.
You can refer to the Java metric document for more details on the following sections: