Modules
This documentation is for an unreleased version of Apache Flink. We recommend you use the latest stable version.

Modules #

Modules allow users to extend Flink’s built-in objects, such as defining functions that behave like Flink built-in functions. They are pluggable, and while Flink provides a few pre-built modules, users can write their own.

For example, users can define their own geo functions and plug them into Flink as built-in functions to be used in Flink SQL and Table APIs. Another example is users can load an out-of-shelf Hive module to use Hive built-in functions as Flink built-in functions.

Module Types #

CoreModule #

CoreModule contains all of Flink’s system (built-in) functions and is loaded by default.

HiveModule #

The HiveModule provides Hive built-in functions as Flink’s system functions to SQL and Table API users. Flink’s Hive documentation provides full details on setting up the module.

User-Defined Module #

Users can develop custom modules by implementing the Module interface. To use custom modules in SQL CLI, users should develop both a module and its corresponding module factory by implementing the ModuleFactory interface.

A module factory defines a set of properties for configuring the module when the SQL CLI bootstraps. Properties are passed to a discovery service where the service tries to match the properties to a ModuleFactory and instantiate a corresponding module instance.

Namespace and Resolution Order #

Objects provided by modules are considered part of Flink’s system (built-in) objects; thus, they don’t have any namespaces.

When there are two objects of the same name residing in two modules, Flink always resolves the object reference to the one in the 1st loaded module.

Module API #

Loading and unloading a Module #

Users can load and unload modules in an existing Flink session.

tableEnv.loadModule("myModule", new CustomModule());
tableEnv.unloadModule("myModule");

All modules defined using YAML must provide a type property that specifies the type. The following types are supported out of the box.

Catalog Type Value
CoreModule core
HiveModule hive
modules:
   - name: core
     type: core
   - name: myhive
     type: hive

List Available Modules #

tableEnv.listModules();
Flink SQL> SHOW MODULES;