public class ExecutionEnvironment extends Object
The environment provides methods to control the job execution (such as setting the parallelism) and to interact with the outside world (data access).
To get an execution environment use the methods on the companion object:
- getExecutionEnvironment()
- createLocalEnvironment(int)
- createRemoteEnvironment(java.lang.String, int, scala.collection.Seq<java.lang.String>)
Use getExecutionEnvironment()
to get the correct environment depending
on where the program is executed. If it is run inside an IDE a local environment will be
created. If the program is submitted to a cluster a remote execution environment will
be created.
Constructor and Description |
---|
ExecutionEnvironment(ExecutionEnvironment javaEnv) |
Modifier and Type | Method and Description |
---|---|
void |
addDefaultKryoSerializer(Class<?> clazz,
Class<? extends com.esotericsoftware.kryo.Serializer<?>> serializer)
Registers a default serializer for the given class and its sub-classes at Kryo.
|
<T extends com.esotericsoftware.kryo.Serializer<?>> |
addDefaultKryoSerializer(Class<?> clazz,
T serializer)
Registers a default serializer for the given class and its sub-classes at Kryo.
|
static ExecutionEnvironment |
createCollectionsEnvironment()
Creates an execution environment that uses Java Collections underneath.
|
<K,V> DataSet<scala.Tuple2<K,V>> |
createHadoopInput(org.apache.hadoop.mapred.InputFormat<K,V> mapredInputFormat,
Class<K> key,
Class<V> value,
org.apache.hadoop.mapred.JobConf job,
TypeInformation<scala.Tuple2<K,V>> tpe)
Deprecated.
|
<K,V> DataSet<scala.Tuple2<K,V>> |
createHadoopInput(org.apache.hadoop.mapreduce.InputFormat<K,V> mapreduceInputFormat,
Class<K> key,
Class<V> value,
org.apache.hadoop.mapreduce.Job job,
TypeInformation<scala.Tuple2<K,V>> tpe)
Deprecated.
|
<T> DataSet<T> |
createInput(InputFormat<T,?> inputFormat,
scala.reflect.ClassTag<T> evidence$7,
TypeInformation<T> evidence$8)
Generic method to create an input DataSet with an
InputFormat . |
static ExecutionEnvironment |
createLocalEnvironment(Configuration customConfiguration)
Creates a local execution environment.
|
static ExecutionEnvironment |
createLocalEnvironment(int parallelism)
Creates a local execution environment.
|
static ExecutionEnvironment |
createLocalEnvironmentWithWebUI(Configuration config)
Creates a
ExecutionEnvironment for local program execution that also starts the
web monitoring UI. |
Plan |
createProgramPlan(String jobName)
Creates the program's
Plan . |
static ExecutionEnvironment |
createRemoteEnvironment(String host,
int port,
Configuration clientConfiguration,
scala.collection.Seq<String> jarFiles)
Creates a remote execution environment.
|
static ExecutionEnvironment |
createRemoteEnvironment(String host,
int port,
int parallelism,
scala.collection.Seq<String> jarFiles)
Creates a remote execution environment.
|
static ExecutionEnvironment |
createRemoteEnvironment(String host,
int port,
scala.collection.Seq<String> jarFiles)
Creates a remote execution environment.
|
JobExecutionResult |
execute()
Triggers the program execution.
|
JobExecutionResult |
execute(String jobName)
Triggers the program execution.
|
<T> DataSet<T> |
fromCollection(scala.collection.Iterable<T> data,
scala.reflect.ClassTag<T> evidence$10,
TypeInformation<T> evidence$11)
Creates a DataSet from the given non-empty
Iterable . |
<T> DataSet<T> |
fromCollection(scala.collection.Iterator<T> data,
scala.reflect.ClassTag<T> evidence$12,
TypeInformation<T> evidence$13)
Creates a DataSet from the given
Iterator . |
<T> DataSet<T> |
fromElements(scala.collection.Seq<T> data,
scala.reflect.ClassTag<T> evidence$14,
TypeInformation<T> evidence$15)
Creates a new data set that contains the given elements.
|
<T> DataSet<T> |
fromParallelCollection(SplittableIterator<T> iterator,
scala.reflect.ClassTag<T> evidence$16,
TypeInformation<T> evidence$17)
Creates a new data set that contains elements in the iterator.
|
DataSet<Object> |
generateSequence(long from,
long to)
Creates a new data set that contains a sequence of numbers.
|
ExecutionConfig |
getConfig()
Gets the config object.
|
static int |
getDefaultLocalParallelism()
Gets the default parallelism that will be used for the local execution environment created by
createLocalEnvironment() . |
static ExecutionEnvironment |
getExecutionEnvironment()
Creates an execution environment that represents the context in which the program is
currently executed.
|
String |
getExecutionPlan()
Creates the plan with which the system will execute the program, and returns it as a String
using a JSON representation of the execution data flow graph.
|
JobID |
getId()
Gets the UUID by which this environment is identified.
|
String |
getIdString()
Gets the UUID by which this environment is identified, as a string.
|
ExecutionEnvironment |
getJavaEnv() |
JobExecutionResult |
getLastJobExecutionResult()
Gets the JobExecutionResult of the last executed job.
|
int |
getNumberOfExecutionRetries()
Deprecated.
This method will be replaced by
getRestartStrategy . The
FixedDelayRestartStrategyConfiguration contains the number of execution retries. |
int |
getParallelism()
Returns the default parallelism for this execution environment.
|
RestartStrategies.RestartStrategyConfiguration |
getRestartStrategy()
Returns the specified restart strategy configuration.
|
long |
getSessionTimeout()
Gets the session timeout for this environment.
|
<T> DataSet<T> |
readCsvFile(String filePath,
String lineDelimiter,
String fieldDelimiter,
Character quoteCharacter,
boolean ignoreFirstLine,
String ignoreComments,
boolean lenient,
int[] includedFields,
String[] pojoFields,
scala.reflect.ClassTag<T> evidence$1,
TypeInformation<T> evidence$2)
Creates a DataSet by reading the given CSV file.
|
<T> DataSet<T> |
readFile(FileInputFormat<T> inputFormat,
String filePath,
scala.reflect.ClassTag<T> evidence$5,
TypeInformation<T> evidence$6)
Creates a new DataSource by reading the specified file using the custom
FileInputFormat . |
<T> DataSet<T> |
readFileOfPrimitives(String filePath,
String delimiter,
scala.reflect.ClassTag<T> evidence$3,
TypeInformation<T> evidence$4)
Creates a DataSet that represents the primitive type produced by reading the
given file in delimited way.This method is similar to
readCsvFile with
single field, but it produces a DataSet not through Tuple. |
<K,V> DataSet<scala.Tuple2<K,V>> |
readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V> mapredInputFormat,
Class<K> key,
Class<V> value,
String inputPath,
org.apache.hadoop.mapred.JobConf job,
TypeInformation<scala.Tuple2<K,V>> tpe)
Deprecated.
|
<K,V> DataSet<scala.Tuple2<K,V>> |
readHadoopFile(org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> mapreduceInputFormat,
Class<K> key,
Class<V> value,
String inputPath,
org.apache.hadoop.mapreduce.Job job,
TypeInformation<scala.Tuple2<K,V>> tpe)
Deprecated.
|
<K,V> DataSet<scala.Tuple2<K,V>> |
readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V> mapredInputFormat,
Class<K> key,
Class<V> value,
String inputPath,
TypeInformation<scala.Tuple2<K,V>> tpe)
Deprecated.
|
<K,V> DataSet<scala.Tuple2<K,V>> |
readHadoopFile(org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> mapreduceInputFormat,
Class<K> key,
Class<V> value,
String inputPath,
TypeInformation<scala.Tuple2<K,V>> tpe)
Deprecated.
|
<K,V> DataSet<scala.Tuple2<K,V>> |
readSequenceFile(Class<K> key,
Class<V> value,
String inputPath,
TypeInformation<scala.Tuple2<K,V>> tpe)
Deprecated.
Please use
HadoopInputs.readSequenceFile(java.lang.Class<K>, java.lang.Class<V>, java.lang.String, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module. |
DataSet<String> |
readTextFile(String filePath,
String charsetName)
Creates a DataSet of Strings produced by reading the given file line wise.
|
DataSet<StringValue> |
readTextFileWithValue(String filePath,
String charsetName)
Creates a DataSet of Strings produced by reading the given file line wise.
|
void |
registerCachedFile(String filePath,
String name,
boolean executable)
Registers a file at the distributed cache under the given name.
|
void |
registerType(Class<?> typeClass)
Registers the given type with the serialization stack.
|
void |
registerTypeWithKryoSerializer(Class<?> clazz,
Class<? extends com.esotericsoftware.kryo.Serializer<?>> serializer)
Registers the given type with the serializer at the
KryoSerializer . |
<T extends com.esotericsoftware.kryo.Serializer<?>> |
registerTypeWithKryoSerializer(Class<?> clazz,
T serializer)
Registers the given type with the serializer at the
KryoSerializer . |
static void |
setDefaultLocalParallelism(int parallelism)
Sets the default parallelism that will be used for the local execution
environment created by
createLocalEnvironment() . |
void |
setNumberOfExecutionRetries(int numRetries)
Deprecated.
This method will be replaced by
setRestartStrategy() . The
FixedDelayRestartStrategyConfiguration contains the number of execution retries. |
void |
setParallelism(int parallelism)
Sets the parallelism (parallelism) for operations executed through this environment.
|
void |
setRestartStrategy(RestartStrategies.RestartStrategyConfiguration restartStrategyConfiguration)
Sets the restart strategy configuration.
|
void |
setSessionTimeout(long timeout)
Sets the session timeout to hold the intermediate results of a job.
|
void |
startNewSession()
Starts a new session, discarding all intermediate results.
|
<T> DataSet<T> |
union(scala.collection.Seq<DataSet<T>> sets) |
public ExecutionEnvironment(ExecutionEnvironment javaEnv)
public static void setDefaultLocalParallelism(int parallelism)
createLocalEnvironment()
.
parallelism
- The default parallelism to use for local execution.public static int getDefaultLocalParallelism()
createLocalEnvironment()
.public static ExecutionEnvironment getExecutionEnvironment()
public static ExecutionEnvironment createLocalEnvironment(int parallelism)
This method sets the environment's default parallelism to given parameter, which
defaults to the value set via setDefaultLocalParallelism(Int)
.
parallelism
- (undocumented)public static ExecutionEnvironment createLocalEnvironment(Configuration customConfiguration)
customConfiguration
- (undocumented)public static ExecutionEnvironment createLocalEnvironmentWithWebUI(Configuration config)
ExecutionEnvironment
for local program execution that also starts the
web monitoring UI.
The local execution environment will run the program in a multi-threaded fashion in the same JVM as the environment was created in. It will use the parallelism specified in the parameter.
If the configuration key 'jobmanager.web.port' was set in the configuration, that particular port will be used for the web UI. Otherwise, the default port (8081) will be used.
config
- optional config for the local executionpublic static ExecutionEnvironment createCollectionsEnvironment()
public static ExecutionEnvironment createRemoteEnvironment(String host, int port, scala.collection.Seq<String> jarFiles)
ExecutionEnvironment.setParallelism()
.
host
- The host name or address of the master (JobManager),
where the program should be executed.port
- The port of the master (JobManager), where the program should be executed.jarFiles
- The JAR files with code that needs to be shipped to the cluster. If the
program uses
user-defined functions, user-defined input formats, or any libraries,
those must be
provided in the JAR files.public static ExecutionEnvironment createRemoteEnvironment(String host, int port, int parallelism, scala.collection.Seq<String> jarFiles)
host
- The host name or address of the master (JobManager),
where the program should be executed.port
- The port of the master (JobManager), where the program should be executed.parallelism
- The parallelism to use during the execution.jarFiles
- The JAR files with code that needs to be shipped to the cluster. If the
program uses
user-defined functions, user-defined input formats, or any libraries,
those must be
provided in the JAR files.public static ExecutionEnvironment createRemoteEnvironment(String host, int port, Configuration clientConfiguration, scala.collection.Seq<String> jarFiles)
ExecutionEnvironment.setParallelism
.
ClusterClient configuration has to be done in the remotely running Flink instance.
host
- The host name or address of the master (JobManager), where the program should be
executed.port
- The port of the master (JobManager), where the program should be executed.clientConfiguration
- Pass a custom configuration to the Client.jarFiles
- The JAR files with code that needs to be shipped to the cluster. If the
program uses user-defined functions, user-defined input formats, or any
libraries, those must be provided in the JAR files.public ExecutionEnvironment getJavaEnv()
public ExecutionConfig getConfig()
public void setParallelism(int parallelism)
DataSet.setParallelism
.parallelism
- (undocumented)public int getParallelism()
DataSet.setParallelism
public void setRestartStrategy(RestartStrategies.RestartStrategyConfiguration restartStrategyConfiguration)
restartStrategyConfiguration
- Restart strategy configuration to be setpublic RestartStrategies.RestartStrategyConfiguration getRestartStrategy()
public void setNumberOfExecutionRetries(int numRetries)
setRestartStrategy()
. The
FixedDelayRestartStrategyConfiguration contains the number of execution retries.numRetries
- (undocumented)public int getNumberOfExecutionRetries()
getRestartStrategy
. The
FixedDelayRestartStrategyConfiguration contains the number of execution retries.public JobID getId()
public JobExecutionResult getLastJobExecutionResult()
public String getIdString()
public void startNewSession()
public void setSessionTimeout(long timeout)
timeout
- The timeout in seconds.public long getSessionTimeout()
public <T extends com.esotericsoftware.kryo.Serializer<?>> void registerTypeWithKryoSerializer(Class<?> clazz, T serializer)
KryoSerializer
.
Note that the serializer instance must be serializable (as defined by java.io.Serializable), because it may be distributed to the worker nodes by java serialization.
clazz
- (undocumented)serializer
- (undocumented)public void registerTypeWithKryoSerializer(Class<?> clazz, Class<? extends com.esotericsoftware.kryo.Serializer<?>> serializer)
KryoSerializer
.clazz
- (undocumented)serializer
- (undocumented)public void addDefaultKryoSerializer(Class<?> clazz, Class<? extends com.esotericsoftware.kryo.Serializer<?>> serializer)
clazz
- (undocumented)serializer
- (undocumented)public <T extends com.esotericsoftware.kryo.Serializer<?>> void addDefaultKryoSerializer(Class<?> clazz, T serializer)
Note that the serializer instance must be serializable (as defined by java.io.Serializable), because it may be distributed to the worker nodes by java serialization.
clazz
- (undocumented)serializer
- (undocumented)public void registerType(Class<?> typeClass)
typeClass
- (undocumented)public DataSet<String> readTextFile(String filePath, String charsetName)
filePath
- The path of the file, as a URI (e.g., "file:///some/local/file" or
"hdfs://host:port/file/path").charsetName
- The name of the character set used to read the file. Default is UTF-0public DataSet<StringValue> readTextFileWithValue(String filePath, String charsetName)
readTextFile
, but it produces a DataSet with mutable
StringValue
objects, rather than Java Strings. StringValues can be used to tune
implementations to be less object and garbage collection heavy.
filePath
- The path of the file, as a URI (e.g., "file:///some/local/file" or
"hdfs://host:port/file/path").charsetName
- The name of the character set used to read the file. Default is UTF-0public <T> DataSet<T> readCsvFile(String filePath, String lineDelimiter, String fieldDelimiter, Character quoteCharacter, boolean ignoreFirstLine, String ignoreComments, boolean lenient, int[] includedFields, String[] pojoFields, scala.reflect.ClassTag<T> evidence$1, TypeInformation<T> evidence$2)
includedFields
parameter can be used
to only read specific fields.
filePath
- The path of the file, as a URI (e.g., "file:///some/local/file" or
"hdfs://host:port/file/path").lineDelimiter
- The string that separates lines, defaults to newline.fieldDelimiter
- The string that separates individual fields, defaults to ",".quoteCharacter
- The character to use for quoted String parsing, disabled by default.ignoreFirstLine
- Whether the first line in the file should be ignored.ignoreComments
- Lines that start with the given String are ignored, disabled by default.lenient
- Whether the parser should silently ignore malformed lines.includedFields
- The fields in the file that should be read. Per default all fields
are read.pojoFields
- The fields of the POJO which are mapped to CSV fields.evidence$1
- (undocumented)evidence$2
- (undocumented)public <T> DataSet<T> readFileOfPrimitives(String filePath, String delimiter, scala.reflect.ClassTag<T> evidence$3, TypeInformation<T> evidence$4)
readCsvFile
with
single field, but it produces a DataSet not through Tuple.
The type parameter must be used to specify the primitive type.
filePath
- The path of the file, as a URI (e.g., "file:///some/local/file" or
"hdfs://host:port/file/path").delimiter
- The string that separates primitives , defaults to newline.evidence$3
- (undocumented)evidence$4
- (undocumented)public <T> DataSet<T> readFile(FileInputFormat<T> inputFormat, String filePath, scala.reflect.ClassTag<T> evidence$5, TypeInformation<T> evidence$6)
FileInputFormat
.inputFormat
- (undocumented)filePath
- (undocumented)evidence$5
- (undocumented)evidence$6
- (undocumented)public <T> DataSet<T> createInput(InputFormat<T,?> inputFormat, scala.reflect.ClassTag<T> evidence$7, TypeInformation<T> evidence$8)
InputFormat
.inputFormat
- (undocumented)evidence$7
- (undocumented)evidence$8
- (undocumented)public <K,V> DataSet<scala.Tuple2<K,V>> readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V> mapredInputFormat, Class<K> key, Class<V> value, String inputPath, org.apache.hadoop.mapred.JobConf job, TypeInformation<scala.Tuple2<K,V>> tpe)
HadoopInputs.readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K, V>, java.lang.Class<K>, java.lang.Class<V>, java.lang.String, org.apache.hadoop.mapred.JobConf, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module.DataSet
from the given FileInputFormat
. The
given inputName is set on the given job.
mapredInputFormat
- (undocumented)key
- (undocumented)value
- (undocumented)inputPath
- (undocumented)job
- (undocumented)tpe
- (undocumented)public <K,V> DataSet<scala.Tuple2<K,V>> readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V> mapredInputFormat, Class<K> key, Class<V> value, String inputPath, TypeInformation<scala.Tuple2<K,V>> tpe)
HadoopInputs.readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K, V>, java.lang.Class<K>, java.lang.Class<V>, java.lang.String, org.apache.hadoop.mapred.JobConf, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module.mapredInputFormat
- (undocumented)key
- (undocumented)value
- (undocumented)inputPath
- (undocumented)tpe
- (undocumented)public <K,V> DataSet<scala.Tuple2<K,V>> readSequenceFile(Class<K> key, Class<V> value, String inputPath, TypeInformation<scala.Tuple2<K,V>> tpe)
HadoopInputs.readSequenceFile(java.lang.Class<K>, java.lang.Class<V>, java.lang.String, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module.key
- (undocumented)value
- (undocumented)inputPath
- (undocumented)tpe
- (undocumented)public <K,V> DataSet<scala.Tuple2<K,V>> createHadoopInput(org.apache.hadoop.mapred.InputFormat<K,V> mapredInputFormat, Class<K> key, Class<V> value, org.apache.hadoop.mapred.JobConf job, TypeInformation<scala.Tuple2<K,V>> tpe)
HadoopInputs.createHadoopInput(org.apache.hadoop.mapred.InputFormat<K, V>, java.lang.Class<K>, java.lang.Class<V>, org.apache.hadoop.mapred.JobConf, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module.DataSet
from the given InputFormat
.
mapredInputFormat
- (undocumented)key
- (undocumented)value
- (undocumented)job
- (undocumented)tpe
- (undocumented)public <K,V> DataSet<scala.Tuple2<K,V>> readHadoopFile(org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> mapreduceInputFormat, Class<K> key, Class<V> value, String inputPath, org.apache.hadoop.mapreduce.Job job, TypeInformation<scala.Tuple2<K,V>> tpe)
HadoopInputs.readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K, V>, java.lang.Class<K>, java.lang.Class<V>, java.lang.String, org.apache.hadoop.mapred.JobConf, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module.DataSet
from the given FileInputFormat
.
The given inputName is set on the given job.
mapreduceInputFormat
- (undocumented)key
- (undocumented)value
- (undocumented)inputPath
- (undocumented)job
- (undocumented)tpe
- (undocumented)public <K,V> DataSet<scala.Tuple2<K,V>> readHadoopFile(org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> mapreduceInputFormat, Class<K> key, Class<V> value, String inputPath, TypeInformation<scala.Tuple2<K,V>> tpe)
HadoopInputs.readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K, V>, java.lang.Class<K>, java.lang.Class<V>, java.lang.String, org.apache.hadoop.mapred.JobConf, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module.mapreduceInputFormat
- (undocumented)key
- (undocumented)value
- (undocumented)inputPath
- (undocumented)tpe
- (undocumented)public <K,V> DataSet<scala.Tuple2<K,V>> createHadoopInput(org.apache.hadoop.mapreduce.InputFormat<K,V> mapreduceInputFormat, Class<K> key, Class<V> value, org.apache.hadoop.mapreduce.Job job, TypeInformation<scala.Tuple2<K,V>> tpe)
HadoopInputs.createHadoopInput(org.apache.hadoop.mapred.InputFormat<K, V>, java.lang.Class<K>, java.lang.Class<V>, org.apache.hadoop.mapred.JobConf, org.apache.flink.api.common.typeinfo.TypeInformation<scala.Tuple2<K, V>>)
from the flink-hadoop-compatibility module.DataSet
from the given InputFormat
.
mapreduceInputFormat
- (undocumented)key
- (undocumented)value
- (undocumented)job
- (undocumented)tpe
- (undocumented)public <T> DataSet<T> fromCollection(scala.collection.Iterable<T> data, scala.reflect.ClassTag<T> evidence$10, TypeInformation<T> evidence$11)
Iterable
.
Note that this operation will result in a non-parallel data source, i.e. a data source with a parallelism of one.
data
- (undocumented)evidence$10
- (undocumented)evidence$11
- (undocumented)public <T> DataSet<T> fromCollection(scala.collection.Iterator<T> data, scala.reflect.ClassTag<T> evidence$12, TypeInformation<T> evidence$13)
Iterator
.
Note that this operation will result in a non-parallel data source, i.e. a data source with a parallelism of one.
data
- (undocumented)evidence$12
- (undocumented)evidence$13
- (undocumented)public <T> DataSet<T> fromElements(scala.collection.Seq<T> data, scala.reflect.ClassTag<T> evidence$14, TypeInformation<T> evidence$15)
* Note that this operation will result in a non-parallel data source, i.e. a data source with a parallelism of one.
data
- (undocumented)evidence$14
- (undocumented)evidence$15
- (undocumented)public <T> DataSet<T> fromParallelCollection(SplittableIterator<T> iterator, scala.reflect.ClassTag<T> evidence$16, TypeInformation<T> evidence$17)
iterator
- (undocumented)evidence$16
- (undocumented)evidence$17
- (undocumented)public DataSet<Object> generateSequence(long from, long to)
from
- The number to start at (inclusive).to
- The number to stop at (inclusive).public void registerCachedFile(String filePath, String name, boolean executable)
The RuntimeContext
can be obtained inside UDFs
via
RichFunction.getRuntimeContext()
and provides
access via
RuntimeContext.getDistributedCache()
filePath
- The path of the file, as a URI (e.g. "file:///some/path" or
"hdfs://host:port/and/path")name
- The name under which the file is registered.executable
- Flag indicating whether the file should be executablepublic JobExecutionResult execute()
DataSet.print
, writing results (e.g. DataSet.writeAsText
, DataSet.write
, or other
generic data sinks created with DataSet.output
.
The program execution will be logged and displayed with a generated default name.
public JobExecutionResult execute(String jobName)
DataSet.print
, writing results (e.g. DataSet.writeAsText
, DataSet.write
, or other
generic data sinks created with DataSet.output
.
The program execution will be logged and displayed with the given name.
jobName
- (undocumented)public String getExecutionPlan()
public Plan createProgramPlan(String jobName)
Plan
.
The plan is a description of all data sources, data sinks,
and operations and how they interact, as an isolated unit that can be executed with a
PlanExecutor
. Obtaining a plan and starting it with an
executor is an alternative way to run a program and is only possible if the program only
consists of distributed operations.jobName
- (undocumented)Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.