public class JobGraph extends Object implements Serializable
The JobGraph is a graph of vertices and intermediate results that are connected together to form a DAG. Note that iterations (feedback edges) are currently not encoded inside the JobGraph but inside certain special vertices that establish the feedback channel amongst themselves.
The JobGraph defines the job-wide configuration settings, while each vertex and intermediate result define the characteristics of the concrete operation and intermediate data.
Constructor and Description |
---|
JobGraph(JobID jobId,
String jobName)
Constructs a new job graph with the given job ID (or a random ID, if
null is passed),
the given name and the given execution configuration (see ExecutionConfig ). |
JobGraph(JobID jobId,
String jobName,
JobVertex... vertices)
Constructs a new job graph with the given name, the given
ExecutionConfig , the given
jobId or a random one if null supplied, and the given job vertices. |
JobGraph(JobVertex... vertices)
Constructs a new job graph with no name, a random job ID, the given
ExecutionConfig ,
and the given job vertices. |
JobGraph(String jobName)
Constructs a new job graph with the given name, the given
ExecutionConfig , and a
random job ID. |
JobGraph(String jobName,
JobVertex... vertices)
Constructs a new job graph with the given name, the given
ExecutionConfig , a random
job ID, and the given job vertices. |
Modifier and Type | Method and Description |
---|---|
void |
addJar(Path jar)
Adds the path of a JAR file required to run the job on a task manager.
|
void |
addJars(List<URL> jarFilesToAttach)
Adds the given jar files to the
JobGraph via addJar(org.apache.flink.core.fs.Path) . |
void |
addUserArtifact(String name,
DistributedCache.DistributedCacheEntry file)
Adds the path of a custom file required to run the job on a task manager.
|
void |
addUserJarBlobKey(PermanentBlobKey key)
Adds the BLOB referenced by the key to the JobGraph's dependencies.
|
void |
addVertex(JobVertex vertex)
Adds a new task vertex to the job graph if it is not already included.
|
JobVertex |
findVertexByID(JobVertexID id)
Searches for a vertex with a matching ID and returns it.
|
JobCheckpointingSettings |
getCheckpointingSettings()
Gets the settings for asynchronous snapshots.
|
List<URL> |
getClasspaths() |
Configuration |
getJobConfiguration()
Returns the configuration object for this job.
|
JobID |
getJobID()
Returns the ID of the job.
|
int |
getMaximumParallelism()
Gets the maximum parallelism of all operations in this job graph.
|
String |
getName()
Returns the name assigned to the job graph.
|
int |
getNumberOfVertices()
Returns the number of all vertices.
|
SavepointRestoreSettings |
getSavepointRestoreSettings()
Returns the configured savepoint restore setting.
|
ScheduleMode |
getScheduleMode() |
SerializedValue<ExecutionConfig> |
getSerializedExecutionConfig()
Returns the
ExecutionConfig |
Map<String,DistributedCache.DistributedCacheEntry> |
getUserArtifacts()
Gets the list of assigned user jar paths.
|
List<PermanentBlobKey> |
getUserJarBlobKeys()
Returns a set of BLOB keys referring to the JAR files required to run this job.
|
List<Path> |
getUserJars()
Gets the list of assigned user jar paths.
|
Iterable<JobVertex> |
getVertices()
Returns an Iterable to iterate all vertices registered with the job graph.
|
JobVertex[] |
getVerticesAsArray()
Returns an array of all job vertices that are registered with the job graph.
|
List<JobVertex> |
getVerticesSortedTopologicallyFromSources() |
boolean |
hasUsercodeJarFiles()
Checks whether the JobGraph has user code JAR files attached.
|
boolean |
isCheckpointingEnabled()
Checks if the checkpointing was enabled for this job graph
|
void |
setClasspaths(List<URL> paths)
Sets the classpaths required to run the job on a task manager.
|
void |
setExecutionConfig(ExecutionConfig executionConfig)
Sets the execution config.
|
void |
setJobID(JobID jobID)
Sets the ID of the job.
|
void |
setSavepointRestoreSettings(SavepointRestoreSettings settings)
Sets the savepoint restore settings.
|
void |
setScheduleMode(ScheduleMode scheduleMode) |
void |
setSnapshotSettings(JobCheckpointingSettings settings)
Sets the settings for asynchronous snapshots.
|
void |
setUserArtifactBlobKey(String entryName,
PermanentBlobKey blobKey) |
void |
setUserArtifactRemotePath(String entryName,
String remotePath) |
String |
toString() |
void |
writeUserArtifactEntriesToConfiguration() |
public JobGraph(String jobName)
ExecutionConfig
, and a
random job ID. The ExecutionConfig will be serialized and can't be modified afterwards.jobName
- The name of the job.public JobGraph(JobID jobId, String jobName)
null
is passed),
the given name and the given execution configuration (see ExecutionConfig
). The
ExecutionConfig will be serialized and can't be modified afterwards.jobId
- The id of the job. A random ID is generated, if null
is passed.jobName
- The name of the job.public JobGraph(JobVertex... vertices)
ExecutionConfig
,
and the given job vertices. The ExecutionConfig will be serialized and can't be modified
afterwards.vertices
- The vertices to add to the graph.public JobGraph(String jobName, JobVertex... vertices)
ExecutionConfig
, a random
job ID, and the given job vertices. The ExecutionConfig will be serialized and can't be
modified afterwards.jobName
- The name of the job.vertices
- The vertices to add to the graph.public JobGraph(JobID jobId, String jobName, JobVertex... vertices)
ExecutionConfig
, the given
jobId or a random one if null supplied, and the given job vertices. The ExecutionConfig will
be serialized and can't be modified afterwards.jobId
- The id of the job. A random ID is generated, if null
is passed.jobName
- The name of the job.vertices
- The vertices to add to the graph.public JobID getJobID()
public void setJobID(JobID jobID)
public String getName()
public Configuration getJobConfiguration()
public SerializedValue<ExecutionConfig> getSerializedExecutionConfig()
ExecutionConfig
public void setScheduleMode(ScheduleMode scheduleMode)
public ScheduleMode getScheduleMode()
public void setSavepointRestoreSettings(SavepointRestoreSettings settings)
settings
- The savepoint restore settings.public SavepointRestoreSettings getSavepointRestoreSettings()
public void setExecutionConfig(ExecutionConfig executionConfig) throws IOException
executionConfig
- The ExecutionConfig to be serialized.IOException
- Thrown if the serialization of the ExecutionConfig failspublic void addVertex(JobVertex vertex)
vertex
- the new task vertex to be addedpublic Iterable<JobVertex> getVertices()
public JobVertex[] getVerticesAsArray()
public int getNumberOfVertices()
public void setSnapshotSettings(JobCheckpointingSettings settings)
null
means that snapshotting
is not enabled.settings
- The snapshot settingspublic JobCheckpointingSettings getCheckpointingSettings()
public boolean isCheckpointingEnabled()
public JobVertex findVertexByID(JobVertexID id)
id
- the ID of the vertex to search fornull
if no vertex with such ID could
be foundpublic void setClasspaths(List<URL> paths)
paths
- paths of the directories/JAR files required to run the job on a task managerpublic int getMaximumParallelism()
public List<JobVertex> getVerticesSortedTopologicallyFromSources() throws InvalidProgramException
InvalidProgramException
public void addJar(Path jar)
jar
- path of the JAR file required to run the job on a task managerpublic void addJars(List<URL> jarFilesToAttach)
JobGraph
via addJar(org.apache.flink.core.fs.Path)
.jarFilesToAttach
- a list of the URLs
of the jar files to attach to the
jobgraph.RuntimeException
- if a jar URL is not valid.public List<Path> getUserJars()
public void addUserArtifact(String name, DistributedCache.DistributedCacheEntry file)
name
- a name under which this artifact will be accessible through DistributedCache
file
- path of a custom file required to run the job on a task managerpublic Map<String,DistributedCache.DistributedCacheEntry> getUserArtifacts()
public void addUserJarBlobKey(PermanentBlobKey key)
key
- path of the JAR file required to run the job on a task managerpublic boolean hasUsercodeJarFiles()
public List<PermanentBlobKey> getUserJarBlobKeys()
public void setUserArtifactBlobKey(String entryName, PermanentBlobKey blobKey) throws IOException
IOException
public void setUserArtifactRemotePath(String entryName, String remotePath)
public void writeUserArtifactEntriesToConfiguration()
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.