Modifier and Type | Method and Description |
---|---|
JobVertex |
ExecutionJobVertex.getJobVertex() |
Modifier and Type | Method and Description |
---|---|
void |
DefaultExecutionGraph.attachJobGraph(List<JobVertex> topologicallySorted) |
void |
ExecutionGraph.attachJobGraph(List<JobVertex> topologicallySorted) |
Constructor and Description |
---|
ExecutionJobVertex(InternalExecutionGraphAccessor graph,
JobVertex jobVertex,
VertexParallelismInformation parallelismInfo) |
SpeculativeExecutionJobVertex(InternalExecutionGraphAccessor graph,
JobVertex jobVertex,
VertexParallelismInformation parallelismInfo) |
Modifier and Type | Class and Description |
---|---|
class |
InputOutputFormatVertex
A task vertex that runs an initialization and a finalization on the master.
|
Modifier and Type | Method and Description |
---|---|
JobVertex |
JobGraph.findVertexByID(JobVertexID id)
Searches for a vertex with a matching ID and returns it.
|
JobVertex |
IntermediateDataSet.getProducer() |
JobVertex |
JobEdge.getTarget()
Returns the vertex connected to this edge.
|
JobVertex[] |
JobGraph.getVerticesAsArray()
Returns an array of all job vertices that are registered with the job graph.
|
Modifier and Type | Method and Description |
---|---|
Iterable<JobVertex> |
JobGraph.getVertices()
Returns an Iterable to iterate all vertices registered with the job graph.
|
List<JobVertex> |
JobGraph.getVerticesSortedTopologicallyFromSources() |
Modifier and Type | Method and Description |
---|---|
JobGraphBuilder |
JobGraphBuilder.addJobVertex(JobVertex jobVertex) |
void |
JobGraph.addVertex(JobVertex vertex)
Adds a new task vertex to the job graph if it is not already included.
|
JobEdge |
JobVertex.connectNewDataSetAsInput(JobVertex input,
DistributionPattern distPattern,
ResultPartitionType partitionType) |
JobEdge |
JobVertex.connectNewDataSetAsInput(JobVertex input,
DistributionPattern distPattern,
ResultPartitionType partitionType,
boolean isBroadcast) |
JobEdge |
JobVertex.connectNewDataSetAsInput(JobVertex input,
DistributionPattern distPattern,
ResultPartitionType partitionType,
IntermediateDataSetID intermediateDataSetId,
boolean isBroadcast) |
void |
JobVertex.setStrictlyCoLocatedWith(JobVertex strictlyCoLocatedWith)
Tells this vertex to strictly co locate its subtasks with the subtasks of the given vertex.
|
Modifier and Type | Method and Description |
---|---|
JobGraphBuilder |
JobGraphBuilder.addJobVertices(Collection<? extends JobVertex> jobVerticesToAdd) |
Constructor and Description |
---|
IntermediateDataSet(IntermediateDataSetID id,
ResultPartitionType resultType,
JobVertex producer) |
JobEdge(IntermediateDataSet source,
JobVertex target,
DistributionPattern distributionPattern,
boolean isBroadcast)
Constructs a new job edge, that connects an intermediate result to a consumer task.
|
JobGraph(JobID jobId,
String jobName,
JobVertex... vertices)
Constructs a new job graph with the given name, the given
ExecutionConfig , the given
jobId or a random one if null supplied, and the given job vertices. |
Modifier and Type | Method and Description |
---|---|
static String |
JsonPlanGenerator.generatePlan(JobID jobID,
String jobName,
JobType jobType,
Iterable<JobVertex> vertices,
VertexParallelism vertexParallelism) |
Modifier and Type | Method and Description |
---|---|
static DefaultLogicalTopology |
DefaultLogicalTopology.fromTopologicallySortedJobVertices(List<JobVertex> jobVertices) |
Modifier and Type | Method and Description |
---|---|
void |
CoLocationGroupImpl.addVertex(JobVertex vertex) |
Constructor and Description |
---|
CoLocationGroupImpl(JobVertex... vertices) |
Modifier and Type | Method and Description |
---|---|
static int |
SchedulerBase.getDefaultMaxParallelism(JobVertex vertex)
Get a default value to use for a given vertex's max parallelism if none was specified.
|
Modifier and Type | Method and Description |
---|---|
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices)
Compute the
VertexParallelismStore for all given vertices, which will set defaults
and ensure that the returned store contains valid parallelisms. |
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices,
java.util.function.Function<JobVertex,Integer> defaultMaxParallelismFunc) |
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices,
java.util.function.Function<JobVertex,Integer> defaultMaxParallelismFunc) |
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices,
java.util.function.Function<JobVertex,Integer> defaultMaxParallelismFunc,
java.util.function.Function<Integer,Integer> normalizeParallelismFunc)
Compute the
VertexParallelismStore for all given vertices, which will set defaults
and ensure that the returned store contains valid parallelisms, with a custom function for
default max parallelism calculation and a custom function for normalizing vertex parallelism. |
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices,
java.util.function.Function<JobVertex,Integer> defaultMaxParallelismFunc,
java.util.function.Function<Integer,Integer> normalizeParallelismFunc)
Compute the
VertexParallelismStore for all given vertices, which will set defaults
and ensure that the returned store contains valid parallelisms, with a custom function for
default max parallelism calculation and a custom function for normalizing vertex parallelism. |
Modifier and Type | Method and Description |
---|---|
static VertexParallelismStore |
AdaptiveBatchScheduler.computeVertexParallelismStoreForDynamicGraph(Iterable<JobVertex> vertices,
int defaultMaxParallelism)
Compute the
VertexParallelismStore for all given vertices in a dynamic graph, which
will set defaults and ensure that the returned store contains valid parallelisms, with the
configured default max parallelism. |
Modifier and Type | Method and Description |
---|---|
static Map<JobVertexID,ForwardGroup> |
ForwardGroupComputeUtil.computeForwardGroups(Iterable<JobVertex> topologicallySortedVertices,
java.util.function.Function<JobVertexID,ExecutionJobVertex> executionJobVertexRetriever) |
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.