Modifier and Type | Method and Description |
---|---|
static JobGraph |
FlinkPipelineTranslationUtil.getJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism)
|
static JobGraph |
FlinkPipelineTranslationUtil.getJobGraphUnderUserClassLoader(ClassLoader userClassloader,
Pipeline pipeline,
Configuration configuration,
int defaultParallelism)
|
JobGraph |
StreamGraphTranslator.translateToJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism) |
JobGraph |
FlinkPipelineTranslator.translateToJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism)
|
JobGraph |
PlanTranslator.translateToJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism) |
Modifier and Type | Method and Description |
---|---|
static JobExecutionResult |
ClientUtils.submitJob(ClusterClient<?> client,
JobGraph jobGraph) |
static JobExecutionResult |
ClientUtils.submitJobAndWaitForResult(ClusterClient<?> client,
JobGraph jobGraph,
ClassLoader classLoader) |
Modifier and Type | Method and Description |
---|---|
ClusterClientProvider<StandaloneClusterId> |
StandaloneClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
ClusterClientProvider<T> |
ClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached)
Deploys a per-job cluster with the given job on the cluster.
|
Modifier and Type | Method and Description |
---|---|
AbstractDispatcherLeaderProcess.DispatcherGatewayService |
ApplicationDispatcherGatewayServiceFactory.create(DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
JobGraphWriter jobGraphWriter) |
Modifier and Type | Method and Description |
---|---|
static JobGraph |
PipelineExecutorUtils.getJobGraph(Pipeline pipeline,
Configuration configuration)
|
Modifier and Type | Method and Description |
---|---|
static JobGraph |
PackagedProgramUtils.createJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
int defaultParallelism,
boolean suppressOutput)
|
static JobGraph |
PackagedProgramUtils.createJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
int defaultParallelism,
JobID jobID,
boolean suppressOutput)
|
Modifier and Type | Method and Description |
---|---|
CompletableFuture<JobClient> |
PerJobMiniClusterFactory.submitJob(JobGraph jobGraph)
Starts a
MiniCluster and submits a job. |
CompletableFuture<JobID> |
MiniClusterClient.submitJob(JobGraph jobGraph) |
CompletableFuture<JobID> |
ClusterClient.submitJob(JobGraph jobGraph)
Submit the given
JobGraph to the cluster. |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<JobID> |
RestClusterClient.submitJob(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
ClusterClientProvider<String> |
KubernetesClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JobGraphGenerator.compileJobGraph(OptimizedPlan program)
Translates a
OptimizedPlan into a JobGraph . |
JobGraph |
JobGraphGenerator.compileJobGraph(OptimizedPlan program,
JobID jobId) |
Modifier and Type | Method and Description |
---|---|
static void |
ClientUtils.extractAndUploadJobGraphFiles(JobGraph jobGraph,
SupplierWithException<BlobClient,IOException> clientSupplier)
Extracts all files required for the execution from the given
JobGraph and uploads
them using the BlobClient from the given Supplier . |
static void |
ClientUtils.uploadJobGraphFiles(JobGraph jobGraph,
Collection<Path> userJars,
Collection<Tuple2<String,Path>> userArtifacts,
SupplierWithException<BlobClient,IOException> clientSupplier)
Uploads the given jars and artifacts required for the execution of the given
JobGraph
using the BlobClient from the given Supplier . |
Modifier and Type | Method and Description |
---|---|
JobGraph |
SingleJobJobGraphStore.recoverJobGraph(JobID jobId) |
Modifier and Type | Method and Description |
---|---|
JobManagerRunner |
DefaultJobManagerRunnerFactory.createJobManagerRunner(JobGraph jobGraph,
Configuration configuration,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
JobManagerSharedServices jobManagerServices,
JobManagerJobMetricGroupFactory jobManagerJobMetricGroupFactory,
FatalErrorHandler fatalErrorHandler) |
JobManagerRunner |
JobManagerRunnerFactory.createJobManagerRunner(JobGraph jobGraph,
Configuration configuration,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
JobManagerSharedServices jobManagerServices,
JobManagerJobMetricGroupFactory jobManagerJobMetricGroupFactory,
FatalErrorHandler fatalErrorHandler) |
void |
SingleJobJobGraphStore.putJobGraph(JobGraph jobGraph) |
CompletableFuture<Acknowledge> |
DispatcherGateway.submitJob(JobGraph jobGraph,
Time timeout)
Submit a job to the dispatcher.
|
CompletableFuture<Acknowledge> |
MiniDispatcher.submitJob(JobGraph jobGraph,
Time timeout) |
CompletableFuture<Acknowledge> |
Dispatcher.submitJob(JobGraph jobGraph,
Time timeout) |
Modifier and Type | Method and Description |
---|---|
Dispatcher |
DispatcherFactory.createDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
PartialDispatcherServicesWithJobGraphStore partialDispatcherServicesWithJobGraphStore)
Create a
Dispatcher . |
MiniDispatcher |
JobDispatcherFactory.createDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
PartialDispatcherServicesWithJobGraphStore partialDispatcherServicesWithJobGraphStore) |
StandaloneDispatcher |
SessionDispatcherFactory.createDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
PartialDispatcherServicesWithJobGraphStore partialDispatcherServicesWithJobGraphStore) |
Constructor and Description |
---|
MiniDispatcher(RpcService rpcService,
DispatcherId fencingToken,
DispatcherServices dispatcherServices,
JobGraph jobGraph,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
ClusterEntrypoint.ExecutionMode executionMode) |
SingleJobJobGraphStore(JobGraph jobGraph) |
Constructor and Description |
---|
Dispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
DispatcherServices dispatcherServices) |
StandaloneDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
DispatcherServices dispatcherServices) |
Modifier and Type | Method and Description |
---|---|
AbstractDispatcherLeaderProcess.DispatcherGatewayService |
AbstractDispatcherLeaderProcess.DispatcherGatewayServiceFactory.create(DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
JobGraphWriter jobGraphWriter) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
FileJobGraphRetriever.retrieveJobGraph(Configuration configuration) |
JobGraph |
JobGraphRetriever.retrieveJobGraph(Configuration configuration)
Retrieve the
JobGraph . |
Modifier and Type | Method and Description |
---|---|
static ExecutionGraph |
ExecutionGraphBuilder.buildGraph(ExecutionGraph prior,
JobGraph jobGraph,
Configuration jobManagerConfig,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
SlotProvider slotProvider,
ClassLoader classLoader,
CheckpointRecoveryFactory recoveryFactory,
Time rpcTimeout,
RestartStrategy restartStrategy,
MetricGroup metrics,
BlobWriter blobWriter,
Time allocationTimeout,
org.slf4j.Logger log,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker)
Builds the ExecutionGraph from the JobGraph.
|
static ExecutionGraph |
ExecutionGraphBuilder.buildGraph(ExecutionGraph prior,
JobGraph jobGraph,
Configuration jobManagerConfig,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
SlotProvider slotProvider,
ClassLoader classLoader,
CheckpointRecoveryFactory recoveryFactory,
Time rpcTimeout,
RestartStrategy restartStrategy,
MetricGroup metrics,
BlobWriter blobWriter,
Time allocationTimeout,
org.slf4j.Logger log,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
FailoverStrategy.Factory failoverStrategyFactory) |
Modifier and Type | Method and Description |
---|---|
static void |
JobGraphUtils.addUserArtifactEntries(Collection<Tuple2<String,DistributedCache.DistributedCacheEntry>> userArtifacts,
JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
static String |
JsonPlanGenerator.generatePlan(JobGraph jg) |
Constructor and Description |
---|
DefaultLogicalTopology(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JobGraphStore.recoverJobGraph(JobID jobId)
|
JobGraph |
StandaloneJobGraphStore.recoverJobGraph(JobID jobId) |
JobGraph |
ZooKeeperJobGraphStore.recoverJobGraph(JobID jobId) |
Modifier and Type | Method and Description |
---|---|
void |
JobGraphWriter.putJobGraph(JobGraph jobGraph)
Adds the
JobGraph instance. |
void |
ThrowingJobGraphWriter.putJobGraph(JobGraph jobGraph) |
void |
StandaloneJobGraphStore.putJobGraph(JobGraph jobGraph) |
void |
ZooKeeperJobGraphStore.putJobGraph(JobGraph jobGraph) |
Constructor and Description |
---|
ZooKeeperJobGraphStore(String zooKeeperFullBasePath,
ZooKeeperStateHandleStore<JobGraph> zooKeeperStateHandleStore,
org.apache.flink.shaded.curator4.org.apache.curator.framework.recipes.cache.PathChildrenCache pathCache)
Submitted job graph store backed by ZooKeeper.
|
Constructor and Description |
---|
JobManagerRunnerImpl(JobGraph jobGraph,
JobMasterServiceFactory jobMasterFactory,
HighAvailabilityServices haServices,
LibraryCacheManager.ClassLoaderLease classLoaderLease,
Executor executor,
FatalErrorHandler fatalErrorHandler)
Exceptions that occur while creating the JobManager or JobManagerRunnerImpl are directly
thrown and not reported to the given
FatalErrorHandler . |
JobMaster(RpcService rpcService,
JobMasterConfiguration jobMasterConfiguration,
ResourceID resourceId,
JobGraph jobGraph,
HighAvailabilityServices highAvailabilityService,
SlotPoolFactory slotPoolFactory,
SchedulerFactory schedulerFactory,
JobManagerSharedServices jobManagerSharedServices,
HeartbeatServices heartbeatServices,
JobManagerJobMetricGroupFactory jobMetricGroupFactory,
OnCompletionActions jobCompletionActions,
FatalErrorHandler fatalErrorHandler,
ClassLoader userCodeLoader,
SchedulerNGFactory schedulerNGFactory,
ShuffleMaster<?> shuffleMaster,
PartitionTrackerFactory partitionTrackerFactory) |
Modifier and Type | Method and Description |
---|---|
JobManagerJobMetricGroup |
JobManagerJobMetricGroupFactory.create(JobGraph jobGraph)
Create a new
JobManagerJobMetricGroup . |
JobManagerJobMetricGroup |
DefaultJobManagerJobMetricGroupFactory.create(JobGraph jobGraph) |
JobManagerJobMetricGroup |
UnregisteredJobManagerJobMetricGroupFactory.create(JobGraph jobGraph) |
JobMaster |
DefaultJobMasterServiceFactory.createJobMasterService(JobGraph jobGraph,
OnCompletionActions jobCompletionActions,
ClassLoader userCodeClassloader) |
JobMasterService |
JobMasterServiceFactory.createJobMasterService(JobGraph jobGraph,
OnCompletionActions jobCompletionActions,
ClassLoader userCodeClassloader) |
Modifier and Type | Method and Description |
---|---|
JobManagerJobMetricGroup |
UnregisteredMetricGroups.UnregisteredJobManagerMetricGroup.addJob(JobGraph job) |
JobManagerJobMetricGroup |
JobManagerMetricGroup.addJob(JobGraph job) |
Modifier and Type | Method and Description |
---|---|
JobExecutionResult |
JobExecutor.executeJobBlocking(JobGraph jobGraph)
Run the given job and block until its execution result can be returned.
|
JobExecutionResult |
MiniCluster.executeJobBlocking(JobGraph job)
This method runs a job in blocking mode.
|
void |
MiniCluster.runDetached(JobGraph job)
This method executes a job in detached mode.
|
CompletableFuture<JobSubmissionResult> |
MiniCluster.submitJob(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
protected JobGraph |
SchedulerBase.getJobGraph() |
Modifier and Type | Method and Description |
---|---|
SchedulerNG |
SchedulerNGFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker) |
SchedulerNG |
DefaultSchedulerFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker) |
Constructor and Description |
---|
SchedulerBase(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
RestartStrategyFactory restartStrategyFactory,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionVertexVersioner executionVertexVersioner,
boolean legacyScheduling) |
Constructor and Description |
---|
JarPlanHandler(GatewayRetriever<? extends RestfulGateway> leaderRetriever,
Time timeout,
Map<String,String> responseHeaders,
MessageHeaders<JarPlanRequestBody,JobPlanInfo,org.apache.flink.runtime.webmonitor.handlers.JarPlanMessageParameters> messageHeaders,
Path jarDir,
Configuration configuration,
Executor executor,
java.util.function.Function<JobGraph,JobPlanInfo> planGenerator) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JarHandlerUtils.JarHandlerContext.toJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
boolean suppressOutput) |
Modifier and Type | Method and Description |
---|---|
static JobGraph |
StreamingJobGraphGenerator.createJobGraph(StreamGraph streamGraph) |
static JobGraph |
StreamingJobGraphGenerator.createJobGraph(StreamGraph streamGraph,
JobID jobID) |
JobGraph |
StreamGraph.getJobGraph()
|
JobGraph |
StreamGraph.getJobGraph(JobID jobID)
|
Modifier and Type | Method and Description |
---|---|
ClusterClientProvider<org.apache.hadoop.yarn.api.records.ApplicationId> |
YarnClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.