Modifier and Type | Method and Description |
---|---|
RestClusterClient<StandaloneClusterId> |
StandaloneClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
ClusterClient<T> |
ClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached)
Deploys a per-job cluster with the given job on the cluster.
|
Modifier and Type | Method and Description |
---|---|
static JobGraph |
PackagedProgramUtils.createJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
int defaultParallelism)
|
static JobGraph |
PackagedProgramUtils.createJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
int defaultParallelism,
JobID jobID)
|
static JobGraph |
ClusterClient.getJobGraph(Configuration flinkConfig,
FlinkPlan optPlan,
List<URL> jarFiles,
List<URL> classpaths,
SavepointRestoreSettings savepointSettings) |
static JobGraph |
ClusterClient.getJobGraph(Configuration flinkConfig,
PackagedProgram prog,
FlinkPlan optPlan,
SavepointRestoreSettings savepointSettings) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<JobSubmissionResult> |
NewClusterClient.submitJob(JobGraph jobGraph)
Submit the given
JobGraph to the cluster. |
CompletableFuture<JobSubmissionResult> |
MiniClusterClient.submitJob(JobGraph jobGraph) |
abstract JobSubmissionResult |
ClusterClient.submitJob(JobGraph jobGraph,
ClassLoader classLoader)
Calls the subclasses' submitJob method.
|
JobSubmissionResult |
MiniClusterClient.submitJob(JobGraph jobGraph,
ClassLoader classLoader) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<JobSubmissionResult> |
RestClusterClient.submitJob(JobGraph jobGraph)
Submits the given
JobGraph to the dispatcher. |
JobSubmissionResult |
RestClusterClient.submitJob(JobGraph jobGraph,
ClassLoader classLoader) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
StreamingPlan.getJobGraph()
|
abstract JobGraph |
StreamingPlan.getJobGraph(JobID jobID)
|
Modifier and Type | Method and Description |
---|---|
JobGraph |
JobGraphGenerator.compileJobGraph(OptimizedPlan program)
Translates a
OptimizedPlan into a
JobGraph . |
JobGraph |
JobGraphGenerator.compileJobGraph(OptimizedPlan program,
JobID jobId) |
Modifier and Type | Method and Description |
---|---|
static void |
JobGraphGenerator.addUserArtifactEntries(Collection<Tuple2<String,DistributedCache.DistributedCacheEntry>> userArtifacts,
JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
static void |
ClientUtils.extractAndUploadJobGraphFiles(JobGraph jobGraph,
SupplierWithException<BlobClient,IOException> clientSupplier)
Extracts all files required for the execution from the given
JobGraph and uploads them using the BlobClient
from the given Supplier . |
static void |
ClientUtils.uploadJobGraphFiles(JobGraph jobGraph,
Collection<Path> userJars,
Collection<Tuple2<String,Path>> userArtifacts,
SupplierWithException<BlobClient,IOException> clientSupplier)
Uploads the given jars and artifacts required for the execution of the given
JobGraph using the BlobClient from
the given Supplier . |
Modifier and Type | Method and Description |
---|---|
JobManagerRunner |
DefaultJobManagerRunnerFactory.createJobManagerRunner(JobGraph jobGraph,
Configuration configuration,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
JobManagerSharedServices jobManagerServices,
JobManagerJobMetricGroupFactory jobManagerJobMetricGroupFactory,
FatalErrorHandler fatalErrorHandler) |
JobManagerRunner |
JobManagerRunnerFactory.createJobManagerRunner(JobGraph jobGraph,
Configuration configuration,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
JobManagerSharedServices jobManagerServices,
JobManagerJobMetricGroupFactory jobManagerJobMetricGroupFactory,
FatalErrorHandler fatalErrorHandler) |
CompletableFuture<Acknowledge> |
DispatcherGateway.submitJob(JobGraph jobGraph,
Time timeout)
Submit a job to the dispatcher.
|
CompletableFuture<Acknowledge> |
Dispatcher.submitJob(JobGraph jobGraph,
Time timeout) |
CompletableFuture<Acknowledge> |
MiniDispatcher.submitJob(JobGraph jobGraph,
Time timeout) |
Constructor and Description |
---|
MiniDispatcher(RpcService rpcService,
String endpointId,
Configuration configuration,
HighAvailabilityServices highAvailabilityServices,
GatewayRetriever<ResourceManagerGateway> resourceManagerGatewayRetriever,
BlobServer blobServer,
HeartbeatServices heartbeatServices,
JobManagerMetricGroup jobManagerMetricGroup,
String metricQueryServiceAddress,
ArchivedExecutionGraphStore archivedExecutionGraphStore,
JobManagerRunnerFactory jobManagerRunnerFactory,
FatalErrorHandler fatalErrorHandler,
HistoryServerArchivist historyServerArchivist,
JobGraph jobGraph,
ClusterEntrypoint.ExecutionMode executionMode) |
SingleJobSubmittedJobGraphStore(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
FileJobGraphRetriever.retrieveJobGraph(Configuration configuration) |
JobGraph |
JobGraphRetriever.retrieveJobGraph(Configuration configuration)
Retrieve the
JobGraph . |
Modifier and Type | Method and Description |
---|---|
static ExecutionGraph |
ExecutionGraphBuilder.buildGraph(ExecutionGraph prior,
JobGraph jobGraph,
Configuration jobManagerConfig,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
SlotProvider slotProvider,
ClassLoader classLoader,
CheckpointRecoveryFactory recoveryFactory,
Time rpcTimeout,
RestartStrategy restartStrategy,
MetricGroup metrics,
BlobWriter blobWriter,
Time allocationTimeout,
org.slf4j.Logger log,
ShuffleMaster<?> shuffleMaster,
PartitionTracker partitionTracker)
Builds the ExecutionGraph from the JobGraph.
|
static ExecutionGraph |
ExecutionGraphBuilder.buildGraph(ExecutionGraph prior,
JobGraph jobGraph,
Configuration jobManagerConfig,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
SlotProvider slotProvider,
ClassLoader classLoader,
CheckpointRecoveryFactory recoveryFactory,
Time rpcTimeout,
RestartStrategy restartStrategy,
MetricGroup metrics,
BlobWriter blobWriter,
Time allocationTimeout,
org.slf4j.Logger log,
ShuffleMaster<?> shuffleMaster,
PartitionTracker partitionTracker,
FailoverStrategy.Factory failoverStrategyFactory) |
Modifier and Type | Method and Description |
---|---|
static String |
JsonPlanGenerator.generatePlan(JobGraph jg) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
SubmittedJobGraph.getJobGraph()
Returns the submitted
JobGraph . |
Constructor and Description |
---|
SubmittedJobGraph(JobGraph jobGraph)
Creates a
SubmittedJobGraph . |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JobManagerRunner.getJobGraph() |
Constructor and Description |
---|
JobManagerRunner(JobGraph jobGraph,
JobMasterServiceFactory jobMasterFactory,
HighAvailabilityServices haServices,
LibraryCacheManager libraryCacheManager,
Executor executor,
FatalErrorHandler fatalErrorHandler)
Exceptions that occur while creating the JobManager or JobManagerRunner are directly
thrown and not reported to the given
FatalErrorHandler . |
JobMaster(RpcService rpcService,
JobMasterConfiguration jobMasterConfiguration,
ResourceID resourceId,
JobGraph jobGraph,
HighAvailabilityServices highAvailabilityService,
SlotPoolFactory slotPoolFactory,
SchedulerFactory schedulerFactory,
JobManagerSharedServices jobManagerSharedServices,
HeartbeatServices heartbeatServices,
JobManagerJobMetricGroupFactory jobMetricGroupFactory,
OnCompletionActions jobCompletionActions,
FatalErrorHandler fatalErrorHandler,
ClassLoader userCodeLoader,
SchedulerNGFactory schedulerNGFactory,
ShuffleMaster<?> shuffleMaster,
PartitionTrackerFactory partitionTrackerFactory) |
Modifier and Type | Method and Description |
---|---|
JobManagerJobMetricGroup |
JobManagerJobMetricGroupFactory.create(JobGraph jobGraph)
Create a new
JobManagerJobMetricGroup . |
JobManagerJobMetricGroup |
DefaultJobManagerJobMetricGroupFactory.create(JobGraph jobGraph) |
JobManagerJobMetricGroup |
UnregisteredJobManagerJobMetricGroupFactory.create(JobGraph jobGraph) |
JobMaster |
DefaultJobMasterServiceFactory.createJobMasterService(JobGraph jobGraph,
OnCompletionActions jobCompletionActions,
ClassLoader userCodeClassloader) |
JobMasterService |
JobMasterServiceFactory.createJobMasterService(JobGraph jobGraph,
OnCompletionActions jobCompletionActions,
ClassLoader userCodeClassloader) |
Modifier and Type | Method and Description |
---|---|
JobManagerJobMetricGroup |
UnregisteredMetricGroups.UnregisteredJobManagerMetricGroup.addJob(JobGraph job) |
JobManagerJobMetricGroup |
JobManagerMetricGroup.addJob(JobGraph job) |
Modifier and Type | Method and Description |
---|---|
JobExecutionResult |
MiniCluster.executeJobBlocking(JobGraph job)
This method runs a job in blocking mode.
|
JobExecutionResult |
JobExecutor.executeJobBlocking(JobGraph jobGraph)
Run the given job and block until its execution result can be returned.
|
void |
MiniCluster.runDetached(JobGraph job)
This method executes a job in detached mode.
|
CompletableFuture<JobSubmissionResult> |
MiniCluster.submitJob(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
SchedulerNG |
DefaultSchedulerFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
PartitionTracker partitionTracker) |
SchedulerNG |
SchedulerNGFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
PartitionTracker partitionTracker) |
SchedulerNG |
LegacySchedulerFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
PartitionTracker partitionTracker) |
Constructor and Description |
---|
DefaultScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
PartitionTracker partitionTracker) |
LegacyScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
RestartStrategyFactory restartStrategyFactory,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
PartitionTracker partitionTracker) |
Modifier and Type | Method and Description |
---|---|
SchedulingStrategy |
SchedulingStrategyFactory.createInstance(SchedulerOperations schedulerOperations,
SchedulingTopology schedulingTopology,
JobGraph jobGraph) |
SchedulingStrategy |
EagerSchedulingStrategy.Factory.createInstance(SchedulerOperations schedulerOperations,
SchedulingTopology schedulingTopology,
JobGraph jobGraph) |
SchedulingStrategy |
LazyFromSourcesSchedulingStrategy.Factory.createInstance(SchedulerOperations schedulerOperations,
SchedulingTopology schedulingTopology,
JobGraph jobGraph) |
Constructor and Description |
---|
JarPlanHandler(GatewayRetriever<? extends RestfulGateway> leaderRetriever,
Time timeout,
Map<String,String> responseHeaders,
MessageHeaders<JarPlanRequestBody,JobPlanInfo,org.apache.flink.runtime.webmonitor.handlers.JarPlanMessageParameters> messageHeaders,
Path jarDir,
Configuration configuration,
Executor executor,
java.util.function.Function<JobGraph,JobPlanInfo> planGenerator) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JarHandlerUtils.JarHandlerContext.toJobGraph(Configuration configuration) |
Modifier and Type | Method and Description |
---|---|
static JobGraph |
StreamingJobGraphGenerator.createJobGraph(StreamGraph streamGraph) |
static JobGraph |
StreamingJobGraphGenerator.createJobGraph(StreamGraph streamGraph,
JobID jobID) |
JobGraph |
StreamGraph.getJobGraph(JobID jobID)
Gets the assembled
JobGraph with a given job id. |
Modifier and Type | Method and Description |
---|---|
JobGraph |
ExecutionContext.EnvironmentInstance.createJobGraph(String name) |
Constructor and Description |
---|
ProgramDeployer(ExecutionContext<C> context,
String jobName,
JobGraph jobGraph,
Result<C> result,
boolean awaitJobResult)
Deploys a table program on the cluster.
|
Modifier and Type | Method and Description |
---|---|
protected ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId> |
AbstractYarnClusterDescriptor.deployInternal(ClusterSpecification clusterSpecification,
String applicationName,
String yarnClusterEntrypoint,
JobGraph jobGraph,
boolean detached)
This method will block until the ApplicationMaster/JobManager have been deployed on YARN.
|
ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId> |
YarnClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
org.apache.hadoop.yarn.api.records.ApplicationReport |
AbstractYarnClusterDescriptor.startAppMaster(Configuration configuration,
String applicationName,
String yarnClusterEntrypoint,
JobGraph jobGraph,
org.apache.hadoop.yarn.client.api.YarnClient yarnClient,
org.apache.hadoop.yarn.client.api.YarnClientApplication yarnApplication,
ClusterSpecification clusterSpecification) |
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.