Modifier and Type | Method and Description |
---|---|
static JobID |
JobID.fromByteArray(byte[] bytes)
Creates a new JobID from the given byte sequence.
|
static JobID |
JobID.fromByteBuffer(ByteBuffer buf) |
static JobID |
JobID.fromHexString(String hexString) |
static JobID |
JobID.generate()
Creates a new (statistically) random JobID.
|
JobID |
Plan.getJobId()
Gets the ID of the job that the dataflow plan belongs to.
|
JobID |
JobSubmissionResult.getJobID()
Returns the JobID assigned to the job by the Flink runtime.
|
Modifier and Type | Method and Description |
---|---|
abstract void |
PlanExecutor.endSession(JobID jobID)
Ends the job session, identified by the given JobID.
|
void |
Plan.setJobId(JobID jobId)
Sets the ID of the job that the dataflow plan belongs to.
|
Constructor and Description |
---|
JobExecutionResult(JobID jobID,
long netRuntime,
Map<String,Object> accumulators)
Creates a new JobExecutionResult.
|
JobSubmissionResult(JobID jobID) |
Modifier and Type | Field and Description |
---|---|
protected JobID |
ExecutionEnvironment.jobID
The ID of the session, defined by this execution environment.
|
Modifier and Type | Method and Description |
---|---|
JobID |
ExecutionEnvironment.getId()
Gets the JobID by which this environment is identified.
|
Modifier and Type | Method and Description |
---|---|
void |
RemoteExecutor.endSession(JobID jobID) |
void |
LocalExecutor.endSession(JobID jobID) |
Modifier and Type | Method and Description |
---|---|
JobID |
DetachedEnvironment.DetachedJobExecutionResult.getJobID() |
Modifier and Type | Method and Description |
---|---|
void |
ClusterClient.cancel(JobID jobId)
Cancels a job identified by the job id.
|
String |
ClusterClient.cancelWithSavepoint(JobID jobId,
String savepointDirectory)
Cancels a job identified by the job id and triggers a savepoint.
|
JobListeningContext |
ClusterClient.connectToJob(JobID jobID)
Reattaches to a running job with the given job id.
|
void |
ClusterClient.endSession(JobID jobId)
Tells the JobManager to finish the session (job) defined by the given ID.
|
Map<String,Object> |
ClusterClient.getAccumulators(JobID jobID)
Requests and returns the accumulators for the given job identifier.
|
Map<String,Object> |
ClusterClient.getAccumulators(JobID jobID,
ClassLoader loader)
Requests and returns the accumulators for the given job identifier.
|
JobExecutionResult |
ClusterClient.retrieveJob(JobID jobID)
Reattaches to a running from from the supplied job id.
|
void |
ClusterClient.stop(JobID jobId)
Stops a program on Flink cluster whose job-manager is configured in this client's configuration.
|
CompletableFuture<String> |
ClusterClient.triggerSavepoint(JobID jobId,
String savepointDirectory)
Triggers a savepoint for the job identified by the job id.
|
Modifier and Type | Method and Description |
---|---|
void |
ClusterClient.endSessions(List<JobID> jobIds)
Tells the JobManager to finish the sessions (jobs) defined by the given IDs.
|
Modifier and Type | Method and Description |
---|---|
void |
RestClusterClient.cancel(JobID jobID) |
String |
RestClusterClient.cancelWithSavepoint(JobID jobId,
String savepointDirectory) |
void |
RestClusterClient.stop(JobID jobID) |
CompletableFuture<String> |
RestClusterClient.triggerSavepoint(JobID jobId,
String savepointDirectory) |
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
RocksDBStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry) |
CheckpointStreamFactory |
RocksDBStateBackend.createSavepointStreamFactory(JobID jobId,
String operatorIdentifier,
String targetLocation) |
CheckpointStreamFactory |
RocksDBStateBackend.createStreamFactory(JobID jobId,
String operatorIdentifier) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JobGraphGenerator.compileJobGraph(OptimizedPlan program,
JobID jobId) |
Modifier and Type | Method and Description |
---|---|
<K,S extends State,V> |
QueryableStateClient.getKvState(JobID jobId,
String queryableStateName,
K key,
TypeHint<K> keyTypeHint,
StateDescriptor<S,V> stateDescriptor)
Returns a future holding the request result.
|
<K,S extends State,V> |
QueryableStateClient.getKvState(JobID jobId,
String queryableStateName,
K key,
TypeInformation<K> keyTypeInfo,
StateDescriptor<S,V> stateDescriptor)
Returns a future holding the request result.
|
Modifier and Type | Method and Description |
---|---|
JobID |
KvStateRequest.getJobId() |
Constructor and Description |
---|
KvStateRequest(JobID jobId,
String stateName,
int keyHashCode,
byte[] serializedKeyAndNamespace) |
Modifier and Type | Field and Description |
---|---|
protected JobID |
AccumulatorRegistry.jobID |
Modifier and Type | Method and Description |
---|---|
JobID |
AccumulatorSnapshot.getJobID() |
Constructor and Description |
---|
AccumulatorRegistry(JobID jobID,
ExecutionAttemptID taskID) |
AccumulatorSnapshot(JobID jobID,
ExecutionAttemptID executionAttemptID,
Map<String,Accumulator<?,?>> userAccumulators) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<Acknowledge> |
AkkaJobManagerGateway.cancelJob(JobID jobId,
Time timeout) |
CompletableFuture<String> |
AkkaJobManagerGateway.cancelJobWithSavepoint(JobID jobId,
String savepointPath,
Time timeout) |
CompletableFuture<Optional<org.apache.flink.runtime.messages.JobManagerMessages.ClassloadingProps>> |
AkkaJobManagerGateway.requestClassloadingProps(JobID jobId,
Time timeout) |
CompletableFuture<AccessExecutionGraph> |
AkkaJobManagerGateway.requestJob(JobID jobId,
Time timeout) |
CompletableFuture<Acknowledge> |
AkkaJobManagerGateway.stopJob(JobID jobId,
Time timeout) |
Modifier and Type | Method and Description |
---|---|
boolean |
BlobServer.cleanupJob(JobID jobId)
Removes all BLOBs from local and HA store belonging to the given job ID.
|
boolean |
FileSystemBlobStore.delete(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey) |
boolean |
BlobStore.delete(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey)
Tries to delete a blob from storage.
|
boolean |
VoidBlobStore.delete(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey) |
boolean |
FileSystemBlobStore.deleteAll(JobID jobId) |
boolean |
BlobStore.deleteAll(JobID jobId)
Tries to delete all blobs for the given job from storage.
|
boolean |
VoidBlobStore.deleteAll(JobID jobId) |
boolean |
TransientBlobService.deleteFromCache(JobID jobId,
TransientBlobKey key)
Deletes the file associated with the provided job ID and blob key from the local cache.
|
boolean |
TransientBlobCache.deleteFromCache(JobID jobId,
TransientBlobKey key) |
boolean |
BlobServer.deleteFromCache(JobID jobId,
TransientBlobKey key)
Deletes the file associated with the blob key in the local storage of the blob server.
|
boolean |
BlobView.get(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey,
File localFile)
Copies a blob to a local file.
|
boolean |
FileSystemBlobStore.get(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey,
File localFile) |
boolean |
VoidBlobStore.get(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey,
File localFile) |
File |
PermanentBlobService.getFile(JobID jobId,
PermanentBlobKey key)
Returns the path to a local copy of the file associated with the provided job ID and blob
key.
|
File |
BlobServer.getFile(JobID jobId,
PermanentBlobKey key)
Returns the path to a local copy of the file associated with the provided job ID and blob
key.
|
File |
PermanentBlobCache.getFile(JobID jobId,
PermanentBlobKey key)
Returns the path to a local copy of the file associated with the provided job ID and blob
key.
|
File |
TransientBlobService.getFile(JobID jobId,
TransientBlobKey key)
Returns the path to a local copy of the file associated with the provided job ID and blob
key.
|
File |
TransientBlobCache.getFile(JobID jobId,
TransientBlobKey key) |
File |
BlobServer.getFile(JobID jobId,
TransientBlobKey key)
Retrieves the local path of a file associated with a job and a blob key.
|
protected File |
TransientBlobCache.getFileInternal(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey) |
protected File |
AbstractBlobCache.getFileInternal(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey)
Returns local copy of the file for the BLOB with the given key.
|
int |
PermanentBlobCache.getNumberOfReferenceHolders(JobID jobId) |
File |
TransientBlobCache.getStorageLocation(JobID jobId,
org.apache.flink.runtime.blob.BlobKey key)
Returns a file handle to the file associated with the given blob key on the blob
server.
|
File |
BlobServer.getStorageLocation(JobID jobId,
org.apache.flink.runtime.blob.BlobKey key)
Returns a file handle to the file associated with the given blob key on the blob
server.
|
File |
PermanentBlobCache.getStorageLocation(JobID jobId,
org.apache.flink.runtime.blob.BlobKey key)
Returns a file handle to the file associated with the given blob key on the blob
server.
|
boolean |
FileSystemBlobStore.put(File localFile,
JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey) |
boolean |
BlobStore.put(File localFile,
JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey)
Copies the local file to the blob store.
|
boolean |
VoidBlobStore.put(File localFile,
JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey) |
PermanentBlobKey |
BlobWriter.putPermanent(JobID jobId,
byte[] value)
Uploads the data of the given byte array for the given job to the BLOB server and makes it
a permanent BLOB.
|
PermanentBlobKey |
BlobServer.putPermanent(JobID jobId,
byte[] value) |
PermanentBlobKey |
VoidBlobWriter.putPermanent(JobID jobId,
byte[] value) |
PermanentBlobKey |
BlobWriter.putPermanent(JobID jobId,
InputStream inputStream)
Uploads the data from the given input stream for the given job to the BLOB server and makes it
a permanent BLOB.
|
PermanentBlobKey |
BlobServer.putPermanent(JobID jobId,
InputStream inputStream) |
PermanentBlobKey |
VoidBlobWriter.putPermanent(JobID jobId,
InputStream inputStream) |
TransientBlobKey |
TransientBlobService.putTransient(JobID jobId,
byte[] value)
Uploads the data of the given byte array for the given job to the BLOB server.
|
TransientBlobKey |
TransientBlobCache.putTransient(JobID jobId,
byte[] value) |
TransientBlobKey |
BlobServer.putTransient(JobID jobId,
byte[] value) |
TransientBlobKey |
TransientBlobService.putTransient(JobID jobId,
InputStream inputStream)
Uploads the data from the given input stream for the given job to the BLOB server.
|
TransientBlobKey |
TransientBlobCache.putTransient(JobID jobId,
InputStream inputStream) |
TransientBlobKey |
BlobServer.putTransient(JobID jobId,
InputStream inputStream) |
void |
PermanentBlobCache.registerJob(JobID jobId)
Registers use of job-related BLOBs.
|
void |
PermanentBlobCache.releaseJob(JobID jobId)
Unregisters use of job-related BLOBs and allow them to be released.
|
static <T> Either<SerializedValue<T>,PermanentBlobKey> |
BlobWriter.serializeAndTryOffload(T value,
JobID jobId,
BlobWriter blobWriter)
Serializes the given value and offloads it to the BlobServer if its size exceeds the minimum
offloading size of the BlobServer.
|
static List<PermanentBlobKey> |
BlobClient.uploadJarFiles(InetSocketAddress serverAddress,
Configuration clientConfig,
JobID jobId,
List<Path> jars)
Uploads the JAR files to the
PermanentBlobService of the BlobServer at the
given address with HA as configured. |
Modifier and Type | Method and Description |
---|---|
JobID |
PendingCheckpoint.getJobId() |
JobID |
CompletedCheckpoint.getJobId() |
Modifier and Type | Method and Description |
---|---|
void |
CheckpointCoordinatorGateway.acknowledgeCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
CheckpointMetrics checkpointMetrics,
TaskStateSnapshot subtaskState) |
CheckpointIDCounter |
StandaloneCheckpointRecoveryFactory.createCheckpointIDCounter(JobID ignored) |
CheckpointIDCounter |
ZooKeeperCheckpointRecoveryFactory.createCheckpointIDCounter(JobID jobID) |
CheckpointIDCounter |
CheckpointRecoveryFactory.createCheckpointIDCounter(JobID jobId)
Creates a
CheckpointIDCounter instance for a job. |
CompletedCheckpointStore |
StandaloneCheckpointRecoveryFactory.createCheckpointStore(JobID jobId,
int maxNumberOfCheckpointsToRetain,
ClassLoader userClassLoader) |
CompletedCheckpointStore |
ZooKeeperCheckpointRecoveryFactory.createCheckpointStore(JobID jobId,
int maxNumberOfCheckpointsToRetain,
ClassLoader userClassLoader) |
CompletedCheckpointStore |
CheckpointRecoveryFactory.createCheckpointStore(JobID jobId,
int maxNumberOfCheckpointsToRetain,
ClassLoader userClassLoader)
Creates a
CompletedCheckpointStore instance for a job. |
void |
CheckpointCoordinatorGateway.declineCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
Throwable cause) |
void |
CheckpointCoordinatorDeActivator.jobStatusChanges(JobID jobId,
JobStatus newJobStatus,
long timestamp,
Throwable error) |
Constructor and Description |
---|
CheckpointCoordinator(JobID job,
long baseInterval,
long checkpointTimeout,
long minPauseBetweenCheckpoints,
int maxConcurrentCheckpointAttempts,
ExternalizedCheckpointSettings externalizeSettings,
ExecutionVertex[] tasksToTrigger,
ExecutionVertex[] tasksToWaitFor,
ExecutionVertex[] tasksToCommitTo,
CheckpointIDCounter checkpointIDCounter,
CompletedCheckpointStore completedCheckpointStore,
String checkpointDirectory,
Executor executor,
SharedStateRegistryFactory sharedStateRegistryFactory) |
CompletedCheckpoint(JobID job,
long checkpointID,
long timestamp,
long completionTimestamp,
Map<OperatorID,OperatorState> operatorStates,
Collection<MasterState> masterHookStates,
CheckpointProperties props,
StreamStateHandle externalizedMetadata,
String externalPointer) |
PendingCheckpoint(JobID jobId,
long checkpointId,
long checkpointTimestamp,
Map<ExecutionAttemptID,ExecutionVertex> verticesToConfirm,
CheckpointProperties props,
String targetDirectory,
Executor executor) |
Modifier and Type | Method and Description |
---|---|
static String |
SavepointStore.createSavepointDirectory(String baseDirectory,
JobID jobId)
Creates a savepoint directory.
|
static CompletedCheckpoint |
SavepointLoader.loadAndValidateSavepoint(JobID jobId,
Map<JobVertexID,ExecutionJobVertex> tasks,
String savepointPath,
ClassLoader classLoader,
boolean allowNonRestoredState)
Loads a savepoint back as a
CompletedCheckpoint . |
Modifier and Type | Method and Description |
---|---|
JobID |
JobStatusMessage.getJobId() |
JobID |
SerializedJobExecutionResult.getJobId() |
JobID |
JobListeningContext.getJobID() |
JobID |
JobExecutionException.getJobID() |
Modifier and Type | Method and Description |
---|---|
static JobListeningContext |
JobClient.attachToRunningJob(JobID jobID,
Configuration configuration,
akka.actor.ActorSystem actorSystem,
HighAvailabilityServices highAvailabilityServices,
scala.concurrent.duration.FiniteDuration timeout,
boolean sysoutLogUpdates)
Attaches to a running Job using the JobID.
|
static ClassLoader |
JobClient.retrieveClassLoader(JobID jobID,
JobManagerGateway jobManager,
Configuration config,
HighAvailabilityServices highAvailabilityServices,
Time timeout)
Reconstructs the class loader by first requesting information about it at the JobManager
and then downloading missing jar files.
|
Constructor and Description |
---|
JobCancellationException(JobID jobID,
String msg,
Throwable cause) |
JobExecutionException(JobID jobID,
String msg) |
JobExecutionException(JobID jobID,
String msg,
Throwable cause)
Constructs a new job execution exception.
|
JobExecutionException(JobID jobID,
Throwable cause) |
JobListeningContext(JobID jobID,
scala.concurrent.Future<Object> jobResultFuture,
akka.actor.ActorRef jobClientActor,
scala.concurrent.duration.FiniteDuration timeout,
akka.actor.ActorSystem actorSystem,
Configuration configuration,
HighAvailabilityServices highAvailabilityServices)
Constructor to use when the class loader is not available.
|
JobListeningContext(JobID jobID,
scala.concurrent.Future<Object> jobResultFuture,
akka.actor.ActorRef jobClientActor,
scala.concurrent.duration.FiniteDuration timeout,
ClassLoader classLoader,
HighAvailabilityServices highAvailabilityServices)
Constructor to use when the class loader is available.
|
JobRetrievalException(JobID jobID,
String msg) |
JobRetrievalException(JobID jobID,
String msg,
Throwable cause) |
JobRetrievalException(JobID jobID,
Throwable cause) |
JobStatusMessage(JobID jobId,
String jobName,
JobStatus jobState,
long startTime) |
JobSubmissionException(JobID jobID,
String msg) |
JobSubmissionException(JobID jobID,
String msg,
Throwable cause) |
JobTimeoutException(JobID jobID,
String msg,
Throwable cause) |
SerializedJobExecutionResult(JobID jobID,
long netRuntime,
Map<String,SerializedValue<Object>> accumulators)
Creates a new SerializedJobExecutionResult.
|
Modifier and Type | Method and Description |
---|---|
JobID |
ShutdownClusterAfterJob.jobId() |
Constructor and Description |
---|
ShutdownClusterAfterJob(JobID jobId) |
Modifier and Type | Method and Description |
---|---|
JobID |
TaskDeploymentDescriptor.getJobId()
Returns the task's job ID.
|
Constructor and Description |
---|
TaskDeploymentDescriptor(JobID jobId,
TaskDeploymentDescriptor.MaybeOffloaded<JobInformation> serializedJobInformation,
TaskDeploymentDescriptor.MaybeOffloaded<TaskInformation> serializedTaskInformation,
ExecutionAttemptID executionAttemptId,
AllocationID allocationId,
int subtaskIndex,
int attemptNumber,
int targetSlotNumber,
TaskStateSnapshot taskStateHandles,
Collection<ResultPartitionDeploymentDescriptor> resultPartitionDeploymentDescriptors,
Collection<InputGateDeploymentDescriptor> inputGateDeploymentDescriptors) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<Collection<JobID>> |
DispatcherGateway.listJobs(Time timeout)
List the current set of submitted jobs.
|
CompletableFuture<Collection<JobID>> |
Dispatcher.listJobs(Time timeout) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<Acknowledge> |
DispatcherGateway.cancelJob(JobID jobId,
Time timeout)
Cancel the given job.
|
CompletableFuture<Acknowledge> |
Dispatcher.cancelJob(JobID jobId,
Time timeout) |
CompletableFuture<AccessExecutionGraph> |
Dispatcher.requestJob(JobID jobId,
Time timeout) |
CompletableFuture<Acknowledge> |
DispatcherGateway.stopJob(JobID jobId,
Time timeout)
Stop the given job.
|
CompletableFuture<Acknowledge> |
Dispatcher.stopJob(JobID jobId,
Time timeout) |
Modifier and Type | Method and Description |
---|---|
JobID |
Environment.getJobID()
Returns the ID of the job that the task belongs to.
|
Modifier and Type | Method and Description |
---|---|
ClassLoader |
BlobLibraryCacheManager.getClassLoader(JobID jobId) |
ClassLoader |
LibraryCacheManager.getClassLoader(JobID id)
Returns the user code class loader associated with id.
|
void |
BlobLibraryCacheManager.registerJob(JobID id,
Collection<PermanentBlobKey> requiredJarFiles,
Collection<URL> requiredClasspaths) |
void |
LibraryCacheManager.registerJob(JobID id,
Collection<PermanentBlobKey> requiredJarFiles,
Collection<URL> requiredClasspaths)
Registers a job with its required jar files and classpaths.
|
void |
BlobLibraryCacheManager.registerTask(JobID jobId,
ExecutionAttemptID task,
Collection<PermanentBlobKey> requiredJarFiles,
Collection<URL> requiredClasspaths) |
void |
LibraryCacheManager.registerTask(JobID id,
ExecutionAttemptID execution,
Collection<PermanentBlobKey> requiredJarFiles,
Collection<URL> requiredClasspaths)
Registers a job task execution with its required jar files and classpaths.
|
void |
BlobLibraryCacheManager.unregisterJob(JobID id) |
void |
LibraryCacheManager.unregisterJob(JobID id)
Unregisters a job from the library cache manager.
|
void |
BlobLibraryCacheManager.unregisterTask(JobID jobId,
ExecutionAttemptID task) |
void |
LibraryCacheManager.unregisterTask(JobID id,
ExecutionAttemptID execution)
Unregisters a job task execution from the library cache manager.
|
Modifier and Type | Method and Description |
---|---|
JobID |
JobInformation.getJobId() |
JobID |
ExecutionVertex.getJobId() |
JobID |
ExecutionJobVertex.getJobId() |
JobID |
AccessExecutionGraph.getJobID()
Returns the
JobID for this execution graph. |
JobID |
ExecutionGraph.getJobID() |
JobID |
ArchivedExecutionGraph.getJobID() |
Modifier and Type | Method and Description |
---|---|
void |
ExecutionStatusListener.executionStatusChanged(JobID jobID,
JobVertexID vertexID,
String taskName,
int totalNumberOfSubTasks,
int subtaskIndex,
ExecutionAttemptID executionID,
ExecutionState newExecutionState,
long timestamp,
String optionalMessage)
Called whenever the execution status of a task changes.
|
void |
StatusListenerMessenger.executionStatusChanged(JobID jobID,
JobVertexID vertexID,
String taskName,
int taskParallelism,
int subtaskIndex,
ExecutionAttemptID executionID,
ExecutionState newExecutionState,
long timestamp,
String optionalMessage) |
void |
JobStatusListener.jobStatusChanges(JobID jobId,
JobStatus newJobStatus,
long timestamp,
Throwable error)
This method is called whenever the status of the job changes.
|
void |
StatusListenerMessenger.jobStatusChanges(JobID jobId,
JobStatus newJobStatus,
long timestamp,
Throwable error) |
Constructor and Description |
---|
ArchivedExecutionGraph(JobID jobID,
String jobName,
Map<JobVertexID,ArchivedExecutionJobVertex> tasks,
List<ArchivedExecutionJobVertex> verticesInCreationOrder,
long[] stateTimestamps,
JobStatus state,
ErrorInfo failureCause,
String jsonPlan,
StringifiedAccumulatorResult[] archivedUserAccumulators,
Map<String,SerializedValue<Object>> serializedUserAccumulators,
ArchivedExecutionConfig executionConfig,
boolean isStoppable,
CheckpointCoordinatorConfiguration jobCheckpointingConfiguration,
CheckpointStatsSnapshot checkpointStatsSnapshot) |
JobInformation(JobID jobId,
String jobName,
SerializedValue<ExecutionConfig> serializedExecutionConfig,
Configuration jobConfiguration,
Collection<PermanentBlobKey> requiredJarFileBlobKeys,
Collection<URL> requiredClasspathURLs) |
Modifier and Type | Method and Description |
---|---|
Future<Path> |
FileCache.createTmpFile(String name,
DistributedCache.DistributedCacheEntry entry,
JobID jobID)
If the file doesn't exists locally, it will copy the file to the temp directory.
|
void |
FileCache.deleteTmpFile(String name,
JobID jobID)
Deletes the local file after a 5 second delay.
|
Modifier and Type | Field and Description |
---|---|
static JobID |
HighAvailabilityServices.DEFAULT_JOB_ID
This JobID should be used to identify the old JobManager when using the
HighAvailabilityServices . |
Modifier and Type | Method and Description |
---|---|
void |
FsNegativeRunningJobsRegistry.clearJob(JobID jobID) |
void |
RunningJobsRegistry.clearJob(JobID jobID)
Clear job state form the registry, usually called after job finish
|
LeaderElectionService |
HighAvailabilityServices.getJobManagerLeaderElectionService(JobID jobID)
Gets the leader election service for the given job.
|
LeaderRetrievalService |
HighAvailabilityServices.getJobManagerLeaderRetriever(JobID jobID)
Deprecated.
This method should only be used by non Flip-6 code where the JobManager acts as the master.
|
LeaderRetrievalService |
HighAvailabilityServices.getJobManagerLeaderRetriever(JobID jobID,
String defaultJobManagerAddress)
Gets the leader retriever for the job JobMaster which is responsible for the given job
|
RunningJobsRegistry.JobSchedulingStatus |
FsNegativeRunningJobsRegistry.getJobSchedulingStatus(JobID jobID) |
RunningJobsRegistry.JobSchedulingStatus |
RunningJobsRegistry.getJobSchedulingStatus(JobID jobID)
Gets the scheduling status of a job.
|
void |
FsNegativeRunningJobsRegistry.setJobFinished(JobID jobID) |
void |
RunningJobsRegistry.setJobFinished(JobID jobID)
Marks a job as completed.
|
void |
FsNegativeRunningJobsRegistry.setJobRunning(JobID jobID) |
void |
RunningJobsRegistry.setJobRunning(JobID jobID)
Marks a job as running.
|
Modifier and Type | Method and Description |
---|---|
LeaderElectionService |
EmbeddedHaServices.getJobManagerLeaderElectionService(JobID jobID) |
LeaderRetrievalService |
EmbeddedHaServices.getJobManagerLeaderRetriever(JobID jobID) |
LeaderRetrievalService |
EmbeddedHaServices.getJobManagerLeaderRetriever(JobID jobID,
String defaultJobManagerAddress) |
Modifier and Type | Method and Description |
---|---|
void |
StandaloneRunningJobsRegistry.clearJob(JobID jobID) |
LeaderElectionService |
StandaloneHaServices.getJobManagerLeaderElectionService(JobID jobID) |
LeaderRetrievalService |
StandaloneHaServices.getJobManagerLeaderRetriever(JobID jobID) |
LeaderRetrievalService |
StandaloneHaServices.getJobManagerLeaderRetriever(JobID jobID,
String defaultJobManagerAddress) |
RunningJobsRegistry.JobSchedulingStatus |
StandaloneRunningJobsRegistry.getJobSchedulingStatus(JobID jobID) |
void |
StandaloneRunningJobsRegistry.setJobFinished(JobID jobID) |
void |
StandaloneRunningJobsRegistry.setJobRunning(JobID jobID) |
Modifier and Type | Method and Description |
---|---|
void |
ZooKeeperRunningJobsRegistry.clearJob(JobID jobID) |
LeaderElectionService |
ZooKeeperHaServices.getJobManagerLeaderElectionService(JobID jobID) |
LeaderRetrievalService |
ZooKeeperHaServices.getJobManagerLeaderRetriever(JobID jobID) |
LeaderRetrievalService |
ZooKeeperHaServices.getJobManagerLeaderRetriever(JobID jobID,
String defaultJobManagerAddress) |
RunningJobsRegistry.JobSchedulingStatus |
ZooKeeperRunningJobsRegistry.getJobSchedulingStatus(JobID jobID) |
void |
ZooKeeperRunningJobsRegistry.setJobFinished(JobID jobID) |
void |
ZooKeeperRunningJobsRegistry.setJobRunning(JobID jobID) |
Modifier and Type | Method and Description |
---|---|
JobID |
Slot.getJobID()
Returns the ID of the job this allocated slot belongs to.
|
Modifier and Type | Method and Description |
---|---|
SharedSlot |
Instance.allocateSharedSlot(JobID jobID,
SlotSharingGroupAssignment sharingGroupAssignment)
Allocates a shared slot on this TaskManager instance.
|
SimpleSlot |
Instance.allocateSimpleSlot(JobID jobID)
Allocates a simple slot on this TaskManager instance.
|
Constructor and Description |
---|
SharedSlot(JobID jobID,
SlotOwner owner,
TaskManagerLocation location,
int slotNumber,
TaskManagerGateway taskManagerGateway,
SlotSharingGroupAssignment assignmentGroup)
Creates a new shared slot that has no parent (is a root slot) and does not belong to any task group.
|
SharedSlot(JobID jobID,
SlotOwner owner,
TaskManagerLocation location,
int slotNumber,
TaskManagerGateway taskManagerGateway,
SlotSharingGroupAssignment assignmentGroup,
SharedSlot parent,
AbstractID groupId)
Creates a new shared slot that has is a sub-slot of the given parent shared slot, and that belongs
to the given task group.
|
SimpleSlot(JobID jobID,
SlotOwner owner,
TaskManagerLocation location,
int slotNumber,
TaskManagerGateway taskManagerGateway)
Creates a new simple slot that stands alone and does not belong to shared slot.
|
SimpleSlot(JobID jobID,
SlotOwner owner,
TaskManagerLocation location,
int slotNumber,
TaskManagerGateway taskManagerGateway,
SharedSlot parent,
AbstractID groupID)
Creates a new simple slot that belongs to the given shared slot and
is identified by the given ID.
|
Slot(JobID jobID,
SlotOwner owner,
TaskManagerLocation location,
int slotNumber,
TaskManagerGateway taskManagerGateway,
SharedSlot parent,
AbstractID groupID)
Base constructor for slots.
|
SlotPool(RpcService rpcService,
JobID jobId) |
SlotPool(RpcService rpcService,
JobID jobId,
Clock clock,
Time slotRequestTimeout,
Time resourceManagerAllocationTimeout,
Time resourceManagerRequestTimeout) |
Modifier and Type | Method and Description |
---|---|
TaskKvStateRegistry |
NetworkEnvironment.createKvStateTaskRegistry(JobID jobId,
JobVertexID jobVertexId) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<ExecutionState> |
PartitionProducerStateChecker.requestPartitionProducerState(JobID jobId,
IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId)
Requests the execution state of the execution producing a result partition.
|
Modifier and Type | Method and Description |
---|---|
JobID |
ResultPartition.getJobId() |
Modifier and Type | Method and Description |
---|---|
void |
ResultPartitionConsumableNotifier.notifyPartitionConsumable(JobID jobId,
ResultPartitionID partitionId,
TaskActions taskActions) |
Constructor and Description |
---|
ResultPartition(String owningTaskName,
TaskActions taskActions,
JobID jobId,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numberOfSubpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
ResultPartitionConsumableNotifier partitionConsumableNotifier,
IOManager ioManager,
boolean sendScheduleOrUpdateConsumersMessage) |
Modifier and Type | Method and Description |
---|---|
static SingleInputGate |
SingleInputGate.create(String owningTaskName,
JobID jobId,
ExecutionAttemptID executionId,
InputGateDeploymentDescriptor igdd,
NetworkEnvironment networkEnvironment,
TaskActions taskActions,
TaskIOMetricGroup metrics)
Creates an input gate and all of its input channels.
|
Constructor and Description |
---|
SingleInputGate(String owningTaskName,
JobID jobId,
IntermediateDataSetID consumedResultId,
ResultPartitionType consumedPartitionType,
int consumedSubpartitionIndex,
int numberOfInputChannels,
TaskActions taskActions,
TaskIOMetricGroup metrics) |
Modifier and Type | Method and Description |
---|---|
JobID |
JobGraph.getJobID()
Returns the ID of the job.
|
Constructor and Description |
---|
JobGraph(JobID jobId,
String jobName)
Constructs a new job graph with the given job ID (or a random ID, if
null is passed),
the given name and the given execution configuration (see ExecutionConfig ). |
JobGraph(JobID jobId,
String jobName,
JobVertex... vertices)
Constructs a new job graph with the given name, the given
ExecutionConfig ,
the given jobId or a random one if null supplied, and the given job vertices. |
Modifier and Type | Method and Description |
---|---|
JobID |
SubmittedJobGraph.getJobId()
|
static JobID |
ZooKeeperSubmittedJobGraphStore.jobIdfromPath(String path)
Returns the JobID from the given path in ZooKeeper.
|
Modifier and Type | Method and Description |
---|---|
Collection<JobID> |
SubmittedJobGraphStore.getJobIds()
Get all job ids of submitted job graphs to the submitted job graph store.
|
Collection<JobID> |
StandaloneSubmittedJobGraphStore.getJobIds() |
Collection<JobID> |
ZooKeeperSubmittedJobGraphStore.getJobIds() |
Modifier and Type | Method and Description |
---|---|
static String |
ZooKeeperSubmittedJobGraphStore.getPathForJob(JobID jobId)
Returns the JobID as a String (with leading slash).
|
void |
SubmittedJobGraphStore.SubmittedJobGraphListener.onAddedJobGraph(JobID jobId)
Callback for
SubmittedJobGraph instances added by a different SubmittedJobGraphStore instance. |
void |
SubmittedJobGraphStore.SubmittedJobGraphListener.onRemovedJobGraph(JobID jobId)
Callback for
SubmittedJobGraph instances removed by a different SubmittedJobGraphStore instance. |
SubmittedJobGraph |
SubmittedJobGraphStore.recoverJobGraph(JobID jobId)
Returns the
SubmittedJobGraph with the given JobID . |
SubmittedJobGraph |
StandaloneSubmittedJobGraphStore.recoverJobGraph(JobID jobId) |
SubmittedJobGraph |
ZooKeeperSubmittedJobGraphStore.recoverJobGraph(JobID jobId) |
void |
SubmittedJobGraphStore.releaseJobGraph(JobID jobId)
Releases the locks on the specified
JobGraph . |
void |
StandaloneSubmittedJobGraphStore.releaseJobGraph(JobID jobId) |
void |
ZooKeeperSubmittedJobGraphStore.releaseJobGraph(JobID jobId) |
void |
SubmittedJobGraphStore.removeJobGraph(JobID jobId)
Removes the
SubmittedJobGraph with the given JobID if it exists. |
void |
StandaloneSubmittedJobGraphStore.removeJobGraph(JobID jobId) |
void |
ZooKeeperSubmittedJobGraphStore.removeJobGraph(JobID jobId) |
Modifier and Type | Method and Description |
---|---|
JobID |
AllocatedSlot.getJobID()
Returns the ID of the job this allocated slot belongs to.
|
Modifier and Type | Method and Description |
---|---|
void |
ActorTaskManagerGateway.notifyCheckpointComplete(ExecutionAttemptID executionAttemptID,
JobID jobId,
long checkpointId,
long timestamp) |
void |
TaskManagerGateway.notifyCheckpointComplete(ExecutionAttemptID executionAttemptID,
JobID jobId,
long checkpointId,
long timestamp)
Notify the given task about a completed checkpoint.
|
void |
ActorTaskManagerGateway.triggerCheckpoint(ExecutionAttemptID executionAttemptID,
JobID jobId,
long checkpointId,
long timestamp,
CheckpointOptions checkpointOptions) |
void |
TaskManagerGateway.triggerCheckpoint(ExecutionAttemptID executionAttemptID,
JobID jobId,
long checkpointId,
long timestamp,
CheckpointOptions checkpointOptions)
Trigger for the given task a checkpoint.
|
Constructor and Description |
---|
AllocatedSlot(AllocationID slotAllocationId,
JobID jobID,
TaskManagerLocation location,
int slotNumber,
ResourceProfile resourceProfile,
TaskManagerGateway taskManagerGateway) |
Modifier and Type | Method and Description |
---|---|
void |
JobMaster.acknowledgeCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
CheckpointMetrics checkpointMetrics,
TaskStateSnapshot checkpointState) |
CompletableFuture<Acknowledge> |
JobManagerGateway.cancelJob(JobID jobId,
Time timeout)
Cancels the given job.
|
CompletableFuture<String> |
JobManagerGateway.cancelJobWithSavepoint(JobID jobId,
String savepointPath,
Time timeout)
Cancels the given job after taking a savepoint and returning its path.
|
void |
JobMaster.declineCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointID,
Throwable reason) |
void |
RpcTaskManagerGateway.notifyCheckpointComplete(ExecutionAttemptID executionAttemptID,
JobID jobId,
long checkpointId,
long timestamp) |
CompletableFuture<Optional<org.apache.flink.runtime.messages.JobManagerMessages.ClassloadingProps>> |
JobManagerGateway.requestClassloadingProps(JobID jobId,
Time timeout)
Requests the class loading properties for the given JobID.
|
CompletableFuture<Acknowledge> |
JobManagerGateway.stopJob(JobID jobId,
Time timeout)
Stops the given job.
|
void |
RpcTaskManagerGateway.triggerCheckpoint(ExecutionAttemptID executionAttemptID,
JobID jobId,
long checkpointId,
long timestamp,
CheckpointOptions checkpointOptions) |
Constructor and Description |
---|
FlinkJobNotFoundException(JobID jobId) |
Modifier and Type | Method and Description |
---|---|
JobID |
AbstractCheckpointMessage.getJob() |
Constructor and Description |
---|
AbstractCheckpointMessage(JobID job,
ExecutionAttemptID taskExecutionId,
long checkpointId) |
AcknowledgeCheckpoint(JobID jobId,
ExecutionAttemptID taskExecutionId,
long checkpointId) |
AcknowledgeCheckpoint(JobID job,
ExecutionAttemptID taskExecutionId,
long checkpointId,
CheckpointMetrics checkpointMetrics,
TaskStateSnapshot subtaskState) |
DeclineCheckpoint(JobID job,
ExecutionAttemptID taskExecutionId,
long checkpointId) |
DeclineCheckpoint(JobID job,
ExecutionAttemptID taskExecutionId,
long checkpointId,
Throwable reason) |
NotifyCheckpointComplete(JobID job,
ExecutionAttemptID taskExecutionId,
long checkpointId,
long timestamp) |
TriggerCheckpoint(JobID job,
ExecutionAttemptID taskExecutionId,
long checkpointId,
long timestamp,
CheckpointOptions checkpointOptions) |
Modifier and Type | Method and Description |
---|---|
JobID |
JobDetails.getJobId() |
Modifier and Type | Method and Description |
---|---|
List<JobID> |
JobsWithIDsOverview.getJobsCancelled() |
List<JobID> |
JobsWithIDsOverview.getJobsFailed() |
List<JobID> |
JobsWithIDsOverview.getJobsFinished() |
List<JobID> |
JobsWithIDsOverview.getJobsRunningOrPending() |
Constructor and Description |
---|
JobDetails(JobID jobId,
String jobName,
long startTime,
long endTime,
long duration,
JobStatus status,
long lastUpdateTime,
int[] tasksPerState,
int numTasks) |
Constructor and Description |
---|
JobsWithIDsOverview(List<JobID> jobsRunningOrPending,
List<JobID> jobsFinished,
List<JobID> jobsCancelled,
List<JobID> jobsFailed) |
JobsWithIDsOverview(List<JobID> jobsRunningOrPending,
List<JobID> jobsFinished,
List<JobID> jobsCancelled,
List<JobID> jobsFailed) |
JobsWithIDsOverview(List<JobID> jobsRunningOrPending,
List<JobID> jobsFinished,
List<JobID> jobsCancelled,
List<JobID> jobsFailed) |
JobsWithIDsOverview(List<JobID> jobsRunningOrPending,
List<JobID> jobsFinished,
List<JobID> jobsCancelled,
List<JobID> jobsFailed) |
Modifier and Type | Field and Description |
---|---|
protected JobID |
JobMetricGroup.jobId
The ID of the job represented by this metrics group.
|
Modifier and Type | Method and Description |
---|---|
JobID |
JobMetricGroup.jobId() |
Modifier and Type | Method and Description |
---|---|
TaskMetricGroup |
TaskManagerMetricGroup.addTaskForJob(JobID jobId,
String jobName,
JobVertexID jobVertexId,
ExecutionAttemptID executionAttemptId,
String taskName,
int subtaskIndex,
int attemptNumber) |
void |
JobManagerMetricGroup.removeJob(JobID jobId) |
void |
TaskManagerMetricGroup.removeJobMetricsGroup(JobID jobId,
TaskManagerJobMetricGroup group) |
Constructor and Description |
---|
JobManagerJobMetricGroup(MetricRegistry registry,
JobManagerMetricGroup parent,
JobID jobId,
String jobName) |
JobMetricGroup(MetricRegistry registry,
C parent,
JobID jobId,
String jobName,
String[] scope) |
TaskManagerJobMetricGroup(MetricRegistry registry,
TaskManagerMetricGroup parent,
JobID jobId,
String jobName) |
Modifier and Type | Method and Description |
---|---|
String[] |
JobManagerJobScopeFormat.formatScope(JobManagerMetricGroup parent,
JobID jid,
String jobName) |
String[] |
TaskManagerJobScopeFormat.formatScope(TaskManagerMetricGroup parent,
JobID jid,
String jobName) |
Modifier and Type | Method and Description |
---|---|
JobID |
KvStateMessage.LookupKvStateLocation.getJobId()
Returns the JobID the KvState instance belongs to.
|
JobID |
KvStateMessage.NotifyKvStateRegistered.getJobId()
Returns the JobID the KvState instance belongs to.
|
JobID |
KvStateMessage.NotifyKvStateUnregistered.getJobId()
Returns the JobID the KvState instance belongs to.
|
JobID |
KvStateLocation.getJobId()
Returns the JobID the KvState instances belong to.
|
Modifier and Type | Method and Description |
---|---|
TaskKvStateRegistry |
KvStateRegistry.createTaskRegistry(JobID jobId,
JobVertexID jobVertexId)
Creates a
TaskKvStateRegistry facade for the Task
identified by the given JobID and JobVertexID instance. |
void |
KvStateRegistryListener.notifyKvStateRegistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName,
KvStateID kvStateId)
Notifies the listener about a registered KvState instance.
|
void |
KvStateRegistryGateway.notifyKvStateRegistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName,
KvStateID kvStateId,
InetSocketAddress kvStateServerAddress)
Notifies the listener about a registered KvState instance.
|
void |
KvStateRegistryListener.notifyKvStateUnregistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName)
Notifies the listener about an unregistered KvState instance.
|
void |
KvStateRegistryGateway.notifyKvStateUnregistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName)
Notifies the listener about an unregistered KvState instance.
|
KvStateID |
KvStateRegistry.registerKvState(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName,
InternalKvState<?> kvState)
Registers the KvState instance and returns the assigned ID.
|
void |
KvStateRegistry.unregisterKvState(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName,
KvStateID kvStateId)
Unregisters the KvState instance identified by the given KvStateID.
|
Constructor and Description |
---|
KvStateLocation(JobID jobId,
JobVertexID jobVertexId,
int numKeyGroups,
String registrationName)
Creates the location information.
|
KvStateLocationRegistry(JobID jobId,
Map<JobVertexID,ExecutionJobVertex> jobVertices)
Creates the registry for the job.
|
LookupKvStateLocation(JobID jobId,
String registrationName)
Requests a
KvStateLocation for the specified JobID and
InternalKvState registration name. |
NotifyKvStateRegistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName,
KvStateID kvStateId,
InetSocketAddress kvStateServerAddress)
Notifies the JobManager about a registered
InternalKvState instance. |
NotifyKvStateUnregistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName)
Notifies the JobManager about an unregistered
InternalKvState instance. |
Modifier and Type | Method and Description |
---|---|
JobID |
SlotRequest.getJobId()
Get the JobID of the slot requested for.
|
Modifier and Type | Method and Description |
---|---|
void |
JobLeaderIdService.addJob(JobID jobId)
Add a job to be monitored to retrieve the job leader id.
|
protected void |
ResourceManager.closeJobManagerConnection(JobID jobId,
Exception cause)
This method should be called by the framework once it detects that a currently registered
job manager has failed.
|
boolean |
JobLeaderIdService.containsJob(JobID jobId)
Check whether the given job is being monitored or not.
|
void |
ResourceManagerGateway.disconnectJobManager(JobID jobId,
Exception cause)
Disconnects a JobManager specified by the given resourceID from the
ResourceManager . |
void |
ResourceManager.disconnectJobManager(JobID jobId,
Exception cause) |
CompletableFuture<JobMasterId> |
JobLeaderIdService.getLeaderId(JobID jobId) |
boolean |
JobLeaderIdService.isValidTimeout(JobID jobId,
UUID timeoutId) |
void |
JobLeaderIdActions.jobLeaderLostLeadership(JobID jobId,
JobMasterId oldJobMasterId)
Callback when a monitored job leader lost its leadership.
|
protected void |
ResourceManager.jobLeaderLostLeadership(JobID jobId,
JobMasterId oldJobMasterId) |
void |
JobLeaderIdActions.notifyJobTimeout(JobID jobId,
UUID timeoutId)
Notify a job timeout.
|
CompletableFuture<RegistrationResponse> |
ResourceManagerGateway.registerJobManager(JobMasterId jobMasterId,
ResourceID jobMasterResourceId,
String jobMasterAddress,
JobID jobId,
Time timeout)
Register a
JobMaster at the resource manager. |
CompletableFuture<RegistrationResponse> |
ResourceManager.registerJobManager(JobMasterId jobMasterId,
ResourceID jobManagerResourceId,
String jobManagerAddress,
JobID jobId,
Time timeout) |
void |
JobLeaderIdService.removeJob(JobID jobId)
Remove the given job from being monitored by the service.
|
protected void |
ResourceManager.removeJob(JobID jobId) |
Constructor and Description |
---|
SlotRequest(JobID jobId,
AllocationID allocationId,
ResourceProfile resourceProfile,
String targetAddress) |
Modifier and Type | Method and Description |
---|---|
JobID |
JobManagerRegistration.getJobID() |
Constructor and Description |
---|
JobManagerRegistration(JobID jobID,
ResourceID jobManagerResourceID,
JobMasterGateway jobManagerGateway) |
Modifier and Type | Method and Description |
---|---|
JobID |
PendingSlotRequest.getJobId() |
Modifier and Type | Method and Description |
---|---|
void |
ResourceActions.notifyAllocationFailure(JobID jobId,
AllocationID allocationId,
Exception cause)
Notifies that an allocation failure has occurred.
|
Modifier and Type | Method and Description |
---|---|
CompletableFuture<AccessExecutionGraph> |
ExecutionGraphCache.getExecutionGraph(JobID jobId,
RestfulGateway restfulGateway)
Gets the
AccessExecutionGraph for the given JobID and caches it. |
Modifier and Type | Method and Description |
---|---|
protected JobID |
JobIDPathParameter.convertFromString(String value) |
JobID |
JobConfigInfo.getJobId() |
Modifier and Type | Method and Description |
---|---|
protected String |
JobIDPathParameter.convertToString(JobID value) |
Constructor and Description |
---|
JobConfigInfo(JobID jobId,
String jobName,
JobConfigInfo.ExecutionConfigInfo executionConfigInfo) |
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
StateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry)
Creates a new
AbstractKeyedStateBackend that is responsible for holding keyed state
and checkpointing it. |
abstract <K> AbstractKeyedStateBackend<K> |
AbstractStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry) |
CheckpointStreamFactory |
StateBackend.createSavepointStreamFactory(JobID jobId,
String operatorIdentifier,
String targetLocation)
Creates a
CheckpointStreamFactory that can be used to create streams
that should end up in a savepoint. |
abstract CheckpointStreamFactory |
AbstractStateBackend.createSavepointStreamFactory(JobID jobId,
String operatorIdentifier,
String targetLocation) |
CheckpointStreamFactory |
StateBackend.createStreamFactory(JobID jobId,
String operatorIdentifier)
Creates a
CheckpointStreamFactory that can be used to create streams
that should end up in a checkpoint. |
abstract CheckpointStreamFactory |
AbstractStateBackend.createStreamFactory(JobID jobId,
String operatorIdentifier) |
Modifier and Type | Method and Description |
---|---|
protected Path |
FsSavepointStreamFactory.createBasePath(FileSystem fs,
Path checkpointDirectory,
JobID jobID) |
protected Path |
FsCheckpointStreamFactory.createBasePath(FileSystem fs,
Path checkpointDirectory,
JobID jobID) |
<K> AbstractKeyedStateBackend<K> |
FsStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry) |
CheckpointStreamFactory |
FsStateBackend.createSavepointStreamFactory(JobID jobId,
String operatorIdentifier,
String targetLocation) |
CheckpointStreamFactory |
FsStateBackend.createStreamFactory(JobID jobId,
String operatorIdentifier) |
Constructor and Description |
---|
FsCheckpointStreamFactory(Path checkpointDataUri,
JobID jobId,
int fileStateSizeThreshold)
Creates a new state backend that stores its checkpoint data in the file system and location
defined by the given URI.
|
FsSavepointStreamFactory(Path checkpointDataUri,
JobID jobId,
int fileStateSizeThreshold) |
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
MemoryStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry) |
CheckpointStreamFactory |
MemoryStateBackend.createSavepointStreamFactory(JobID jobId,
String operatorIdentifier,
String targetLocation) |
CheckpointStreamFactory |
MemoryStateBackend.createStreamFactory(JobID jobId,
String operatorIdentifier) |
Modifier and Type | Method and Description |
---|---|
JobID |
JobManagerConnection.getJobID() |
JobID |
SlotStatus.getJobID()
Get the job id of the slot allocated for
|
Modifier and Type | Method and Description |
---|---|
void |
JobLeaderService.addJob(JobID jobId,
String defaultTargetAddress)
Add the given job to be monitored.
|
boolean |
JobManagerTable.contains(JobID jobId) |
boolean |
JobLeaderService.containsJob(JobID jobId)
Check whether the service monitors the given job.
|
void |
TaskExecutorGateway.disconnectJobManager(JobID jobId,
Exception cause)
Disconnects the given JobManager from the TaskManager.
|
void |
TaskExecutor.disconnectJobManager(JobID jobId,
Exception cause) |
JobManagerConnection |
JobManagerTable.get(JobID jobId) |
void |
JobLeaderListener.jobManagerGainedLeadership(JobID jobId,
JobMasterGateway jobManagerGateway,
JMTMRegistrationSuccess registrationMessage)
Callback if a job manager has gained leadership for the job identified by the job id and a
connection could be established to this job manager.
|
void |
JobLeaderListener.jobManagerLostLeadership(JobID jobId,
JobMasterId jobMasterId)
Callback if the job leader for the job with the given job id lost its leadership.
|
boolean |
JobManagerTable.put(JobID jobId,
JobManagerConnection jobManagerConnection) |
JobManagerConnection |
JobManagerTable.remove(JobID jobId) |
void |
JobLeaderService.removeJob(JobID jobId)
Remove the given job from being monitored by the job leader service.
|
CompletableFuture<Acknowledge> |
TaskExecutorGateway.requestSlot(SlotID slotId,
JobID jobId,
AllocationID allocationId,
String targetAddress,
ResourceManagerId resourceManagerId,
Time timeout)
Requests a slot from the TaskManager
|
CompletableFuture<Acknowledge> |
TaskExecutor.requestSlot(SlotID slotId,
JobID jobId,
AllocationID allocationId,
String targetAddress,
ResourceManagerId resourceManagerId,
Time timeout) |
Constructor and Description |
---|
JobManagerConnection(JobID jobID,
ResourceID resourceID,
JobMasterGateway jobMasterGateway,
TaskManagerActions taskManagerActions,
CheckpointResponder checkpointResponder,
BlobCacheService blobService,
LibraryCacheManager libraryCacheManager,
ResultPartitionConsumableNotifier resultPartitionConsumableNotifier,
PartitionProducerStateChecker partitionStateChecker) |
SlotStatus(SlotID slotID,
ResourceProfile resourceProfile,
JobID jobID,
AllocationID allocationID) |
Modifier and Type | Method and Description |
---|---|
void |
RpcCheckpointResponder.acknowledgeCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
CheckpointMetrics checkpointMetrics,
TaskStateSnapshot subtaskState) |
void |
RpcCheckpointResponder.declineCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
Throwable cause) |
void |
RpcKvStateRegistryListener.notifyKvStateRegistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName,
KvStateID kvStateId) |
void |
RpcKvStateRegistryListener.notifyKvStateUnregistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName) |
void |
RpcResultPartitionConsumableNotifier.notifyPartitionConsumable(JobID jobId,
ResultPartitionID partitionId,
TaskActions taskActions) |
CompletableFuture<ExecutionState> |
RpcPartitionStateChecker.requestPartitionProducerState(JobID jobId,
IntermediateDataSetID resultId,
ResultPartitionID partitionId) |
Modifier and Type | Method and Description |
---|---|
JobID |
TaskSlot.getJobId() |
Modifier and Type | Method and Description |
---|---|
boolean |
TaskSlot.allocate(JobID newJobId,
AllocationID newAllocationId)
Allocate the task slot for the given job and allocation id.
|
boolean |
TaskSlotTable.allocateSlot(int index,
JobID jobId,
AllocationID allocationId,
Time slotTimeout)
Allocate the slot with the given index for the given job and allocation id.
|
boolean |
TaskSlotTable.existsActiveSlot(JobID jobId,
AllocationID allocationId)
Check whether there exists an active slot for the given job and allocation id.
|
Iterator<AllocationID> |
TaskSlotTable.getActiveSlots(JobID jobId)
Return an iterator of active slots (their application ids) for the given job id.
|
Iterator<TaskSlot> |
TaskSlotTable.getAllocatedSlots(JobID jobId)
Return an iterator of allocated slots for the given job id.
|
Iterator<Task> |
TaskSlotTable.getTasks(JobID jobId)
Return an iterator over all tasks for a given job.
|
boolean |
TaskSlotTable.hasAllocatedSlots(JobID jobId)
Check whether the job has allocated (not active) slots.
|
boolean |
TaskSlot.isActive(JobID activeJobId,
AllocationID activeAllocationId) |
boolean |
TaskSlotTable.isAllocated(int index,
JobID jobId,
AllocationID allocationId)
Check whether the slot for the given index is allocated for the given job and allocation id.
|
boolean |
TaskSlot.isAllocated(JobID jobIdToCheck,
AllocationID allocationIDToCheck) |
Constructor and Description |
---|
SlotNotActiveException(JobID jobId,
AllocationID allocationId) |
Modifier and Type | Method and Description |
---|---|
JobID |
RuntimeEnvironment.getJobID() |
JobID |
TaskExecutionState.getJobID()
The ID of the job the task belongs to
|
JobID |
Task.getJobID() |
Modifier and Type | Method and Description |
---|---|
void |
CheckpointResponder.acknowledgeCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
CheckpointMetrics checkpointMetrics,
TaskStateSnapshot subtaskState)
Acknowledges the given checkpoint.
|
void |
ActorGatewayCheckpointResponder.acknowledgeCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
CheckpointMetrics checkpointMetrics,
TaskStateSnapshot checkpointStateHandles) |
void |
CheckpointResponder.declineCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
Throwable cause)
Declines the given checkpoint.
|
void |
ActorGatewayCheckpointResponder.declineCheckpoint(JobID jobID,
ExecutionAttemptID executionAttemptID,
long checkpointId,
Throwable reason) |
void |
ActorGatewayKvStateRegistryListener.notifyKvStateRegistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName,
KvStateID kvStateId) |
void |
ActorGatewayKvStateRegistryListener.notifyKvStateUnregistered(JobID jobId,
JobVertexID jobVertexId,
KeyGroupRange keyGroupRange,
String registrationName) |
void |
ActorGatewayResultPartitionConsumableNotifier.notifyPartitionConsumable(JobID jobId,
ResultPartitionID partitionId,
TaskActions taskActions) |
CompletableFuture<ExecutionState> |
ActorGatewayPartitionProducerStateChecker.requestPartitionProducerState(JobID jobId,
IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId) |
void |
TaskActions.triggerPartitionProducerStateCheck(JobID jobId,
IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId)
Check the execution state of the execution producing a result partition.
|
void |
Task.triggerPartitionProducerStateCheck(JobID jobId,
IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId) |
Constructor and Description |
---|
RuntimeEnvironment(JobID jobId,
JobVertexID jobVertexId,
ExecutionAttemptID executionId,
ExecutionConfig executionConfig,
TaskInfo taskInfo,
Configuration jobConfiguration,
Configuration taskConfiguration,
ClassLoader userCodeClassLoader,
MemoryManager memManager,
IOManager ioManager,
BroadcastVariableManager bcVarManager,
AccumulatorRegistry accumulatorRegistry,
TaskKvStateRegistry kvStateRegistry,
InputSplitProvider splitProvider,
Map<String,Future<Path>> distCacheEntries,
ResultPartitionWriter[] writers,
InputGate[] inputGates,
CheckpointResponder checkpointResponder,
TaskManagerRuntimeInfo taskManagerInfo,
TaskMetricGroup metrics,
Task containingTask) |
TaskExecutionState(JobID jobID,
ExecutionAttemptID executionId,
ExecutionState executionState)
Creates a new task execution state update, with no attached exception and no accumulators.
|
TaskExecutionState(JobID jobID,
ExecutionAttemptID executionId,
ExecutionState executionState,
Throwable error)
Creates a new task execution state update, with an attached exception but no accumulators.
|
TaskExecutionState(JobID jobID,
ExecutionAttemptID executionId,
ExecutionState executionState,
Throwable error,
AccumulatorSnapshot accumulators,
IOMetrics ioMetrics)
Creates a new task execution state update, with an attached exception.
|
TaskInputSplitProvider(ActorGateway jobManager,
JobID jobID,
JobVertexID vertexID,
ExecutionAttemptID executionID,
scala.concurrent.duration.FiniteDuration timeout) |
Modifier and Type | Method and Description |
---|---|
static ZooKeeperCheckpointIDCounter |
ZooKeeperUtils.createCheckpointIDCounter(org.apache.curator.framework.CuratorFramework client,
Configuration configuration,
JobID jobId)
Creates a
ZooKeeperCheckpointIDCounter instance. |
static CompletedCheckpointStore |
ZooKeeperUtils.createCompletedCheckpoints(org.apache.curator.framework.CuratorFramework client,
Configuration configuration,
JobID jobId,
int maxNumberOfCheckpointsToRetain,
Executor executor)
Creates a
ZooKeeperCompletedCheckpointStore instance. |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<AccessExecutionGraph> |
RestfulGateway.requestJob(JobID jobId,
Time timeout)
Requests the AccessExecutionGraph for the given jobId.
|
Modifier and Type | Method and Description |
---|---|
static String |
StreamIterationHead.createBrokerIdString(JobID jid,
String iterationID,
int subtaskIndex)
Creates the identification string with which head and tail task find the shared blocking
queue for the back channel.
|
Modifier and Type | Method and Description |
---|---|
LeaderElectionService |
YarnPreConfiguredMasterNonHaServices.getJobManagerLeaderElectionService(JobID jobID) |
LeaderElectionService |
YarnIntraNonHaMasterServices.getJobManagerLeaderElectionService(JobID jobID) |
LeaderRetrievalService |
YarnPreConfiguredMasterNonHaServices.getJobManagerLeaderRetriever(JobID jobID) |
LeaderRetrievalService |
YarnIntraNonHaMasterServices.getJobManagerLeaderRetriever(JobID jobID) |
LeaderRetrievalService |
YarnPreConfiguredMasterNonHaServices.getJobManagerLeaderRetriever(JobID jobID,
String defaultJobManagerAddress) |
LeaderRetrievalService |
YarnIntraNonHaMasterServices.getJobManagerLeaderRetriever(JobID jobID,
String defaultJobManagerAddress) |
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.