public class FsNegativeRunningJobsRegistry extends Object implements RunningJobsRegistry
RunningJobsRegistry
tracks the status jobs via marker files,
marking running jobs via running marker files, marking finished jobs via finished marker files.
The general contract is the following:
This registry is especially tailored towards deployment modes like for example YARN, where HDFS is available as a persistent file system, and the YARN application's working directories on HDFS are automatically cleaned up after the application completed.
RunningJobsRegistry.JobSchedulingStatus
Constructor and Description |
---|
FsNegativeRunningJobsRegistry(FileSystem fileSystem,
Path workingDirectory)
Creates a new registry that writes its files to the given FileSystem at
the given working directory path.
|
FsNegativeRunningJobsRegistry(Path workingDirectory)
Creates a new registry that writes to the FileSystem and working directory
denoted by the given path.
|
Modifier and Type | Method and Description |
---|---|
void |
clearJob(JobID jobID)
Clear job state form the registry, usually called after job finish.
|
RunningJobsRegistry.JobSchedulingStatus |
getJobSchedulingStatus(JobID jobID)
Gets the scheduling status of a job.
|
void |
setJobFinished(JobID jobID)
Marks a job as completed.
|
void |
setJobRunning(JobID jobID)
Marks a job as running.
|
public FsNegativeRunningJobsRegistry(Path workingDirectory) throws IOException
The initialization will attempt to write to the given working directory, in order to catch setup/configuration errors early.
workingDirectory
- The working directory for files to track the job status.IOException
- Thrown, if the specified directory cannot be accessed.public FsNegativeRunningJobsRegistry(FileSystem fileSystem, Path workingDirectory) throws IOException
The initialization will attempt to write to the given working directory, in order to catch setup/configuration errors early.
fileSystem
- The FileSystem to use for the marker files.workingDirectory
- The working directory for files to track the job status.IOException
- Thrown, if the specified directory cannot be accessed.public void setJobRunning(JobID jobID) throws IOException
RunningJobsRegistry
RunningJobsRegistry.getJobSchedulingStatus(JobID)
method will return RunningJobsRegistry.JobSchedulingStatus.RUNNING
.setJobRunning
in interface RunningJobsRegistry
jobID
- The id of the job.IOException
- Thrown when the communication with the highly-available storage or registry
failed and could not be retried.public void setJobFinished(JobID jobID) throws IOException
RunningJobsRegistry
RunningJobsRegistry.getJobSchedulingStatus(JobID)
method will return RunningJobsRegistry.JobSchedulingStatus.DONE
.setJobFinished
in interface RunningJobsRegistry
jobID
- The id of the job.IOException
- Thrown when the communication with the highly-available storage or registry
failed and could not be retried.public RunningJobsRegistry.JobSchedulingStatus getJobSchedulingStatus(JobID jobID) throws IOException
RunningJobsRegistry
getJobSchedulingStatus
in interface RunningJobsRegistry
jobID
- The id of the job to check.IOException
- Thrown when the communication with the highly-available storage or registry
failed and could not be retried.public void clearJob(JobID jobID) throws IOException
RunningJobsRegistry
clearJob
in interface RunningJobsRegistry
jobID
- The id of the job to check.IOException
- Thrown when the communication with the highly-available storage or registry
failed and could not be retried.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.