public class WithinCheckpointFileMergingSnapshotManager extends FileMergingSnapshotManagerBase
FileMergingSnapshotManager
that merging files within a checkpoint.FileMergingSnapshotManager.SpaceStat, FileMergingSnapshotManager.SubtaskKey
checkpointDir, filePoolType, fs, ioExecutor, lock, managedExclusiveStateDir, managedExclusiveStateDirHandle, maxPhysicalFileSize, physicalFileDeleter, sharedStateDir, shouldSyncAfterClosingLogicalFile, spaceStat, taskOwnedStateDir, uploadedStates, writeBufferSize
Constructor and Description |
---|
WithinCheckpointFileMergingSnapshotManager(String id,
long maxFileSize,
PhysicalFilePool.Type filePoolType,
Executor ioExecutor) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
protected void |
discardCheckpoint(long checkpointId)
The callback which will be triggered when all subtasks discarded (aborted or subsumed).
|
protected PhysicalFile |
getOrCreatePhysicalFileForCheckpoint(FileMergingSnapshotManager.SubtaskKey subtaskKey,
long checkpointId,
CheckpointedStateScope scope)
Get a reused physical file or create one.
|
void |
notifyCheckpointAborted(FileMergingSnapshotManager.SubtaskKey subtaskKey,
long checkpointId)
This method is called as a notification once a distributed checkpoint has been aborted.
|
void |
notifyCheckpointComplete(FileMergingSnapshotManager.SubtaskKey subtaskKey,
long checkpointId)
Notifies the manager that the checkpoint with the given
checkpointId completed and
was committed. |
protected void |
returnPhysicalFileForNextReuse(FileMergingSnapshotManager.SubtaskKey subtaskKey,
long checkpointId,
PhysicalFile physicalFile)
Try to return an existing physical file to the manager for next reuse.
|
createCheckpointStateOutputStream, createLogicalFile, createPhysicalFile, createPhysicalPool, deletePhysicalFile, discardSingleLogicalFile, generatePhysicalFilePath, getLogicalFile, getManagedDir, getManagedDirStateHandle, initFileSystem, notifyCheckpointSubsumed, registerSubtaskForSharedStates, restoreStateHandles, reusePreviousStateHandle
public WithinCheckpointFileMergingSnapshotManager(String id, long maxFileSize, PhysicalFilePool.Type filePoolType, Executor ioExecutor)
public void notifyCheckpointComplete(FileMergingSnapshotManager.SubtaskKey subtaskKey, long checkpointId) throws Exception
FileMergingSnapshotManager
checkpointId
completed and
was committed.notifyCheckpointComplete
in interface FileMergingSnapshotManager
notifyCheckpointComplete
in class FileMergingSnapshotManagerBase
subtaskKey
- the subtask key identifying the subtask.checkpointId
- The ID of the checkpoint that has been completed.Exception
- thrown if anything goes wrong with the listener.public void notifyCheckpointAborted(FileMergingSnapshotManager.SubtaskKey subtaskKey, long checkpointId) throws Exception
FileMergingSnapshotManager
notifyCheckpointAborted
in interface FileMergingSnapshotManager
notifyCheckpointAborted
in class FileMergingSnapshotManagerBase
subtaskKey
- the subtask key identifying the subtask.checkpointId
- The ID of the checkpoint that has been completed.Exception
- thrown if anything goes wrong with the listener.@Nonnull protected PhysicalFile getOrCreatePhysicalFileForCheckpoint(FileMergingSnapshotManager.SubtaskKey subtaskKey, long checkpointId, CheckpointedStateScope scope) throws IOException
FileMergingSnapshotManagerBase
Basic logic of file reusing: whenever a physical file is needed, this method is called
with necessary information provided for acquiring a file. The file will not be reused until
it is written and returned to the reused pool by calling FileMergingSnapshotManagerBase.returnPhysicalFileForNextReuse(org.apache.flink.runtime.checkpoint.filemerging.FileMergingSnapshotManager.SubtaskKey, long, org.apache.flink.runtime.checkpoint.filemerging.PhysicalFile)
.
getOrCreatePhysicalFileForCheckpoint
in class FileMergingSnapshotManagerBase
subtaskKey
- the subtask key for the callercheckpointId
- the checkpoint idscope
- checkpoint scopeIOException
- thrown if anything goes wrong with file system.protected void returnPhysicalFileForNextReuse(FileMergingSnapshotManager.SubtaskKey subtaskKey, long checkpointId, PhysicalFile physicalFile) throws IOException
FileMergingSnapshotManagerBase
Basic logic of file reusing, see FileMergingSnapshotManagerBase.getOrCreatePhysicalFileForCheckpoint(org.apache.flink.runtime.checkpoint.filemerging.FileMergingSnapshotManager.SubtaskKey, long, org.apache.flink.runtime.state.CheckpointedStateScope)
.
returnPhysicalFileForNextReuse
in class FileMergingSnapshotManagerBase
subtaskKey
- the subtask key for the callercheckpointId
- in which checkpoint this physical file is requested.physicalFile
- the returning checkpointIOException
- thrown if anything goes wrong with file system.#getOrCreatePhysicalFileForCheckpoint(SubtaskKey, long, CheckpointedStateScope)
protected void discardCheckpoint(long checkpointId) throws IOException
FileMergingSnapshotManagerBase
discardCheckpoint
in class FileMergingSnapshotManagerBase
checkpointId
- the discarded checkpoint id.IOException
- if anything goes wrong with file system.public void close() throws IOException
close
in interface Closeable
close
in interface AutoCloseable
close
in class FileMergingSnapshotManagerBase
IOException
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.