public class AcrossCheckpointFileMergingSnapshotManager extends FileMergingSnapshotManagerBase
FileMergingSnapshotManager
that merging files across checkpoints.FileMergingSnapshotManager.SpaceStat, FileMergingSnapshotManager.SubtaskKey
checkpointDir, filePoolType, fs, ioExecutor, lock, managedExclusiveStateDir, managedExclusiveStateDirHandle, maxPhysicalFileSize, physicalFileDeleter, sharedStateDir, shouldSyncAfterClosingLogicalFile, spaceStat, taskOwnedStateDir, uploadedStates, writeBufferSize
Constructor and Description |
---|
AcrossCheckpointFileMergingSnapshotManager(String id,
long maxFileSize,
PhysicalFilePool.Type filePoolType,
Executor ioExecutor) |
Modifier and Type | Method and Description |
---|---|
protected void |
discardCheckpoint(long checkpointId)
The callback which will be triggered when all subtasks discarded (aborted or subsumed).
|
protected PhysicalFile |
getOrCreatePhysicalFileForCheckpoint(FileMergingSnapshotManager.SubtaskKey subtaskKey,
long checkpointID,
CheckpointedStateScope scope)
Get a reused physical file or create one.
|
protected void |
returnPhysicalFileForNextReuse(FileMergingSnapshotManager.SubtaskKey subtaskKey,
long checkpointId,
PhysicalFile physicalFile)
Try to return an existing physical file to the manager for next reuse.
|
close, createCheckpointStateOutputStream, createLogicalFile, createPhysicalFile, createPhysicalPool, deletePhysicalFile, discardSingleLogicalFile, generatePhysicalFilePath, getLogicalFile, getManagedDir, getManagedDirStateHandle, initFileSystem, notifyCheckpointAborted, notifyCheckpointComplete, notifyCheckpointSubsumed, registerSubtaskForSharedStates, restoreStateHandles, reusePreviousStateHandle
public AcrossCheckpointFileMergingSnapshotManager(String id, long maxFileSize, PhysicalFilePool.Type filePoolType, Executor ioExecutor)
@Nonnull protected PhysicalFile getOrCreatePhysicalFileForCheckpoint(FileMergingSnapshotManager.SubtaskKey subtaskKey, long checkpointID, CheckpointedStateScope scope) throws IOException
FileMergingSnapshotManagerBase
Basic logic of file reusing: whenever a physical file is needed, this method is called
with necessary information provided for acquiring a file. The file will not be reused until
it is written and returned to the reused pool by calling FileMergingSnapshotManagerBase.returnPhysicalFileForNextReuse(org.apache.flink.runtime.checkpoint.filemerging.FileMergingSnapshotManager.SubtaskKey, long, org.apache.flink.runtime.checkpoint.filemerging.PhysicalFile)
.
getOrCreatePhysicalFileForCheckpoint
in class FileMergingSnapshotManagerBase
subtaskKey
- the subtask key for the callercheckpointID
- the checkpoint idscope
- checkpoint scopeIOException
- thrown if anything goes wrong with file system.protected void discardCheckpoint(long checkpointId)
FileMergingSnapshotManagerBase
discardCheckpoint
in class FileMergingSnapshotManagerBase
checkpointId
- the discarded checkpoint id.protected void returnPhysicalFileForNextReuse(FileMergingSnapshotManager.SubtaskKey subtaskKey, long checkpointId, PhysicalFile physicalFile) throws IOException
FileMergingSnapshotManagerBase
Basic logic of file reusing, see FileMergingSnapshotManagerBase.getOrCreatePhysicalFileForCheckpoint(org.apache.flink.runtime.checkpoint.filemerging.FileMergingSnapshotManager.SubtaskKey, long, org.apache.flink.runtime.state.CheckpointedStateScope)
.
returnPhysicalFileForNextReuse
in class FileMergingSnapshotManagerBase
subtaskKey
- the subtask key for the callercheckpointId
- in which checkpoint this physical file is requested.physicalFile
- the returning checkpointIOException
- thrown if anything goes wrong with file system.#getOrCreatePhysicalFileForCheckpoint(SubtaskKey, long, CheckpointedStateScope)
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.