public abstract class AbstractBlobCache extends Object implements Closeable
Modifier and Type | Field and Description |
---|---|
protected Configuration |
blobClientConfig
Configuration for the blob client like ssl parameters required to connect to the blob
server.
|
protected BlobView |
blobView
Blob store for distributed file storage, e.g.
|
protected org.slf4j.Logger |
log
The log object used for debugging.
|
protected int |
numFetchRetries
The number of retries when the transfer fails.
|
protected ReadWriteLock |
readWriteLock
Lock guarding concurrent file accesses.
|
protected InetSocketAddress |
serverAddress |
protected Thread |
shutdownHook
Shutdown hook thread to ensure deletion of the local storage directory.
|
protected AtomicBoolean |
shutdownRequested |
protected File |
storageDir
Root directory for local file storage.
|
protected AtomicLong |
tempFileCounter
Counter to generate unique names for temporary files.
|
Constructor and Description |
---|
AbstractBlobCache(InetSocketAddress serverAddress,
Configuration blobClientConfig,
BlobView blobView,
org.slf4j.Logger logger) |
Modifier and Type | Method and Description |
---|---|
protected abstract void |
cancelCleanupTask()
Cancels any cleanup task that subclasses may be executing.
|
void |
close() |
protected File |
getFileInternal(JobID jobId,
org.apache.flink.runtime.blob.BlobKey blobKey)
Returns local copy of the file for the BLOB with the given key.
|
int |
getPort()
Returns the port the BLOB server is listening on.
|
protected final org.slf4j.Logger log
protected final AtomicLong tempFileCounter
protected final InetSocketAddress serverAddress
protected final File storageDir
protected final BlobView blobView
protected final AtomicBoolean shutdownRequested
protected final Thread shutdownHook
protected final int numFetchRetries
protected final Configuration blobClientConfig
protected final ReadWriteLock readWriteLock
public AbstractBlobCache(InetSocketAddress serverAddress, Configuration blobClientConfig, BlobView blobView, org.slf4j.Logger logger) throws IOException
IOException
protected File getFileInternal(@Nullable JobID jobId, org.apache.flink.runtime.blob.BlobKey blobKey) throws IOException
The method will first attempt to serve the BLOB from its local cache. If the BLOB is not in the cache, the method will try to download it from this cache's BLOB server via a distributed BLOB store (if available) or direct end-to-end download.
jobId
- ID of the job this blob belongs to (or null if job-unrelated)blobKey
- The key of the desired BLOB.IOException
- Thrown if an I/O error occurs while downloading the BLOBs from the BLOB server.public int getPort()
public void close() throws IOException
close
in interface Closeable
close
in interface AutoCloseable
IOException
protected abstract void cancelCleanupTask()
This is called during close()
.
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.