@Internal public class SafetyNetWrapperFileSystem extends FileSystem implements WrappingProxy<FileSystem>
WrappingProxy
around FileSystem
which (i) wraps all opened streams as
ClosingFSDataInputStream
or ClosingFSDataOutputStream
and (ii) registers them to
a SafetyNetCloseableRegistry
.
Streams obtained by this are therefore managed by the SafetyNetCloseableRegistry
to prevent resource leaks
from unclosed streams.FileSystem.FSKey, FileSystem.WriteMode
Constructor and Description |
---|
SafetyNetWrapperFileSystem(FileSystem unsafeFileSystem,
SafetyNetCloseableRegistry registry) |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
create(Path f,
boolean overwrite)
Opens an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize)
Opens an FSDataOutputStream at the indicated Path.
|
boolean |
delete(Path f,
boolean recursive)
Delete a file.
|
boolean |
exists(Path f)
Check if exists.
|
long |
getDefaultBlockSize()
Return the number of bytes that large input files should be optimally be split into to minimize I/O time.
|
BlockLocation[] |
getFileBlockLocations(FileStatus file,
long start,
long len)
Return an array containing hostnames, offset and size of
portions of the given file.
|
FileStatus |
getFileStatus(Path f)
Return a file status object that represents the path.
|
Path |
getHomeDirectory()
Returns the path of the user's home directory in this file system.
|
URI |
getUri()
Returns a URI whose scheme and authority identify this file system.
|
Path |
getWorkingDirectory()
Returns the path of the file system's current working directory.
|
FileSystem |
getWrappedDelegate() |
void |
initialize(URI name)
Called after a new FileSystem instance is constructed.
|
boolean |
initOutPathDistFS(Path outPath,
FileSystem.WriteMode writeMode,
boolean createDirectory)
Initializes output directories on distributed file systems according to the given write mode.
|
boolean |
initOutPathLocalFS(Path outPath,
FileSystem.WriteMode writeMode,
boolean createDirectory)
Initializes output directories on local file systems according to the given write mode.
|
boolean |
isDistributedFS()
Returns true if this is a distributed file system, false otherwise.
|
FileStatus[] |
listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
boolean |
mkdirs(Path f)
Make the given file and all non-existent parents into directories.
|
FSDataInputStream |
open(Path f)
Opens an FSDataInputStream at the indicated Path.
|
FSDataInputStream |
open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
boolean |
rename(Path src,
Path dst)
Renames the file/directory src to dst.
|
closeAndDisposeFileSystemCloseableRegistryForThread, createAndSetFileSystemCloseableRegistryForThread, get, getLocalFileSystem, getUnguardedFileSystem, isFlinkSupportedScheme, setDefaultScheme
public SafetyNetWrapperFileSystem(FileSystem unsafeFileSystem, SafetyNetCloseableRegistry registry)
public Path getWorkingDirectory()
FileSystem
getWorkingDirectory
in class FileSystem
public Path getHomeDirectory()
FileSystem
getHomeDirectory
in class FileSystem
public URI getUri()
FileSystem
getUri
in class FileSystem
public void initialize(URI name) throws IOException
FileSystem
initialize
in class FileSystem
name
- a URI
whose authority section names the host, port, etc. for this file systemIOException
public FileStatus getFileStatus(Path f) throws IOException
FileSystem
getFileStatus
in class FileSystem
f
- The path we want information fromFileNotFoundException
- when the path does not exist;
IOException see specific implementationIOException
public BlockLocation[] getFileBlockLocations(FileStatus file, long start, long len) throws IOException
FileSystem
getFileBlockLocations
in class FileSystem
IOException
public FSDataInputStream open(Path f, int bufferSize) throws IOException
FileSystem
open
in class FileSystem
f
- the file name to openbufferSize
- the size of the buffer to be used.IOException
public FSDataInputStream open(Path f) throws IOException
FileSystem
open
in class FileSystem
f
- the file to openIOException
public long getDefaultBlockSize()
FileSystem
getDefaultBlockSize
in class FileSystem
public FileStatus[] listStatus(Path f) throws IOException
FileSystem
listStatus
in class FileSystem
f
- given pathIOException
public boolean exists(Path f) throws IOException
FileSystem
exists
in class FileSystem
f
- source fileIOException
public boolean delete(Path f, boolean recursive) throws IOException
FileSystem
delete
in class FileSystem
f
- the path to deleterecursive
- if path is a directory and set to true
, the directory is deleted else throws an exception. In
case of a file the recursive can be set to either true
or false
true
if delete is successful, false
otherwiseIOException
public boolean mkdirs(Path f) throws IOException
FileSystem
mkdirs
in class FileSystem
f
- the directory/directories to be createdtrue
if at least one new directory has been created, false
otherwiseIOException
- thrown if an I/O error occurs while creating the directorypublic FSDataOutputStream create(Path f, boolean overwrite, int bufferSize, short replication, long blockSize) throws IOException
FileSystem
create
in class FileSystem
f
- the file name to openoverwrite
- if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.bufferSize
- the size of the buffer to be used.replication
- required block replication for the file.blockSize
- the size of the file blocksIOException
public FSDataOutputStream create(Path f, boolean overwrite) throws IOException
FileSystem
create
in class FileSystem
f
- the file name to openoverwrite
- if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.IOException
public boolean rename(Path src, Path dst) throws IOException
FileSystem
rename
in class FileSystem
src
- the file/directory to renamedst
- the new name of the file/directorytrue
if the renaming was successful, false
otherwiseIOException
public boolean initOutPathLocalFS(Path outPath, FileSystem.WriteMode writeMode, boolean createDirectory) throws IOException
FileSystem
Files contained in an existing directory are not deleted, because multiple instances of a DataSinkTask might call this function at the same time and hence might perform concurrent delete operations on the file system (possibly deleting output files of concurrently running tasks). Since concurrent DataSinkTasks are not aware of each other, coordination of delete and create operations would be difficult.
initOutPathLocalFS
in class FileSystem
outPath
- Output path that should be prepared.writeMode
- Write mode to consider.createDirectory
- True, to initialize a directory at the given path, false to prepare space for a file.IOException
- Thrown, if any of the file system access operations failed.public boolean initOutPathDistFS(Path outPath, FileSystem.WriteMode writeMode, boolean createDirectory) throws IOException
FileSystem
initOutPathDistFS
in class FileSystem
outPath
- Output path that should be prepared.writeMode
- Write mode to consider.createDirectory
- True, to initialize a directory at the given path, false otherwise.IOException
- Thrown, if any of the file system access operations failed.public boolean isDistributedFS()
FileSystem
isDistributedFS
in class FileSystem
public FileSystem getWrappedDelegate()
getWrappedDelegate
in interface WrappingProxy<FileSystem>
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.