K
- The type of the first tuple field.V
- The type of the second tuple field.public class SequenceFileWriter<K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> extends Object implements Writer<Tuple2<K,V>>, InputTypeConfigurable
Writer
that writes the bucket files as Hadoop SequenceFiles
.
The input to the RollingSink
must
be a Tuple2
of two Hadopo
Writables
.Constructor and Description |
---|
SequenceFileWriter()
Creates a new
SequenceFileWriter that writes sequence files without compression. |
SequenceFileWriter(String compressionCodecName,
org.apache.hadoop.io.SequenceFile.CompressionType compressionType)
Creates a new
SequenceFileWriter that writes sequence with the given
compression codec and compression type. |
Modifier and Type | Method and Description |
---|---|
void |
close()
Closes the
Writer . |
Writer<Tuple2<K,V>> |
duplicate()
Duplicates the
Writer . |
void |
flush()
Flushes out any internally held data.
|
void |
open(org.apache.hadoop.fs.FSDataOutputStream outStream)
Initializes the
Writer for a newly opened bucket file. |
void |
setInputType(TypeInformation<?> type,
ExecutionConfig executionConfig)
Method that is called on an
OutputFormat when it is passed to
the DataSet's output method. |
void |
write(Tuple2<K,V> element)
Writes one element to the bucket file.
|
public SequenceFileWriter()
SequenceFileWriter
that writes sequence files without compression.public SequenceFileWriter(String compressionCodecName, org.apache.hadoop.io.SequenceFile.CompressionType compressionType)
SequenceFileWriter
that writes sequence with the given
compression codec and compression type.compressionCodecName
- Name of a Hadoop Compression Codec.compressionType
- The compression type to use.public void open(org.apache.hadoop.fs.FSDataOutputStream outStream) throws IOException
Writer
Writer
for a newly opened bucket file.
Any internal per-bucket initialization should be performed here.public void flush() throws IOException
Writer
public void close() throws IOException
Writer
Writer
. This must not close the FSDataOutputStream
that
was handed in in the Writer.open(org.apache.hadoop.fs.FSDataOutputStream)
method. Only internally held state should be
closed.public void write(Tuple2<K,V> element) throws IOException
Writer
public void setInputType(TypeInformation<?> type, ExecutionConfig executionConfig)
InputTypeConfigurable
OutputFormat
when it is passed to
the DataSet's output method. May be used to configures the output format based on the data type.setInputType
in interface InputTypeConfigurable
type
- The data type of the input.Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.