public class DateTimeBucketer<T> extends Object implements Bucketer<T>
Bucketer
that assigns to buckets based on current system time.
The DateTimeBucketer
will create directories of the following form:
/{basePath}/{dateTimePath}/
. The basePath
is the path
that was specified as a base path when creating the
BucketingSink
. The dateTimePath
is determined based on the current system time and the user provided format string.
SimpleDateFormat
is used to derive a date string from the current system time and
the date format string. The default format string is "yyyy-MM-dd--HH"
so the rolling
files will have a granularity of hours.
Example:
Bucketer buck = new DateTimeBucketer("yyyy-MM-dd--HH");
This will create for example the following bucket path:
/base/1976-12-31-14/
Constructor and Description |
---|
DateTimeBucketer()
Creates a new
DateTimeBucketer with format string "yyyy-MM-dd--HH" . |
DateTimeBucketer(String formatString)
Creates a new
DateTimeBucketer with the given date/time format string. |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.fs.Path |
getBucketPath(Clock clock,
org.apache.hadoop.fs.Path basePath,
T element)
Returns the
Path of a bucket file. |
String |
toString() |
public DateTimeBucketer()
DateTimeBucketer
with format string "yyyy-MM-dd--HH"
.public DateTimeBucketer(String formatString)
DateTimeBucketer
with the given date/time format string.formatString
- The format string that will be given to SimpleDateFormat
to determine
the bucket path.public org.apache.hadoop.fs.Path getBucketPath(Clock clock, org.apache.hadoop.fs.Path basePath, T element)
Bucketer
Path
of a bucket file.getBucketPath
in interface Bucketer<T>
basePath
- The base path containing all the buckets.element
- The current element being processed.Path
of the bucket which the provided element should fall in. This
should include the basePath
and also the subtaskIndex
to avoid clashes with
parallel sinks.Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.