public class DateTimeBucketer<T> extends Object implements Bucketer<T>
Bucketer
that assigns to buckets based on current system time.
The DateTimeBucketer
will create directories of the following form:
/{basePath}/{dateTimePath}/
. The basePath
is the path
that was specified as a base path when creating the
BucketingSink
. The dateTimePath
is determined based on the current system time and the user provided format string.
DateTimeFormatter
is used to derive a date string from the current system time and
the date format string with a timezone. The default format string is "yyyy-MM-dd--HH"
so the rolling
files will have a granularity of hours.
Example:
Bucketer buck = new DateTimeBucketer("yyyy-MM-dd--HH");
This will create for example the following bucket path:
/base/1976-12-31-14/
Constructor and Description |
---|
DateTimeBucketer()
Creates a new
DateTimeBucketer with format string "yyyy-MM-dd--HH" using JVM's default timezone. |
DateTimeBucketer(String formatString)
Creates a new
DateTimeBucketer with the given date/time format string using JVM's default timezone. |
DateTimeBucketer(String formatString,
java.time.ZoneId zoneId)
Creates a new
DateTimeBucketer with the given date/time format string using the given timezone. |
DateTimeBucketer(java.time.ZoneId zoneId)
Creates a new
DateTimeBucketer with format string "yyyy-MM-dd--HH" using the given timezone. |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.fs.Path |
getBucketPath(Clock clock,
org.apache.hadoop.fs.Path basePath,
T element)
Returns the
Path of a bucket file. |
String |
toString() |
public DateTimeBucketer()
DateTimeBucketer
with format string "yyyy-MM-dd--HH"
using JVM's default timezone.public DateTimeBucketer(String formatString)
DateTimeBucketer
with the given date/time format string using JVM's default timezone.formatString
- The format string that will be given to DateTimeFormatter
to determine
the bucket path.public DateTimeBucketer(java.time.ZoneId zoneId)
DateTimeBucketer
with format string "yyyy-MM-dd--HH"
using the given timezone.zoneId
- The timezone used to format DateTimeFormatter
for bucket path.public DateTimeBucketer(String formatString, java.time.ZoneId zoneId)
DateTimeBucketer
with the given date/time format string using the given timezone.formatString
- The format string that will be given to DateTimeFormatter
to determine
the bucket path.zoneId
- The timezone used to format DateTimeFormatter
for bucket path.public org.apache.hadoop.fs.Path getBucketPath(Clock clock, org.apache.hadoop.fs.Path basePath, T element)
Bucketer
Path
of a bucket file.getBucketPath
in interface Bucketer<T>
basePath
- The base path containing all the buckets.element
- The current element being processed.Path
of the bucket which the provided element should fall in. This
should include the basePath
and also the subtaskIndex
to avoid clashes with
parallel sinks.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.