flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shuang <shu...@open42.com>
Subject Use metadata field for output bucketing
Date Fri, 04 Nov 2011 22:17:50 GMT
Hi, guys,
  After reading the Flume User Guide, I thought this is possible, but would
like to confirm with your guys. Currently, I have collectors configured as:
collectorSink("s3n://mybucket/flume/%Y-%m-%d/\",\"log-%{host}-\", 3600000),
and I have a field called "source" in Flume event's metadata table, and
would like to use in the collectorSink path, something like this:
collectorSink("s3n://mybucket/flume/%{metadata['source']}/%Y-%m-%d/\",\"log-%{host}-\",
3600000),

I wonder what's the right syntax to refer to field in the metadata table. I
search the user guide and couldn't find any example. Also I'd like to point
out, this is kind of similar to how Scribe's message category is used.

Shuang

Mime
View raw message