flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: Flume Message Headers?
Date Sat, 19 Sep 2015 16:05:26 GMT
Try creating and adding a simple debug interceptor which can emit
information you need and have controls to enable/disable via JMX.

On Sat, Sep 19, 2015 at 4:49 AM, Doug McClure <dmcclure@gmail.com> wrote:
> I was hoping this would, but I see only the contents being passed and not
> the message headers.  I quickly tested using the console producer to console
> consumer and get only what I enter via stdin.  Is there some way to put this
> into a debug level to see more?
>
> On Fri, Sep 18, 2015 at 10:28 PM, Sharninder <sharninder@gmail.com> wrote:
>>
>> Use the Kafka console utility to see what's being fed to Kafka.
>>
>>
>> On Saturday, September 19, 2015, Doug McClure <dmcclure@gmail.com> wrote:
>>>
>>> Ideally as they appear passing through broker in topic and at the input
>>> to another agent source. This is where I need to parse them and make routing
>>> decisions.
>>>
>>> Doug
>>>
>>> ---
>>> Doug McClure
>>> e: dmcclure@gmail.com
>>> c: +1-678-561-2288
>>>
>>> On Sep 18, 2015, at 8:58 PM, Hari Shreedharan <hshreedharan@cloudera.com>
>>> wrote:
>>>
>>> Where do you want to see the headers? After the event lands on HDFS? Use
>>> the Header and body serializer if you using text, avro serializer writes out
>>> the headers.
>>>
>>> Thanks,
>>> Hari
>>>
>>>
>>> On Fri, Sep 18, 2015 at 3:51 PM, Doug McClure <dmcclure@gmail.com> wrote:
>>>>
>>>> Is there a debug level or other Flume 1.6 utility I can use to see the
>>>> full message headers?  I need to see what's being collected in the spooldir
>>>> source and sent via kafka sink, specifically the values of fileHeader and
>>>> basenameHeader data.
>>>>
>>>> Tks!
>>>>
>>>> Doug
>>>
>>>
>>
>>
>> --
>> --
>> Sharninder
>>
>>
>>
>



-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

Mime
View raw message