flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharninder <sharnin...@gmail.com>
Subject Re: Flume Syslog source
Date Thu, 16 Oct 2014 06:42:18 GMT
I just looked at the existing syslogtcp source and it seems it does take
pains to parse the hostname from the message and I think that is the best
bet for me. Ofcourse, it might fail for a few devices, but I'll just have
to think of something else for those.


On Thu, Oct 16, 2014 at 10:40 AM, Sharninder <sharninder@gmail.com> wrote:

> Yes Jeff. That's a possiblity but I'm not sure (actually pretty sure) that
> there would be a some random device which will not send their logs in the
> proper format and my regex will break. This is the way I'll implement it if
> I can't find anything better.
> Thanks,
> Sharninder
> On Thu, Oct 16, 2014 at 10:22 AM, Jeff Lord <jlord@cloudera.com> wrote:
>> You can also use a regex interceptor to extract hostname from the message
>> (assuming it's there) and put that in an event header. From there you can
>> route and create partitions with the header.
>> On Wednesday, October 15, 2014, Hari Shreedharan <
>> hshreedharan@cloudera.com> wrote:
>>> The Multiport syslog source can add the port number on which the data
>>> was received to the event headers. You can use with a multiplexing channel
>>> selector to separate this to different channels.
>>> Thanks,
>>> Hari
>>> On Wed, Oct 15, 2014 at 9:45 PM, Sharninder <sharninder@gmail.com>
>>> wrote:
>>>> Hi Guys,
>>>> I'm trying to implement a system to archive syslogs using flume. I've
>>>> played around with it a bit but haven't really been able to figure out a
>>>> way to segregate logs according to the host they're coming from? Is there
>>>> way for me to add the hostname to the event header somehow? I can then use
>>>> either an interceptor to read the header or even a custom sink to deal with
>>>> events based on the hostname.
>>>> --
>>>> Sharninder

View raw message