flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shankar <varun.c...@gmail.com>
Subject Re: Centralizing logs generated in EC2 instances.
Date Wed, 26 Dec 2012 13:37:58 GMT

Thanks for the quick reply.

I went through this<http://flume.apache.org/FlumeUserGuide.html#flume-sources>.
Not able to decide which Flume Source should I use.
--> It doesn't seem to recommend using Exec Source as the event may get
lost and hence no guarantee of delivery.
--> Spooling dir source won't work for me as it works only for immutable

Reliable delivery is very important for me.

Can you suggest Flume Source which will work for me?

On Wed, Dec 26, 2012 at 6:58 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:

> yes flume will definitely solve this problem
> look at flume's source
> there are multiple ways you can feed the logs to flume .. simplest would
> be tail -f in the flume exec source
> On Wed, Dec 26, 2012 at 6:37 PM, Shankar <varun.ccet@gmail.com> wrote:
>> I have an application running on multiple EC2 instances. I need to
>> aggregate the logs generated in those instances to a central location.
>> i.e., the logs generated on the EC2 instances automatically become
>> available on the centralized server. I have logs generated by my
>> application and many 3rd part applications, system logs such as syslog,
>> secure log etc.
>> I have following questions:
>> 1. Can Flume solve this requirement?
>> 2. Who is going to feed the log files to Flume agent? Do I need some
>> other tool to feed my logs to Flume?
>> --
>> Regards,
>> Varun Shankar
> --
> Nitin Pawar

Varun Shankar

View raw message