flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From =?ks_c_5601-1987?B?QXNodXRvc2hzaGFybWEov8DHwsfDt6fG+7Czud/GwCk=?= <sharma.ashut...@kt.com>
Subject RE: multiple agents
Date Thu, 08 Nov 2012 07:37:09 GMT
Hi Alex,

Thank you for your response and inputs. Yes, we have different types of logs sources. Even,
I was thinking the same solution and it's well explained in the document. Just wanted to hear
from experts about the ideal way to define the agents.

----------------------------------------
Thanks & Regards,
Ashutosh Sharma
----------------------------------------

-----Original Message-----
From: Alexander Lorenz [mailto:wget.null@gmail.com]
Sent: Thursday, November 08, 2012 4:20 PM
To: user@flume.apache.org
Subject: Re: multiple agents

Hi,

you can use one config-file and define the agents with flow1, flow2 and so on. I assume, the
sources of the logs are different types, isn't it? When you let grab them from one agent you
could create multiple flows with (http://flume.apache.org/FlumeUserGuide.html#multiplexing-channel-selector).

Please note, you've to tweak the settings in flume-env.sh to reflect to the needed memory
consumption (rename flume-env.sh.template into flume-env.sh and comment the Xmx parameters
out).

cheers,
 Alex

On Nov 8, 2012, at 2:39 AM, Ashutoshsharma(오픈플랫폼개발팀) <sharma.ashutosh@kt.com>
wrote:

> Hi,
>
> I have sources to collect multiple types of logs(mainly three types). Most of them generate
at least two types of logs. That mean, a server generates two types of log. For my use case,
I created two separate agents running on a server to collect the logs. I am running these
agents in foreground using “flume-ng agent -n agent1” command, so two flume process.
>
> Now, I have doubt, should I merge these two agents configuration into single flume.conf
file? If I want to continue to use two different conf file, then how can I run the two agents
in background? Is there any known performance issue with any of these approaches?
>
> Please share your suggestions and thoughts.
>
> ----------------------------------------
> Thanks & Regards,
> Ashutosh Sharma
> ----------------------------------------
>
>
>
> 이 메일은 지정된 수취인만을 위해 작성되었으며, 중요한 정보나 저작권을 포함하고 있을 수
있습니다. 어떠한 권한 없이, 본 문서에 포함된 정보의 전부 또는 일부를 무단으로 제3자에게 공개,
배포, 복사 또는 사용하는 것을 엄격히 금지합니다. 만약, 본 메일이 잘못 전송된 경우, 발신인
또는 당사에 알려주시고, 본 메일을 즉시 삭제하여 주시기 바랍니다.
> This E-mail may contain confidential information and/or copyright material. This email
is intended for the use of the addressee only. If you receive this email by mistake, please
either delete it without reproducing, distributing or retaining copies thereof or notify the
sender immediately.

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF



이 메일은 지정된 수취인만을 위해 작성되었으며, 중요한 정보나 저작권을 포함하고 있을 수 있습니다.
어떠한 권한 없이, 본 문서에 포함된 정보의 전부 또는 일부를 무단으로 제3자에게 공개, 배포,
복사 또는 사용하는 것을 엄격히 금지합니다. 만약, 본 메일이 잘못 전송된 경우, 발신인 또는 당사에
알려주시고, 본 메일을 즉시 삭제하여 주시기 바랍니다.
This E-mail may contain confidential information and/or copyright material. This email is
intended for the use of the addressee only. If you receive this email by mistake, please either
delete it without reproducing, distributing or retaining copies thereof or notify the sender
immediately.
Mime
View raw message