flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manohar CS <Manohar...@itcinfotech.com>
Subject RE: Notification support from flume?
Date Fri, 05 Dec 2014 12:47:27 GMT
Thanks for the response. With these monitoring JSON result we can basic metrics such as no.of
events processed. But I have custom requirement to signify completion of transfer of file.

From: Ahmed Vila [mailto:avila@devlogic.eu]
Sent: Friday, December 5, 2014 5:59 PM
To: user@flume.apache.org
Subject: Re: Notification support from flume?

Hi Manohar,

You can turn Flume monitoring web server that reports the numbers as JSON.
http://archive.cloudera.com/cdh/3/flume-ng/FlumeUserGuide.html#json-reporting

It explains that flume-ng agentshould be started with parameters "-Dflume.monitoring.type=HTTP
-Dflume.monitoring.port=34545", but there is no info on how to configure those params in packed
Flume for f.e. Ubuntu.

For ubunti package that you install via apt-get, you can change /etc/flume-ng/conf/flume-env.sh
and add those params into JAVA_OPTS variable export... something like this:
export JAVA_OPTS="-Xms256m -Xmx512m -Dflume.monitoring.type=HTTP -Dflume.monitoring.port=34545"



On Fri, Dec 5, 2014 at 1:19 PM, Manohar CS <Manohar.CS@itcinfotech.com<mailto:Manohar.CS@itcinfotech.com>>
wrote:
Hi All,

I wanted to know if there is a way of notification mechanism or some way of finding out if
flume has finished transfer of certain file from spoolDir to HDFS sink? We know by looking
at .COMPLETED files in spoolDir we can assume its completed but wanted to know if there is
more reliable way of call back mechanism ?


Thanks,
Manohar.


Please consider the environment before printing this e-mail

Disclaimer: This  communication  is  for the exclusive use of the intended recipient(s) and
 shall  not attach any liability on the originator or ITC Infotech India Ltd./its  Holding
company/ its Subsidiaries/ its Group Companies. If you are the addressee, the contents of
this e-mail are intended for your use only and it shall  not be forwarded to any third party,
without first obtaining written authorization from the originator or ITC Infotech India Ltd./
its Holding company/its  Subsidiaries/ its Group Companies. It may contain information which
is confidential and legally privileged and the same shall not be used or dealt with  by any
 third  party  in  any manner whatsoever without the specific consent  of  ITC  Infotech India
Ltd./ its Holding company/ its Subsidiaries/ its Group Companies.




--

Best regards,
Ahmed Vila | Senior software developer
DevLogic | Sarajevo | Bosnia and Herzegovina

Office : +387 33 942 123
Mobile: +387 62 139 348

Website: www.devlogic.eu<http://www.devlogic.eu>
E-mail   : avila@devlogic.eu<mailto:avila@devlogic.eu>
---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended recipient(s) only. This
email contains confidential information. It should not be copied, disclosed to, retained or
used by, any party other than the intended recipient. Any unauthorised distribution, dissemination
or copying of this E-mail or its attachments, and/or any use of any information contained
in them, is strictly prohibited and may be illegal. If you are not an intended recipient then
please promptly delete this e-mail and any attachment and all copies and inform the sender
directly via email. Any emails that you send to us may be monitored by systems or persons
other than the named communicant for the purposes of ascertaining whether the communication
complies with the law and company policies.

---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended recipient(s) only. This
email contains confidential information. It should not be copied, disclosed to, retained or
used by, any party other than the intended recipient. Any unauthorised distribution, dissemination
or copying of this E-mail or its attachments, and/or any use of any information contained
in them, is strictly prohibited and may be illegal. If you are not an intended recipient then
please promptly delete this e-mail and any attachment and all copies and inform the sender
directly via email. Any emails that you send to us may be monitored by systems or persons
other than the named communicant for the purposes of ascertaining whether the communication
complies with the law and company policies.
Please consider the environment before printing this e-mail

Disclaimer: This  communication  is  for the exclusive use of the intended recipient(s) and
 shall  not attach any liability on the originator or ITC Infotech India Ltd./its  Holding
company/ its Subsidiaries/ its Group Companies. If you are the addressee, the contents of
this e-mail are intended for your use only and it shall  not be forwarded to any third party,
without first obtaining written authorization from the originator or ITC Infotech India Ltd./
its Holding company/its  Subsidiaries/ its Group Companies. It may contain information which
is confidential and legally privileged and the same shall not be used or dealt with  by any
 third  party  in  any manner whatsoever without the specific consent  of  ITC  Infotech India
Ltd./ its Holding company/ its Subsidiaries/ its Group Companies.
Mime
View raw message