flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Schneider <cschneiderpub...@gmail.com>
Subject Archive Task Logs (Stdout, Stderr, Sysogs) and Job Tracker logs of a Hadoop Cluster for later analysis
Date Mon, 08 Apr 2013 14:33:36 GMT
I need to collect log data from our Cluster.

For this I think I need to copy the Contents of:
* JobTracker: /var/log/hadoop-0.20-mapreduce/history/
* TaskTracker: /var/log/hadoop-0.20-mapreduce/userlogs/

It should also follow symlinks and copy recusrive.

Is flume the right tool to do this?

E.g. with the "Spooling Directory Source"?

Best Regards,

View raw message