flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Lord <jl...@cloudera.com>
Subject Re: Import files from a directory on remote machine
Date Thu, 17 Apr 2014 00:16:30 GMT
http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source


On Wed, Apr 16, 2014 at 5:14 PM, Something Something <
mailinglists19@gmail.com> wrote:

> Hello,
>
> Needless to say I am newbie to Flume, but I've got a basic flow working in
> which I am importing a log file from my linux box to hdfs.  I am using
>
> a1.sources.r1.command = tail -F /var/log/xyz.log
>
> which is working like a stream of messages.  This is good!
>
> Now what I want to do is copy log files from a directory on a remote
> machine on a regular basis.  For example:
>
> username@machinename:/var/log/logdir/<multiple files>
>
> One way to do it is to simply 'scp' files from the remote directory into
> my box on a regular basis, but what's the best way to do this in Flume?
> Please let me know.
>
> Thanks for the help.
>
>
>

Mime
View raw message