How many files do you have? Each time a new file is to be selected, a file listing is done.
If you a lot of files, this could be slow.
Thanks,
Hari
On Thu, Oct 9, 2014 at 10:41 PM, azcmcw2@126.com <azcmcw2@126.com> wrote:
> i'm currently using CDH5's flume-ng. i've been testing an agent whose sink is hdfs and
source is spooldir. when i put files around the size of 30m into the spooldir, this agent
worked perfectly well, but when i put files around 80m the agent is still working but the
transfer speed is ridiculously slow. maybe have to spend an hour to tranfer a 80m file. does
anyone have any clue why this is happening??? thx.
> azcmcw2@126.com
|