On Mon, Jul 9, 2012 at 8:51 AM, Juhani Connolly <
juhani_connolly@cyberagent.co.jp> wrote:
> - Intended setup with flume was a file channel connected to an avro
> sink. With only a single disk available, it is extremely slow. JDBC channel
> is also extremely slow, and MemoryChannel will fill up and start refusing
> puts as soon as a network issue comes up.
>
Have you taken a few thread dumps or done other analysis? When you say
"extremely slow" what do you mean? Configured for no dataloss FileChannel
is going to be doing a lot of fsync'ing so I am not surprised it's slow.
That is a property of disks not FileChannel. I think we should use group
commit but that shouldn't make it 10x faster.
Brock
|