flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Non blocking IO using Flume client
Date Thu, 29 Jan 2015 06:43:07 GMT
In your case, you'd end up having too many threads - that is right. I don't
know if there is anything you can do right now

The interesting thing is that the underlying Flume Avro RPC client is
actually non-blocking. It is exposed as a blocking client for ease of use.
Also, there is a bug in Avro that can cause the client to block during an
initial handshake - https://issues.apache.org/jira/browse/AVRO-1122

For now, I am not sure what we can do. But if you do happen to have some
time, please take a look!


On Wed, Jan 28, 2015 at 10:35 PM, Loic Descotte <loic.descotte@gmail.com>

> Of course this kind of problem can always be solved on the server side,
> but it would be better be able to handle latency with non blocking IO on
> the client. In particular if you are working with asynchronous web
> frameworks like Play Framework (or Spray.io) that work better with a low
> number of threads.

View raw message