flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Loic Descotte <loic.desco...@gmail.com>
Subject Re: Non blocking IO using Flume client
Date Thu, 29 Jan 2015 06:45:53 GMT
Thanks for the link, I'll take a look.

2015-01-29 7:43 GMT+01:00 Hari Shreedharan <hshreedharan@cloudera.com>:

> In your case, you'd end up having too many threads - that is right. I
> don't know if there is anything you can do right now
> The interesting thing is that the underlying Flume Avro RPC client is
> actually non-blocking. It is exposed as a blocking client for ease of use.
> Also, there is a bug in Avro that can cause the client to block during an
> initial handshake - https://issues.apache.org/jira/browse/AVRO-1122
> For now, I am not sure what we can do. But if you do happen to have some
> time, please take a look!
> Hari
> On Wed, Jan 28, 2015 at 10:35 PM, Loic Descotte <loic.descotte@gmail.com>
> wrote:
>> Of course this kind of problem can always be solved on the server side,
>> but it would be better be able to handle latency with non blocking IO on
>> the client. In particular if you are working with asynchronous web
>> frameworks like Play Framework (or Spray.io) that work better with a low
>> number of threads.

Loïc Descotte

View raw message