flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Babu, Prashanth" <Prashanth.B...@nttdata.com>
Subject Flume -- InMemory options?
Date Thu, 06 Sep 2012 21:49:08 GMT

Is it possible to use Flume to write to HBase thru InMemory?

I have a steady ingested data of 40,000 events / second.
My requirement is to write to HBase thru Flume and all this in memory. 
Basically I am aware that we can write to a log file which will be tailed by Flume and the
data is dumped into HBase. 
In our requirement, instead of this, we want to directly write to HBase [and if not possible,
at least to HDFS].

I had a look at MemoryChannel, but it seems its very limited.
Might not suffice our requirement.

Can someone please shed some light on this?
Did anyone try something like this before?


My Environment:
Hadoop - 0.20.203
Flume - 0.90.4
HBase - 0.9.4

Prashanth Babu.

Disclaimer:This email and any attachments are sent in strictest confidence for the sole use
of the addressee and may contain legally privileged, confidential, and proprietary data. 
If you are not the intended recipient, please advise the sender by replying promptly to this
email and then delete and destroy this email and any attachments without any further use,
copying or forwarding

View raw message