flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Bertrand <E...@bertpc.com>
Subject Re: Sinking data to a Node.js server/listener
Date Thu, 21 Feb 2013 13:41:43 GMT
Hello Bertrand.  Thanks very much for the reply!  (I love that you we share
a common first name/last name!)

I was not aware that there is a syslog sink.  I see there's an HDFS sink
and a "File Roll" sink, but those do not seem to utilize syslog (I'm very
new to this, so I may be misguided).  How would I go about setting up such
a sync - are you saying I would be able to post the data to an IP/port that
Node.js is listening on?

Again, many thanks for the assistance!


On Sun, Feb 17, 2013 at 5:43 PM, Bertrand Dechoux <dechouxb@gmail.com>wrote:

> Should it be assumed that structured data are transferred using flume and
> that's why Avro is mentioned? One easy solution I can see would be to use a
> syslog output that would actually point to your nodejs instance (and not a
> real syslog endpoint).
> Regards
> Bertrand
> On Sun, Feb 17, 2013 at 5:50 AM, Erik Bertrand <Erik@bertpc.com> wrote:
>> I'm looking to get Flume data transferred to a Node.js server listening
>> on a specific port.  I'm not interested in actually storing the data
>> anywhere, as it's simply using that data to display "current activity" and
>> doesn't need to be persist it anywhere.  I'm transferring a very small
>> amount of data - just a series of IP addresses, actually.  So I'd like to
>> keep the architecture simple, too.
>> At first I was thinking I could use the Avro sink to send the data
>> directly to Node.js using a dnode <https://npmjs.org/package/dnode> server
>> object (i.e. RPC), but there seems to be more to it than the basic setup.
>>  I'm just not sure how to configure the Node.js side to understand the Avro
>> sink RPC request (or if that's even possible).
>> I've been looking at creating a custom sink to do this; I've not written
>> one before, much less written anything in Java, so that'd be new to me.
>>  Any pointers?
>> Erik
> --
> Bertrand Dechoux

View raw message