flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vikram Kulkarni <vikulka...@expedia.com>
Subject HTTPSource with JSONHandler
Date Tue, 15 Jan 2013 03:08:01 GMT
Hello All,

I am trying to setup a HTTPSource with the built-in JSONHandler. Per the user guide :
http://flume.apache.org/FlumeUserGuide.html#jsonhandler
It is not clear if the data sent to flume has to contain the 'Header' and 'Body' part clearly
called out? In that case, if I am getting my JSON from a different system then do I have to
actually add the header and body strings and package it before sending to Flume. If so, then
it could cause a lot of overhead to do that.
Example:
[
 {
  "headers" : {
             "timestamp" : "434324343",
             "host" : "random_host.example.com"
             },
  "body" : "random_body"
  },
  {
  "headers" : {
             "namenode" : "namenode.example.com",
             "datanode" : "random_datanode.example.com"
             },
  "body" : "really_random_body"
  }
]

So I have a huge JSON (XML actually but for now I am converting it to Json as I would need
a XMLHandler written) payload that I need to send to Flume. Do I have to wrap the Json payload
into the "body" tag?

In other words, given a standard Json, how do I convert it to Json format expected by Flume?

The user guide also says"One way to create an event in the format expected by this handler,
is to use JSONEvent provided in the Flume SDK and use Google Gson to create the JSON string
using the Gson#fromJson(Object, Type) method. "
So does tho mean I have handle this event somewhere?

Thanks in advance.

Mime
View raw message