flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Json over netcat source
Date Thu, 10 Apr 2014 04:43:07 GMT
Then I really don't know what the issue is. Someone more familiar with
elastic search sink will need to look at it.

Hari

On Wednesday, April 9, 2014, Simeon Zaharici <simeon_zaharici@yahoo.com>
wrote:

> Hello
>
> This is to be able to easily integrate log centralization clients such as
> nxlog with flume
>
> The problem is not related to the splitting, or the size, even a message
> like { "blah": "blah" } will result in the exception "
> org.elasticsearch.common.xcontent.XContentBuilder"
>
> Simeon
>   On Wednesday, April 9, 2014 9:09:18 PM, Hari Shreedharan <
> hshreedharan@cloudera.com<javascript:_e(%7B%7D,'cvml','hshreedharan@cloudera.com');>>
> wrote:
>  Why exactly do you want to send JSON over netcat? Why don't you simply
> use the Avro Source with Flume SDK? I am not entirely sure about the
> Elastic search sink. The reason your JSON is not working is likely because
> netcat is splitting the events per newline or when it reaches a fixed size.
>
>
> Hari
>
>
> On Wed, Apr 9, 2014 at 5:58 PM, Simeon Zaharici <simeon_zaharici@yahoo.com<javascript:_e(%7B%7D,'cvml','simeon_zaharici@yahoo.com');>
> > wrote:
>
> Hello
>
> No, I did not find any solution to this
>
> Simeon
>   On Monday, April 7, 2014 6:09:21 AM, Deepak Subhramanian <
> deepak.subhramanian@gmail.com<javascript:_e(%7B%7D,'cvml','deepak.subhramanian@gmail.com');>>
> wrote:
>  Hi Simeon,
> I am facing the same problem. Did you find any solution ?
>
> Thanks, Deepak
>
>
> On Wed, Mar 5, 2014 at 5:52 PM, Simeon Zaharici <simeon_zaharici@yahoo.com<javascript:_e(%7B%7D,'cvml','simeon_zaharici@yahoo.com');>
> > wrote:
>
> Hi guys
>
> I would like to send json data to a flume netcat source and then send it
> to an elasticsearch sink that uses the
>
> org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer.
>
>
> It works to send plain text on the netcat source, however when I send json the event
ends up in elasticsearch as *'*org.elasticsearch.common.xcontent.XContentBuilder'.
>
> Which would be the easiest way to go about this ? I saw from reading docs that creating
a morphline interceptor could be the way to go but I did not fully understand how that
>  works.
>
> Thanks a lot
>
>
>
>
> --
> Deepak Subhramanian
>
>
>
>
>
>

Mime
View raw message