flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From iain wright <iainw...@gmail.com>
Subject Re: Flume - elasticsearch sink fails
Date Mon, 02 May 2016 19:51:40 GMT
Hi,

These look relevant to your goals:

https://github.com/elastic/elasticsearch/issues/14187

https://github.com/lucidfrontier45/ElasticsearchSink2

HTH,

-- 
Iain Wright

This email message is confidential, intended only for the recipient(s)
named above and may contain information that is privileged, exempt from
disclosure under applicable law. If you are not the intended recipient, do
not disclose or disseminate the message to anyone except the intended
recipient. If you have received this message in error, or are not the named
recipient(s), please immediately notify the sender by return email, and
delete all copies of this message.

On Mon, May 2, 2016 at 12:44 PM, Arif,Mubaraka <arif.mubaraka@heb.com>
wrote:

> Hello,
>
>
>
> Trying to flume tweets from Tibco-JMS source into elastic search sink.
>
> Running Elasticsearch on our edge nodes independent of Hadoop.
>
> Running cloudera Hadoop 5.4.8
>
>
>
> On running the error is  :
>
>
>
> May 2, 2:15:14.047 PM
>
> INFO
>
> org.apache.flume.instrumentation.MonitoredCounterGroup
>
> Component type: SINK, name: k1 started
>
> May 2, 2:15:14.050 PM
>
> WARN
>
> org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient
>
> [arcapl0021707:9200]
>
> May 2, 2:15:14.052 PM
>
> ERROR
>
> org.apache.flume.lifecycle.LifecycleSupervisor
>
> Unable to start SinkRunner: {
> policy:org.apache.flume.sink.DefaultSinkProcessor@50482278 counterGroup:{
> name:null counters:{} } } - Exception follows.
>
> java.lang.NoSuchMethodError:
> org.elasticsearch.common.transport.InetSocketTransportAddress.<init>(Ljava/lang/String;I)V
>
>         at
> org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.configureHostnames(ElasticSearchTransportClient.java:143)
>
>         at
> org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.<init>(ElasticSearchTransportClient.java:77)
>
>         at
> org.apache.flume.sink.elasticsearch.client.ElasticSearchClientFactory.getClient(ElasticSearchClientFactory.java:48)
>
>         at
> org.apache.flume.sink.elasticsearch.ElasticSearchSink.start(ElasticSearchSink.java:357)
>
>         at
> org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
>
>         at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
>
>         at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
>
> Flume configuration for ElasticSearch is :
>
>
>
> #confgiure elastic search sink
>
> test.sinks.k1.type = elasticsearch
>
> test.sinks.k1.hostNames = 172.17.43.207:9200
>
> test.sinks.k1.indexName = TwitterComp_index
>
> test.sinks.k1.indexType = TwitterComp_type
>
> test.sinks.k1.clusterName = twittersearch
>
> test.sinks.k1.batchSize = 500
>
> test.sinks.k1.ttl = 5d
>
> test.sinks.k1.serializer =
> org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer
>
>
>
> Any help is greatly appreciated.
>
>
>
> Thanks,
>
> *Muby*
>

Mime
View raw message