flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From أنس الليثي <dev.fano...@gmail.com>
Subject Re: Flume MorphlineSolrSink
Date Wed, 18 May 2016 08:08:37 GMT
Put that rule in your mind

NoSuchMethodError = There is a mismatch in the versions of libraries you
are using.

According to your case, most probably the version of the solr jar in the
flume lib directory is different than the version of Solr you installed.


On 18 May 2016 at 10:48, Gonzalo Herreros <gherreros@gmail.com> wrote:

> Flume 1.5.0 is pretty old,
> Why don't you use version 1.6.0 included in CDH, that will ensure the
> library compatibility.
>
> On 18 May 2016 at 08:43, Baris Akgun (Garanti Teknoloji) <
> BarisAkgu@garanti.com.tr> wrote:
>
>> Hi,
>>
>>
>>
>> I am trying to make real time indexing with using flume 1.5.0 and
>> morphline solr sink. I am using packed base solr in my CDH.
>>
>>
>>
>>
>>
>> tier1.sinks = solrSink
>>
>> tier1.channels = channel1
>>
>> #
>>
>> tier1.channels.channel1.type=memory
>>
>> tier1.channels.channel1.capacity=1000000
>>
>> #
>>
>> tier1.sinks.solrSink.type =
>> org.apache.flume.sink.solr.morphline.MorphlineSolrSink
>>
>> tier1.sinks.solrSink.batchSize = 100
>>
>> tier1.sinks.solrSink.batchDurationMillis = 10000
>>
>> tier1.sinks.solrSink.morphlineFile = morphlines.conf
>>
>> tier1.sinks.solrSink.morphlineId = morphline1
>>
>> #
>>
>> tier1.sinks.solrSink.channel = channel1
>>
>>
>>
>>
>>
>> The flume throw error in sink step.  The error is given below. What is
>> your ideas to solve the problem?
>>
>>
>>
>> Thanks.
>>
>>
>>
>>
>>
>> ERROR
>>
>> Unable to start SinkRunner: {
>> policy:org.apache.flume.sink.DefaultSinkProcessor@748016d1
>> counterGroup:{ name:null counters:{} } } - Exception follows.
>>
>> java.lang.NoSuchMethodError:
>> org.apache.lucene.util.fst.FST$BytesReader.skipBytes(J)V
>>
>>          at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1117)
>>
>>          at
>> org.apache.lucene.util.fst.FST.readFirstRealTargetArc(FST.java:1030)
>>
>>          at org.apache.lucene.util.fst.FST.readRootArcs(FST.java:487)
>>
>>          at org.apache.lucene.util.fst.FST.cacheRootArcs(FST.java:475)
>>
>>          at org.apache.lucene.util.fst.FST.finish(FST.java:458)
>>
>>          at org.apache.lucene.util.fst.Builder.finish(Builder.java:464)
>>
>>          at
>> org.apache.lucene.analysis.synonym.SynonymMap$Builder.build(SynonymMap.java:284)
>>
>>          at
>> org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.loadSynonyms(FSTSynonymFilterFactory.java:145)
>>
>>          at
>> org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.inform(FSTSynonymFilterFactory.java:112)
>>
>>          at
>> org.apache.lucene.analysis.synonym.SynonymFilterFactory.inform(SynonymFilterFactory.java:89)
>>
>>          at
>> org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:675)
>>
>>          at
>> org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:167)
>>
>>          at
>> org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
>>
>>          at
>> org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
>>
>>          at
>> org.kitesdk.morphline.solr.SolrLocator.getIndexSchema(SolrLocator.java:181)
>>
>>          at
>> org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder$SanitizeUnknownSolrFields.<init>(SanitizeUnknownSolrFieldsBuilder.java:70)
>>
>>          at
>> org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder.build(SanitizeUnknownSolrFieldsBuilder.java:52)
>>
>>          at
>> org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
>>
>>          at
>> org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
>>
>>          at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:46)
>>
>>          at
>> org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)
>>
>>          at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:126)
>>
>>          at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:55)
>>
>>          at
>> org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl.configure(MorphlineHandlerImpl.java:101)
>>
>>          at
>> org.apache.flume.sink.solr.morphline.MorphlineSink.start(MorphlineSink.java:97)
>>
>>          at
>> org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
>>
>>          at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
>>
>>          at
>> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>>
>>          at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>
>>          at
>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>>
>>          at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>>
>>          at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>>
>>          at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>>          at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>>          at java.lang.Thread.run(Thread.java:745)
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Barış Akgün*
>> Analitik Veri Ambarı ve Büyük Veri Yönetimi
>> Uzman
>>
>> Tel
>>
>> :
>>
>> Dahili
>>
>> :
>>
>> Faks
>>
>> :
>>
>>
>> Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir
>> ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis
>> olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune
>> uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde
>> yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da
>> sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin
>> ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan,
>> butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve
>> bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.
>>
>> This message and attachments are confidential and intended solely for the
>> individual(s) stated in this message. If you received this message although
>> you are not the addressee, you are responsible to keep the message
>> confidential. The sender has no responsibility for the accuracy or
>> correctness of the information in the message and its attachments. Our
>> company shall have no liability for any changes or late receiving, loss of
>> integrity and confidentiality, viruses and any damages caused in anyway to
>> your computer system.
>>
>
>


-- 
Anas Rabei
Senior Software Developer
Mubasher.info
anas.rabei@mubasher.info

Mime
View raw message