flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Flume Exception - http source
Date Thu, 12 Nov 2015 23:23:46 GMT
I don't think all is in order. HTTP Source is trying to listen on a port
which has another process already listening on.


Thanks,
Hari

On Thu, Nov 12, 2015 at 3:01 PM, Timothy Garza <
timothy.garza@collinsongroup.com> wrote:

> So why the exception if all is in order?
>
>
>
>
>
> *Kind Regards *
>
>
> *Timothy Garza*
>
> Data Integration Developer
> *Collinson Technology Services*
>
> Skype: timothy.garza.cts
>
> collinsongroup.com <http://www.collinsongroup.com/>
>
>
>
> [image: Collinson Group] <http://www.collinsongroup.com/>
>
>
>
>
>
> *From:* iain wright [mailto:iainwrig@gmail.com]
> *Sent:* 12 November 2015 22:53
>
> *To:* user@flume.apache.org
> *Subject:* Re: Flume Exception - http source
>
>
>
> "A source which accepts Flume Events by HTTP POST and GET. GET should be
> used for experimentation only." --
> https://flume.apache.org/FlumeUserGuide.html#http-source
>
>
>
> This is a source that *accepts* data, it will not *poll* an api. You
> would need your API to push events to flume, or have some tooling
> inbetween, that polled the API & posted the result to flume.
>
>
>
> bind
>
> 0.0.0.0
>
> The hostname or IP address to listen on
>
>
> --
>
> Iain Wright
>
>
>
> This email message is confidential, intended only for the recipient(s)
> named above and may contain information that is privileged, exempt from
> disclosure under applicable law. If you are not the intended recipient, do
> not disclose or disseminate the message to anyone except the intended
> recipient. If you have received this message in error, or are not the named
> recipient(s), please immediately notify the sender by return email, and
> delete all copies of this message.
>
>
>
> On Thu, Nov 12, 2015 at 2:35 PM, Timothy Garza <
> timothy.garza@collinsongroup.com> wrote:
>
> Should there be an exception, whilst Flume is listening?
>
>
>
>
>
> *Kind Regards *
>
>
> *Timothy Garza*
>
> Database and BI Developer
> *Collinson Technology Services*
>
> Skype: timothy.garza.cts
>
> collinsongroup.com <http://www.collinsongroup.com/>
>
>
>
> [image: Collinson Group] <http://www.collinsongroup.com/>
>
>
>
>
>
> *From:* Hari Shreedharan [mailto:hshreedharan@cloudera.com]
> *Sent:* 12 November 2015 22:02
> *To:* user@flume.apache.org
> *Subject:* Re: Flume Exception - http source
>
>
>
> HTTP Source does not pull anything from an address, instead it binds to
> the address specified in the config. It can listen on that port and then
> accept data from applications.
>
>
> Thanks,
> Hari Shreedharan
>
>
>
> On Nov 12, 2015, at 1:59 PM, Timothy Garza <
> timothy.garza@collinsongroup.com> wrote:
>
>
>
> Hi Flumers
>
>
>
> I’m new to Flume so please go easy on me, but this is a placeholder for
> those that follow.
>
>
>
> I’m seeing the following Exception when configuring a http source for
> Flume v1.6.0, pointing at an externally hosted web API I know I can connect
> to, and GET/POST to otherwise
>
>
>
> 15/11/12 21:23:34 ERROR lifecycle.LifecycleSupervisor: Unable to start
> EventDrivenSourceRunner: {
> source:org.apache.flume.source.http.HTTPSource{name:<source-name>,state:IDLE}
> } - Exception follows.
>
> java.lang.IllegalStateException: Running HTTP Server found in source:
> <source-name> before I started one.Will not attempt to start.  “ßException
> text!!!”
>
>         at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>
>         at
> org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:176)
>
>         at
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
>
>         at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
> Here are my config lines for the Flume source:
>
> *flume-ng agent *
>
> -Dflume.root.logger=WARN,console
>
>
>
> *--conf-file*
>
> <Flume-Agent>.sources.<source-name>.type = http
>
> <Flume-Agent>.sources.<source-name>.port = 443
>
> <Flume-Agent>.sources.<source-name>.bind = https://<url to a perfectly
> healthy web API> ß serving JSON data
>
> <Flume-Agent>.sources.<source-name>.handler =
> org.apache.flume.source.http.JSONHandler
>
> <Flume-Agent>.sources.<source-name>.selector.type = replicating ß multiple
> sinks
>
>
>
> Now I’ve seen others have had to recompile the Flume Jar to resolve this
> but is that really what’s going on here?
>
>
> https://community.cloudera.com/t5/Data-Ingestion-Integration/Flume-TwitterSource-language-filter/td-p/23519
>
> https://github.com/cloudera/cdh-twitter-example/issues/10
>
>
>
> Apart from the Exceptions, what other logging will help diagnose this?
>
>
>
>
>
> The Collinson Group Limited; Registered number: 2577557, Registered in
> England & Wales; Registered Office: Cutlers Exchange, 123 Houndsditch,
> London, EC3A 7BU.
>
>
> This e-mail may contain privileged and confidential information and/or
> copyright material and is intended for the use of the addressee only. If
> you receive this e-mail by mistake please advise the sender immediately by
> using the reply facility in your e-mail software and delete this e-mail
> from your computer system. You may not deliver, copy or disclose its
> contents to anyone else. Any unauthorised use may be unlawful. Any views
> expressed in this e-mail are those of the individual sender and may not
> necessarily reflect the views of The Collinson Group Ltd and/or its
> subsidiaries or any other associated company (collectively “Collinson
> Group”).
>
> As communications via the Internet are not secure Collinson Group cannot
> accept any liability if this e-mail is accessed by third parties during the
> course of transmission or is modified or amended in any way following
> despatch. Collinson Group cannot guarantee that any attachment to this
> email does not contain a virus, therefore it is strongly recommended that
> you carry out your own virus check before opening any attachment, as we
> cannot accept liability for any damage sustained as a result of software
> virus infection. Senders of messages shall be taken to consent to the
> monitoring and recording of e-mails addressed to members of the Company.
>
>
>
>
>

Mime
View raw message