flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denes Arvay <de...@cloudera.com>
Subject Re: Flume sink exeption:org.apache.flume.EventDeliveryException: Failed to send events
Date Mon, 29 Aug 2016 15:40:55 GMT
Hi,

Isn't it possible that your spark streaming application is not running when
flume tries to send events to it? That might be the reason of flume not
being able to connect to the port 31002.
According to the Spark streaming + Flume integration guide (
http://spark.apache.org/docs/latest/streaming-flume-integration.html) when
you use the push-based approach "the streaming application needs to be up,
with the receiver scheduled and listening on the chosen port, for Flume to
be able push data."

On the other hand you might want to consider using the more reliable
pull-based approach.

Regards,
Denes

On Mon, Aug 29, 2016 at 4:02 AM 罗辉 <luohui@ifeng.com> wrote:

> Hi Iain,
>
>      telnet is ok,  if I switch the sink to logger. Please check below log:
>
> source:
>
> [hadoop@master60 flume]$ telnet localhost 44444
>
> Trying ::1...
>
> telnet: connect to address ::1: Connection refused
>
> Trying 127.0.0.1...
>
> Connected to localhost.
>
> Escape character is '^]'.
>
> abc
>
> OK
>
> abc
>
> OK
>
> gggd
>
> OK
>
>
>
>
>
> Sink:
>
> 16/08/29 09:57:16 INFO node.Application: Starting Sink k1
>
> 16/08/29 09:57:16 INFO node.Application: Starting Source r1
>
> 16/08/29 09:57:16 INFO source.NetcatSource: Source starting
>
> 16/08/29 09:57:16 INFO source.NetcatSource: Created
> serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:44444]
>
> 16/08/29 09:58:10 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63
> 0D                                     abc. }
>
> 16/08/29 09:58:14 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63
> 0D                                     abc. }
>
> 16/08/29 09:58:16 INFO sink.LoggerSink: Event: { headers:{} body: 67 67 67
> 64 0D                                  gggd. }
>
>
>
> Seems it is ok when the sink is logger , but when I change the sink type
> from logger to avro. It failed with previous exception.
>
>
>
>
>
>
>
> *发件人:* iain wright [mailto:iainwrig@gmail.com]
> *发送时间:* 2016年8月27日 0:13
> *收件人:* user@flume.apache.org
> *主题:* Re: Flume sink exeption:org.apache.flume.EventDeliveryException:
> Failed to send events
>
>
>
> Can you telnet master60 31002 from the server the flume agent is running
> on?
>
>
> Sent from my iPhone
>
>
> On Aug 26, 2016, at 1:00 AM, 罗辉 <luohui@ifeng.com> wrote:
>
> 31002
>
>

Mime
View raw message