flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siva Gudavalli <siva.gudava...@synechron.com>
Subject RE: avro client fails to connect to flume
Date Fri, 02 Nov 2012 04:16:24 GMT
Hi Hari,

Agent is up and the issue is with avro client that comes along  with flume...
it is not able to establish connection with flume.

these are the logs for my avro client

> Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh
> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop) for HDFS access
> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m -Dcom.sun.management.jmxremote
-Dflume.root.logger=DEBUG,console -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf'
-Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient
-H localhost -p 41414 -F /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
> 2012-11-01 20:25:50,176 (main) [DEBUG - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)]
Batch size string = 5
> 2012-11-01 20:25:50,284 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)]
Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 41414 }:
RPC connection error
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
>         at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
>         at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>         at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
>         at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
> Caused by: java.io.IOException: Error connecting to localhost/127.0.0.1:41414<http://127.0.0.1:41414>
>         at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
>         at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
>         at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
>         ... 5 more
> Caused by: java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
>         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
>         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
>         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:722)
> 2012-11-01 20:25:50,288 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:81)]
Exiting
>


________________________________________
From: Hari Shreedharan [hshreedharan@cloudera.com]
Sent: Friday, November 02, 2012 8:18 AM
To: user@flume.apache.org
Subject: Re: avro client fails to connect to flume

Please send your logs also. It will help us understand if there is an
error during startup.

Hari

On Thu, Nov 1, 2012 at 6:44 PM, Siva Gudavalli
<siva.gudavalli@synechron.com> wrote:
> hi Roshan,
>
> here is the config....
>
> agent.sources = avro-AppSrv-source
> agent.sinks = hdfs-Cluster-sink
> agent.channels = mem-channel
>
> # set channel for sources, sinks
> agent.sources.avro-AppSrv-source.channels = mem-channel
> agent.sources.hdfs-Cluster-sink.channel = mem-channel
>
> # set source properties
> agent.sources.avro-AppSrv-source.type = avro
> agent.sources.avro-AppSrv-source.bind = localhost
> agent.sources.avro-AppSrv-source.port = 41414
>
> # set channel properties
> agent.channels.mem-channel.type = memory
> agent.channels.mem-channel.capacity = 10000
> agent.channels.mem-channel.transactionCapacity = 10000
>
> # set sink properties
> agent.sinks.hdfs-Cluster-sink.type = hdfs
> agent.sinks.hdfs-Cluster-sink.hdfs.path = hdfs://localhost:54310/flume
>
> ________________________________________
> From: Roshan Naik [roshan@hortonworks.com]
> Sent: Friday, November 02, 2012 4:09 AM
> To: user@flume.apache.org
> Subject: Re: avro client fails to connect to flume
>
> Could you provide your flume configs ?
> -roshan
>
> On Thu, Nov 1, 2012 at 8:13 AM, Siva Gudavalli <siva.gudavalli@synechron.com<mailto:siva.gudavalli@synechron.com>>
wrote:
> hi,
>
> i am tryint to use avro client to source my log file to avro-source.
>
> flume-source -> avro source
> flume-channel -> memory
> flume-sink -> hdfs
>
> all of the services are up; but when i am calling avro client its failing connection
to flume.
>
> please help!
>
> environment:
>
> hadoop 0.20.2
> flume 1.2
>
> logs from agent:
>
> 2012-11-01 20:03:49,977 INFO source.AvroSource: Avro source avro-AppSrv-source started.
> 2012-11-01 20:04:01,685 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
=> /127.0.0.1:41414<http://127.0.0.1:41414>] OPEN
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
=> /127.0.0.1:41414<http://127.0.0.1:41414>] BOUND: /127.0.0.1:41414<http://127.0.0.1:41414>
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
=> /127.0.0.1:41414<http://127.0.0.1:41414>] CONNECTED: /127.0.0.1:43370<http://127.0.0.1:43370>
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
:> /127.0.0.1:41414<http://127.0.0.1:41414>] DISCONNECTED
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
:> /127.0.0.1:41414<http://127.0.0.1:41414>] UNBOUND
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
:> /127.0.0.1:41414<http://127.0.0.1:41414>] CLOSED
>
> logs from avro client:
>
> Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh
> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop) for HDFS access
> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m -Dcom.sun.management.jmxremote
-Dflume.root.logger=DEBUG,console -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf'
-Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient
-H localhost -p 41414 -F /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
> 2012-11-01 20:25:50,176 (main) [DEBUG - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)]
Batch size string = 5
> 2012-11-01 20:25:50,284 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)]
Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 41414 }:
RPC connection error
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
>         at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
>         at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>         at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
>         at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
> Caused by: java.io.IOException: Error connecting to localhost/127.0.0.1:41414<http://127.0.0.1:41414>
>         at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
>         at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
>         at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
>         ... 5 more
> Caused by: java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
>         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
>         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
>         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:722)
> 2012-11-01 20:25:50,288 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:81)]
Exiting
>

Mime
View raw message