flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Lorenz <wget.n...@gmail.com>
Subject Re: avro client fails to connect to flume
Date Wed, 07 Nov 2012 12:58:34 GMT
Hi,

> Caused by: java.io.IOException: Error connecting to localhost/127.0.0.1:41414<http://127.0.0.1:41414>
> Caused by: java.net.ConnectException: Connection refused


Can you telnet to this port? Do you have a firewall running? And did you try to bound to eth0
or 0.0.0.0?

cheers
- Alex

On Nov 2, 2012, at 5:16 AM, Siva Gudavalli <siva.gudavalli@synechron.com> wrote:

> Hi Hari,
> 
> Agent is up and the issue is with avro client that comes along  with flume...
> it is not able to establish connection with flume.
> 
> these are the logs for my avro client
> 
>> Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh
>> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop) for HDFS
access
>> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m -Dcom.sun.management.jmxremote
-Dflume.root.logger=DEBUG,console -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf'
-Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient
-H localhost -p 41414 -F /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
>> 2012-11-01 20:25:50,176 (main) [DEBUG - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)]
Batch size string = 5
>> 2012-11-01 20:25:50,284 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)]
Unable to open connection to Flume. Exception follows.
>> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 41414
}: RPC connection error
>>        at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>>        at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
>>        at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
>>        at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>>        at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
>>        at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
>> Caused by: java.io.IOException: Error connecting to localhost/127.0.0.1:41414<http://127.0.0.1:41414>
>>        at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
>>        at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
>>        at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
>>        at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
>>        ... 5 more
>> Caused by: java.net.ConnectException: Connection refused
>>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
>>        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
>>        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
>>        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
>>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>        at java.lang.Thread.run(Thread.java:722)
>> 2012-11-01 20:25:50,288 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:81)]
Exiting
>> 
> 
> 
> ________________________________________
> From: Hari Shreedharan [hshreedharan@cloudera.com]
> Sent: Friday, November 02, 2012 8:18 AM
> To: user@flume.apache.org
> Subject: Re: avro client fails to connect to flume
> 
> Please send your logs also. It will help us understand if there is an
> error during startup.
> 
> Hari
> 
> On Thu, Nov 1, 2012 at 6:44 PM, Siva Gudavalli
> <siva.gudavalli@synechron.com> wrote:
>> hi Roshan,
>> 
>> here is the config....
>> 
>> agent.sources = avro-AppSrv-source
>> agent.sinks = hdfs-Cluster-sink
>> agent.channels = mem-channel
>> 
>> # set channel for sources, sinks
>> agent.sources.avro-AppSrv-source.channels = mem-channel
>> agent.sources.hdfs-Cluster-sink.channel = mem-channel
>> 
>> # set source properties
>> agent.sources.avro-AppSrv-source.type = avro
>> agent.sources.avro-AppSrv-source.bind = localhost
>> agent.sources.avro-AppSrv-source.port = 41414
>> 
>> # set channel properties
>> agent.channels.mem-channel.type = memory
>> agent.channels.mem-channel.capacity = 10000
>> agent.channels.mem-channel.transactionCapacity = 10000
>> 
>> # set sink properties
>> agent.sinks.hdfs-Cluster-sink.type = hdfs
>> agent.sinks.hdfs-Cluster-sink.hdfs.path = hdfs://localhost:54310/flume
>> 
>> ________________________________________
>> From: Roshan Naik [roshan@hortonworks.com]
>> Sent: Friday, November 02, 2012 4:09 AM
>> To: user@flume.apache.org
>> Subject: Re: avro client fails to connect to flume
>> 
>> Could you provide your flume configs ?
>> -roshan
>> 
>> On Thu, Nov 1, 2012 at 8:13 AM, Siva Gudavalli <siva.gudavalli@synechron.com<mailto:siva.gudavalli@synechron.com>>
wrote:
>> hi,
>> 
>> i am tryint to use avro client to source my log file to avro-source.
>> 
>> flume-source -> avro source
>> flume-channel -> memory
>> flume-sink -> hdfs
>> 
>> all of the services are up; but when i am calling avro client its failing connection
to flume.
>> 
>> please help!
>> 
>> environment:
>> 
>> hadoop 0.20.2
>> flume 1.2
>> 
>> logs from agent:
>> 
>> 2012-11-01 20:03:49,977 INFO source.AvroSource: Avro source avro-AppSrv-source started.
>> 2012-11-01 20:04:01,685 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
=> /127.0.0.1:41414<http://127.0.0.1:41414>] OPEN
>> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
=> /127.0.0.1:41414<http://127.0.0.1:41414>] BOUND: /127.0.0.1:41414<http://127.0.0.1:41414>
>> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
=> /127.0.0.1:41414<http://127.0.0.1:41414>] CONNECTED: /127.0.0.1:43370<http://127.0.0.1:43370>
>> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
:> /127.0.0.1:41414<http://127.0.0.1:41414>] DISCONNECTED
>> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
:> /127.0.0.1:41414<http://127.0.0.1:41414>] UNBOUND
>> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370>
:> /127.0.0.1:41414<http://127.0.0.1:41414>] CLOSED
>> 
>> logs from avro client:
>> 
>> Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh
>> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop) for HDFS
access
>> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m -Dcom.sun.management.jmxremote
-Dflume.root.logger=DEBUG,console -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf'
-Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient
-H localhost -p 41414 -F /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
>> 2012-11-01 20:25:50,176 (main) [DEBUG - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)]
Batch size string = 5
>> 2012-11-01 20:25:50,284 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)]
Unable to open connection to Flume. Exception follows.
>> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 41414
}: RPC connection error
>>        at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>>        at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
>>        at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
>>        at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>>        at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
>>        at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
>> Caused by: java.io.IOException: Error connecting to localhost/127.0.0.1:41414<http://127.0.0.1:41414>
>>        at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
>>        at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
>>        at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
>>        at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
>>        ... 5 more
>> Caused by: java.net.ConnectException: Connection refused
>>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
>>        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
>>        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
>>        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
>>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>        at java.lang.Thread.run(Thread.java:722)
>> 2012-11-01 20:25:50,288 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:81)]
Exiting
>> 

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF


Mime
View raw message