The root cause here appears to be that the Python library has only an
HTTPTransciever and the FlumeSource is expecting data in
"NettyTransciever" format.
On Mon, Jul 23, 2012 at 7:00 AM, <dinesh.simkhada@gridcore.se> wrote:
> Hello Mubarak,
> Thanks for your quick reply, is there any workaround for sending data to Avro source
using python ?
>
> Thanks
> simk
>
> -----Mubarak Seyed <seyed@apple.com> wrote: -----
> To: user@flume.apache.org
> From: Mubarak Seyed <seyed@apple.com>
> Date: 07/23/2012 01:19PM
> Subject: Re: Flume-NG Python avro Client
>
> Hi,
>
> This is a known issue in Avro. Please refer https://issues.apache.org/jira/browse/AVRO-1111
>
>
>
> Thanks.
> -Mubarak
>
>
>
>
> On Jul 23, 2012, at 4:15 AM, dinesh.simkhada@gridcore.se wrote:
> Hi,
> I am trying to build a flume client in python that send message to Avro source but always
got 'java.lang.OutOfMemoryError: Java heap space' error. I tried same thing with java it
works properly
> I have attached my source and error log, any help/suggestions exports out there ?
>
> ---------- Source
> import sys
> import httplib
> import avro.ipc as ipc
> import avro.protocol as protocol
>
> server_addr = ('192.168.104.2', 3452)
> PROTOCOL = protocol.parse(open("flume.avpr").read())
>
> def sendData():
> client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
> requestor = ipc.Requestor(PROTOCOL, client)
>
> event = dict()
> event['headers'] = {'name': 'abc', 'address': 'zyx'}
> event['body'] = bytes('hello')
>
> params = dict()
> params['event'] = event
> print("Result : " + requestor.request('append', params))
>
> client.close()
>
>
> if __name__ == '__main__':
> sendData()
>
>
> ----------- error msg
> 2012-07-23 10:57:54,903 WARN ipc.NettyServer: Unexpected exception from downstream.
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.<init>(ArrayList.java:112)
> at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:154)
> at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:131)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:282)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
> at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
>
>
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
|