flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dinesh.simkh...@gridcore.se
Subject Re: Flume-NG Python avro Client
Date Mon, 23 Jul 2012 12:00:17 GMT
Hello Mubarak,
Thanks for your quick reply, is there any workaround for sending data to Avro source using
python ?


-----Mubarak Seyed <seyed@apple.com> wrote: -----
To: user@flume.apache.org
From: Mubarak Seyed <seyed@apple.com>
Date: 07/23/2012 01:19PM
Subject: Re: Flume-NG Python avro Client


This is a known issue in Avro. Please refer https://issues.apache.org/jira/browse/AVRO-1111


On Jul 23, 2012, at 4:15 AM, dinesh.simkhada@gridcore.se wrote:
I am trying to build a flume client in python that send message to Avro source but always
got 'java.lang.OutOfMemoryError: Java heap space'  error. I tried same thing with java it
works properly
I have attached my source and error log, any help/suggestions exports out there ?

---------- Source
import sys
import httplib
import avro.ipc as ipc
import avro.protocol as protocol

server_addr = ('', 3452)
PROTOCOL = protocol.parse(open("flume.avpr").read())

def sendData():
    client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
    requestor = ipc.Requestor(PROTOCOL, client)
    event = dict()
    event['headers'] = {'name': 'abc', 'address': 'zyx'}
    event['body'] = bytes('hello')

    params = dict()
    params['event'] = event   
    print("Result : " + requestor.request('append', params))


if __name__ == '__main__':

----------- error msg
2012-07-23 10:57:54,903 WARN ipc.NettyServer: Unexpected exception from downstream.
java.lang.OutOfMemoryError: Java heap space
    at java.util.ArrayList.<init>(ArrayList.java:112)
    at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:154)
    at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:131)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:282)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
    at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

View raw message