flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammad Yaseen <yaseenaftab...@gmail.com>
Subject Re: Flume HTTP Source Handler Request size limit
Date Thu, 31 Aug 2017 09:45:58 GMT
Hi,

Thanks for the suggestion. I will take a look at Blob Handler.

On Thu, Aug 31, 2017 at 1:28 PM, Archana Ravindran <srarchana93@gmail.com>
wrote:

> You can try using blob handler in http source. Though blob is for binary,
> we were able to push relatively large event.
>
> For size limit, in our case we were not able to find default support in
> flume. So we customized the code with a small function for checking the
> size.
> You can try similar.
>
> Thanks,
> Archana.
>
>
> On Aug 31, 2017 12:29 PM, "Muhammad Yaseen" <yaseenaftab.gt@gmail.com>
> wrote:
>
> Hello,
>
> I am using an http source for my Flume agent and would like to know if
> there a size limit (explicit or implicit) on POST request body/data
> (content length) for Flume's HTTP Source ?
>
> I have a handler which is throwing* "MalformedJsonException: Unterminated
> string at line 1 column 966657*" (number varies) . I know for a fact that
> the JSON is correct, for some reason the Handler is dropping the POST body.
> This leads to *"HTTPBadRequestException: Request has invalid JSON
> Syntax."*
>
> The source receives a huge request with an array of JSON objects as events
> (10000-30000 events per request body).
> This problem occurs when the content length ( request.getContentLength() )
> is large e.g. 2142675 but works fine when the length is smaller, e.g. 38320.
>
> Has anyone experienced this ?
>
> I am interested in the reason causing this problem.
>
> Regards,
> Muhammad Yaseen
>
>
>

Mime
View raw message