I find out the json the source get is like this : JAONHandler::getEvents print
{"ts":"1970-01-01T00:00:00.000000Z","peer":"bro","src_name":"parent","level":"info","message":"raised
pipe\u0027s socket buffer size from 208K to
1024K"}{"ts":"1970-01-01T00:00:00.000000Z","peer":"bro","src_name":"parent","level":"info","message":"communication
started, parent pid is 8972, child pid is
8976"}{"ts":"2015-05-12T15:04:17.859868Z","peer":"bro","src_name":"child","level":"info","message":"listening
on 0.0.0.0:47760
(clear)"}{"ts":"2015-05-12T15:04:17.859868Z","peer":"bro","src_name":"child","level":"info","message":"listening
on [::]:47760 (clear)"}
something like {}{}{}{}
But what my custom client send is {} one by one , Is http handler
cached the source and handle it together?
So the getEevents fun must handle jsons like {}{}{} , which type
should I send to `gson.fromJson(reader, type);`?
2015-05-12 3:26 GMT+08:00 Mo Jia <life.130815@gmail.com>:
> The offical handle like this:
>
> List<Event> eventList = new ArrayList<Event>(0);
> JSONEvent event=new JSONEvent();
> try {
> eventList = gson.fromJson(reader, listType);
> } catch (JsonSyntaxException ex) {
> throw new HTTPBadRequestException("Request has invalid JSON Syntax.", ex);
> }
>
> for (Event e : eventList) {
> ((JSONEvent) e).setCharset(charset);
> }
>
> Post must like this:
> curl -X POST -d '[{ "headers" : { "timestamp" : "110434324343",
> "host" :"random_host.example.com", "field1" : "val1", "m_user" :
> "m_user", "m_year" : "m_year", "m_month" : "m_month", "m_day" :
> "m_day" }, "body" : "random_body" }]' 127.0.0.1:8181
>
> For me , my post always is a single json, so I change the code to this:
>
> List<Event> eventList = new ArrayList<Event>(0);
> Type type = new TypeToken<JSONEvent>(){}.getType();
> JSONEvent event=new JSONEvent();
> try {
> //eventList = gson.fromJson(reader, listType);
> event = gson.fromJson(reader, type);
> } catch (JsonSyntaxException ex) {
> LOG.info("{}",reader);
> throw new HTTPBadRequestException("Request has invalid JSON Syntax.", ex);
> }
>
> // for (Event e : eventList) {
> // ((JSONEvent) e).setCharset(charset);
> // }
> ((JSONEvent) event).setCharset(charset);
> eventList.add(((Event)event));
>
> When test like this :
> curl -X POST -d '{"headers" : { "timestamp" : "140434324343", "host"
> :"random_host.example.com", "field1" : "val1", "m_user" : "m_user"}}'
> 127.0.0.1:8181
>
> It works fine.
>
> However , my c++ client using libcurl,
>
> My c++ client send POST like this :
> curl_easy_setopt(handle, CURLOPT_URL, bulk_url.c_str());
> curl_easy_setopt(handle, CURLOPT_POST, 1);
> curl_easy_setopt(handle, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)buffer.Len());
> curl_easy_setopt(handle, CURLOPT_POSTFIELDS, buffer.Bytes());
> curl_easy_setopt(handle, CURLOPT_HTTPHEADER, http_headers);
> curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION,
> &logging::writer::Flume::HTTPReceive); // This gets called with the
> result.
> curl_easy_setopt(handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0);
> curl_easy_setopt(handle, CURLOPT_NOSIGNAL, 1);
> curl_easy_setopt(handle, CURLOPT_CONNECTTIMEOUT, transfer_timeout);
> curl_easy_setopt(handle, CURLOPT_TIMEOUT, transfer_timeout);
> curl_easy_setopt(handle, CURLOPT_DNS_CACHE_TIMEOUT, 60*60);
> CURLcode return_code = curl_easy_perform(handle);
>
> and get error like this:
>
> 15/05/12 03:17:02 INFO http.JSONHandler: org.mortbay.jetty.Request$1@43d06911
> 15/05/12 03:17:02 WARN http.HTTPSource: Received bad request from client.
> org.apache.flume.source.http.HTTPBadRequestException: Request has
> invalid JSON Syntax.
>
>
> from the error, it seem curl don't send a data in a single json.
>
>
> My question is :
> 1 how can I print http.JSONHandler:
> org.mortbay.jetty.Request$1@43d06911 to show what logs I get ?
> 2 This time I modify the core source , how can I write a simple custom
> file and can use it in conf file like this:
> a1.sources.r1.handler = org.xxx.xxx.my_custom_json
> 3 Is there a better way don't need write handler to handle my
> situation here? [ A single json post]
|