From reviews-return-90413-apmail-mesos-reviews-archive=mesos.apache.org@mesos.apache.org Thu Nov 21 16:45:28 2019 Return-Path: X-Original-To: apmail-mesos-reviews-archive@minotaur.apache.org Delivered-To: apmail-mesos-reviews-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by minotaur.apache.org (Postfix) with SMTP id B954919FD7 for ; Thu, 21 Nov 2019 16:45:27 +0000 (UTC) Received: (qmail 71973 invoked by uid 500); 21 Nov 2019 16:45:27 -0000 Delivered-To: apmail-mesos-reviews-archive@mesos.apache.org Received: (qmail 71924 invoked by uid 500); 21 Nov 2019 16:45:27 -0000 Mailing-List: contact reviews-help@mesos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: reviews@mesos.apache.org Delivered-To: mailing list reviews@mesos.apache.org Received: (qmail 71841 invoked by uid 99); 21 Nov 2019 16:45:26 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Nov 2019 16:45:26 +0000 Received: from reviews.apache.org (unknown [10.41.0.12]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id DCEEBE0E2E; Thu, 21 Nov 2019 16:45:25 +0000 (UTC) Received: from reviews-vm2.apache.org (localhost [IPv6:::1]) by reviews.apache.org (ASF Mail Server at reviews-vm2.apache.org) with ESMTP id 87D3EC40220; Thu, 21 Nov 2019 16:45:25 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============6806915490591726461==" MIME-Version: 1.0 Subject: Re: Review Request 71750: Improved performance of v1 operator API GetAgents call. From: Andrei Sekretenko To: Meng Zhu , Andrei Sekretenko , Greg Mann Cc: Benjamin Mahler , mesos Date: Thu, 21 Nov 2019 16:45:25 -0000 Message-ID: <20191121164525.46675.37147@reviews-vm2.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: Andrei Sekretenko X-ReviewGroup: mesos X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/71750/ X-Sender: Andrei Sekretenko References: <20191112000308.61126.47648@reviews-vm2.apache.org> In-Reply-To: <20191112000308.61126.47648@reviews-vm2.apache.org> Reply-To: Andrei Sekretenko X-ReviewRequest-Repository: mesos --===============6806915490591726461== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71750/#review218732 ----------------------------------------------------------- src/master/http.cpp Lines 2235-2237 (patched) One more thought after looking at r71751-r71754: it should be possible to precalculate the size of the would-be `output` and serialize directly into the "top-level" `CodedOutputStream` (in this patch: the one in getAgents), without the intermediate serialization into string and copying it into the "top-level" stream, right? Given that the plan is to refactor the common parts later, this should probably be left as one more TODO item... - Andrei Sekretenko On Nov. 12, 2019, 12:03 a.m., Benjamin Mahler wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71750/ > ----------------------------------------------------------- > > (Updated Nov. 12, 2019, 12:03 a.m.) > > > Review request for mesos, Andrei Sekretenko, Greg Mann, and Meng Zhu. > > > Bugs: MESOS-10026 > https://issues.apache.org/jira/browse/MESOS-10026 > > > Repository: mesos > > > Description > ------- > > This updates the handling to serialize directly to protobuf or json > from the in-memory v0 state, bypassing expensive intermediate > serialization / de-serialization / object construction / object > destruction. > > This initial patch shows the approach that will be used for the > other expensive calls. Note that this type of manual writing is > more brittle and complex, but it can be mostly eliminated if we > keep an up-to-date v1 GetState in memory in the future. > > When this approach is applied fully to GetState, it leads to the > following improvement: > > Before: > v0 '/state' response took 6.55 secs > v1 'GetState' application/x-protobuf response took 24.08 secs > v1 'GetState' application/json response took 22.76 secs > > After: > v0 '/state' response took 8.00 secs > v1 'GetState' application/x-protobuf response took 5.73 secs > v1 'GetState' application/json response took 9.62 secs > > > Diffs > ----- > > src/master/http.cpp 60765c9b9d6903f6ed94fa8c614055698caad0da > src/master/master.hpp dc45028d2ecfb61bf9ea82d90d2393af648a6023 > > > Diff: https://reviews.apache.org/r/71750/diff/1/ > > > Testing > ------- > > make check > > > Thanks, > > Benjamin Mahler > > --===============6806915490591726461==--