openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Thoemmes" <markus.thoem...@de.ibm.com>
Subject Re: Deprecation of the "old" loadbalancer
Date Tue, 27 Feb 2018 07:55:03 GMT
Hi Carlos,

in general I agree but Travis is not a good way of running those tests at all. It has only
1.5 cores for each build making the improvements go unnoticed.

Please note that this is not only an improvement in terms of throughput but also in other
not-directly-performance-related attributes. For instance its state is consistent (both locally
and when used with multiple controllers). Moreover, a lot of those improvements will only
be visible at scale. The old scheduling code for instance does not scale well over a lot of
invokers while the new one has no issues here. All of this will not be shown by running a
load-test in a small environment.

FWIW (and this is a performance number that matters here) the non-blocking invocation throughput
(not awaiting the result) is now equal to the raw Kafka produce throughput of the controller.

I can work on publishing the performance numbers gathered on our internal deployments but
in general we still need a more principled approach here.

Cheers,
Markus

-----Carlos Santana <csantana23@gmail.com> wrote: -----
To: dev@openwhisk.apache.org
From: Carlos Santana <csantana23@gmail.com>
Date: 02/26/2018 05:56PM
Subject: Re: Deprecation of the "old" loadbalancer

Markus

  You think it would be useful as an exercise to run 2 loads using
https://github.com/apache/incubator-openwhisk-performance on Travis.
One with the old load balancer, and another one with the new load balancer
and share the results side by side with data.

This way is not speculative the improvements, and we will have at least one
data point.


-- Carlos


On Mon, Feb 26, 2018 at 8:31 AM Markus Thoemmes <markus.thoemmes@de.ibm.com>
wrote:

> Heyho out there,
>
> as mentioned a couple of weeks ago, we implemented a new loadbalancer (see
> https://github.com/apache/incubator-openwhisk/blob/master/core/controller/src/main/scala/whisk/core/loadBalancer/ShardingContainerPoolBalancer.scala).
> This loadbalancer has proven pretty stable so far and is used in IBM's
> deployed version of OpenWhisk. I hereby propose that we swap the default
> implementation to the new one as quickly as possible (in one week, starting
> today) and remove the old source ASAP as well (1-2 weeks after initial
> deprecation).
>
> I'd like to progress with this aggressively because having multiple
> implementations with similar characteristics is confusing to newcomers. The
> new implementation should be flat-out superior to the old one anyway (at
> least it's supposed to).
>
> Please let me know if anybody has concerns with this approach and or would
> like to stay on the old implementation (with state sharing).
>
> Cheers
> Markus
>
>


Mime
View raw message