openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Santana <csantan...@gmail.com>
Subject Re: Deprecation of the "old" loadbalancer
Date Tue, 27 Feb 2018 10:24:03 GMT
I see Markus

I don’t think I’m asking for exact numbers it could be a simple statement
as.

For example: “perf test with 2 controllers and 4 invokers and saw 10%
improvement in throughput and a 5% in other indirect attributes. “

If it’s not possible to get ball park estimate of a percentage for at least
one attribute that’s OK no need to invest more time.

I think the new load balancer too awesome not to brag about it :-)

I guess anyone is free to have a taste at scale for their particular app on
their own deployment cluster or from one of the providers/vendors like IBM

+1 to your proposal to deprecate eventually remove old load balancer.

— Carlos

On Tue, Feb 27, 2018 at 2:55 AM Markus Thoemmes <markus.thoemmes@de.ibm.com>
wrote:

> Hi Carlos,
>
> in general I agree but Travis is not a good way of running those tests at
> all. It has only 1.5 cores for each build making the improvements go
> unnoticed.
>
> Please note that this is not only an improvement in terms of throughput
> but also in other not-directly-performance-related attributes. For instance
> its state is consistent (both locally and when used with multiple
> controllers). Moreover, a lot of those improvements will only be visible at
> scale. The old scheduling code for instance does not scale well over a lot
> of invokers while the new one has no issues here. All of this will not be
> shown by running a load-test in a small environment.
>
> FWIW (and this is a performance number that matters here) the non-blocking
> invocation throughput (not awaiting the result) is now equal to the raw
> Kafka produce throughput of the controller.
>
> I can work on publishing the performance numbers gathered on our internal
> deployments but in general we still need a more principled approach here.
>
> Cheers,
> Markus
>
> -----Carlos Santana <csantana23@gmail.com> wrote: -----
> To: dev@openwhisk.apache.org
> From: Carlos Santana <csantana23@gmail.com>
> Date: 02/26/2018 05:56PM
> Subject: Re: Deprecation of the "old" loadbalancer
>
> Markus
>
>   You think it would be useful as an exercise to run 2 loads using
> https://github.com/apache/incubator-openwhisk-performance on Travis.
> One with the old load balancer, and another one with the new load balancer
> and share the results side by side with data.
>
> This way is not speculative the improvements, and we will have at least one
> data point.
>
>
> -- Carlos
>
>
> On Mon, Feb 26, 2018 at 8:31 AM Markus Thoemmes <
> markus.thoemmes@de.ibm.com>
> wrote:
>
> > Heyho out there,
> >
> > as mentioned a couple of weeks ago, we implemented a new loadbalancer
> (see
> >
> https://github.com/apache/incubator-openwhisk/blob/master/core/controller/src/main/scala/whisk/core/loadBalancer/ShardingContainerPoolBalancer.scala
> ).
> > This loadbalancer has proven pretty stable so far and is used in IBM's
> > deployed version of OpenWhisk. I hereby propose that we swap the default
> > implementation to the new one as quickly as possible (in one week,
> starting
> > today) and remove the old source ASAP as well (1-2 weeks after initial
> > deprecation).
> >
> > I'd like to progress with this aggressively because having multiple
> > implementations with similar characteristics is confusing to newcomers.
> The
> > new implementation should be flat-out superior to the old one anyway (at
> > least it's supposed to).
> >
> > Please let me know if anybody has concerns with this approach and or
> would
> > like to stay on the old implementation (with state sharing).
> >
> > Cheers
> > Markus
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message