mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <bmah...@apache.org>
Subject Re: Review Request 57564: Changed DRFSorter's representation of inactive clients.
Date Fri, 17 Mar 2017 19:47:06 GMT


> On March 17, 2017, 1:33 a.m., Benjamin Mahler wrote:
> > Hm.. this seems to introduce a performance regression in the case where there are
many inactive roles in the system? I assume it's difficult to avoid in the hierarchical case?
Otherwise, this looks good.
> 
> Neil Conway wrote:
>     True -- if there are many inactive clients, we'll now calculate their share and include
them in the `std::set`, whereas we wouldn't do so before (conversely, activating and deactivating
clients is a lot faster than it used to be). My guess is that the bottlenecks in sorter performance
likely lie elsewhere (e.g., updating resources, generating the entire `vector` in `sort()`
when the allocator might only need the first few entries).
>     
>     We could avoid this overhead by skipping `calculateShare` for inactive clients, and
then dirtying the whole sorter when a client is activated. My guess is that this isn't a net
win (if you have a lot of inactive clients, it seems unfortunate to dirty the entire sorter
whenever a client becomes active), but I can do some benchmarks if you think this case is
important.

Another alternative to avoid dirtying the whole sorter when a client is activated is to a
per-client dirty bit, or `Option<double> share` where if set, it is accurate (and we
can unset to induce the lazy calculation). `dirty = true` would be equivalent to clearing
all of the clients' shares. Thinking about it, the `Option<double>` seems simpler to
understand as well (no way for a stale value to be there).


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57564/#review169245
-----------------------------------------------------------


On March 13, 2017, 6:04 p.m., Neil Conway wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57564/
> -----------------------------------------------------------
> 
> (Updated March 13, 2017, 6:04 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Benjamin Mahler, and Michael Park.
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> DRFSorter previously removed inactive clients from the `clients`
> collection, and then re-added clients when they were reactivated. This
> resulted in resetting the allocation count for the client, which is
> unfortunate. This scheme would also be more difficult to adapt to
> hierarchical sorting.
> 
> This commit changes DRFSorter to continue to store inactive clients in
> the `clients`; inactive clients are indicated by a new field in the
> `Client` struct, and are omitted from the return value of
> `DRFSorter::sort`.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/sorter/drf/sorter.hpp 76329220e1115c1de7810fb69b943c78c078be59

>   src/master/allocator/sorter/drf/sorter.cpp ed54680cecb637931fc344fbcf8fd3b14cc24295

>   src/tests/sorter_tests.cpp ec0636beb936d46a253d19322f2157abe95156b6 
> 
> 
> Diff: https://reviews.apache.org/r/57564/diff/1/
> 
> 
> Testing
> -------
> 
> `make check`
> 
> 
> Thanks,
> 
> Neil Conway
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message