mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meng Zhu <m...@mesosphere.io>
Subject Re: Review Request 70419: Refactor `Sorter::sorted()` to return `class SortedCandidates`.
Date Tue, 09 Apr 2019 06:37:07 GMT


> On April 8, 2019, 1:07 p.m., Benjamin Mahler wrote:
> > There is some unnecessary copying into the SortedCandidates vectors, perhaps that's
what made this slower as well? Can you re-run the benchmarks after those changes?

Ooops, thanks for catching that! Reran the benchmark, updated the test result, minimal performance
impact as expected.


> On April 8, 2019, 1:07 p.m., Benjamin Mahler wrote:
> > src/master/allocator/sorter/sorter.hpp
> > Line 139 (original), 143 (patched)
> > <https://reviews.apache.org/r/70419/diff/1/?file=2137905#file2137905line143>
> >
> >     Hm.. the unique_ptr seems cumbersome, can we just return `SortedClients` directly?
And if SortedCandidates needs to have a dynamic impl, we do that inside Sorted Candidates?
> >     
> >     ```
> >       virtual SortedClients sort() = 0;
> >     ```
> >     
> >     ```
> >     class Sorter::SortedClients
> >     {
> >     public:
> >       SortedClients(unique_ptr<SortedClientsImpl> impl);
> >       // ban copying, default should do that since unique_ptr member?
> >       // allow move construction, default should work since unique_ptr member?
> >       virtual ~SortedClients();
> >     
> >       // TODO(mzhu): Use `reference_wrapper` to avoid value copying.
> >       virtual Option<std::string> next();
> >     
> >     private:
> >       unique_ptr<SortedClientsImpl> impl;
> >     };
> >     ```
> >     
> >     That way, it acts the same as unique_ptr (caller can't copy it, can only move
it) and the caller doesn't have to have the pointer indirection.

Sounds good.
Though I personally think a `unique_ptr` is not that bad, especially considering we will need
to add extra complexities to remove it.
Can we do it in a separate patch? This way, we can keep this patch simple and land it sooner.

I have added a TODO here:

```
// TODO(mzhu): Use PIMPLE pattern here to make `SortedClients`
// a concrete class so that we could return by value in `sort()`.
```


> On April 8, 2019, 1:07 p.m., Benjamin Mahler wrote:
> > src/master/allocator/sorter/sorter.hpp
> > Lines 155-156 (patched)
> > <https://reviews.apache.org/r/70419/diff/1/?file=2137905#file2137905line155>
> >
> >     Can you document why we don't just make this a forward iterator? Even if it's
simply that we need to investigate it more deeply to figure out if it's viable: https://en.cppreference.com/w/cpp/named_req/ForwardIterator

It is entirely valid and possible to make it an input iterator (forward iterator's lvalue
dereferencing semantic is not applicable here). It's just in this MVP, a `next()` is enough.
I have added a TODO to make it an input iterator.


> On April 8, 2019, 1:07 p.m., Benjamin Mahler wrote:
> > src/master/allocator/sorter/sorter.hpp
> > Lines 156 (patched)
> > <https://reviews.apache.org/r/70419/diff/1/?file=2137905#file2137905line156>
> >
> >     How about SortedClients? and referring to clients instead of candidates here
and elsewhere?

I was thinking clients is more internal to the sorter. Since this class is what we return,
I named it `SortedCandidates` thinking that it fits better in the caller's context. But, I
am also OK with `SortedClients`. Fixed.


> On April 8, 2019, 1:07 p.m., Benjamin Mahler wrote:
> > src/master/allocator/sorter/sorter.hpp
> > Lines 162 (patched)
> > <https://reviews.apache.org/r/70419/diff/1/?file=2137905#file2137905line162>
> >
> >     Any reason not to in this patch?
> >     
> >     If there's some wrinkle in doing so, perhaps a `bool hasNext()` and `const string&
next()`?

Done.


> On April 8, 2019, 1:07 p.m., Benjamin Mahler wrote:
> > src/tests/sorter_tests.cpp
> > Lines 57 (patched)
> > <https://reviews.apache.org/r/70419/diff/1/?file=2137906#file2137906line57>
> >
> >     Move it in? `push_back(*std::move(client))`

Since we are returning `reference_wrapper` now, move is not needed/applicable here.


- Meng


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70419/#review214474
-----------------------------------------------------------


On April 8, 2019, 10:15 a.m., Meng Zhu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70419/
> -----------------------------------------------------------
> 
> (Updated April 8, 2019, 10:15 a.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This patch refactors `Sorter::sorted` to return an abstract
> class `SortedCandidates` instead of a whole `vector<string>`.
> Callers can then use `SortedCandidates::next()` to get the
> next sorted candidate. This paves the way for sort optimization
> where sorting of the whole clients can be lazily done as callers
> ask for the next client.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87

>   src/master/allocator/sorter/drf/sorter.hpp 91a9d668b87079158f7072780dc86bb08865166e

>   src/master/allocator/sorter/drf/sorter.cpp 554ac84ee585d1d07048a58cf7d7d1e6586252ee

>   src/master/allocator/sorter/random/sorter.hpp 125ce84761e4c930370912151700ddda35d7b6c1

>   src/master/allocator/sorter/random/sorter.cpp bbe130dbf3b158ea14f9572bc5d14200fcd85127

>   src/master/allocator/sorter/sorter.hpp d56a1166a9e82b034564842ac071874ec2885004 
>   src/tests/sorter_tests.cpp 9d52a80eafb6f955386a6575875daacf5d4b4e9e 
> 
> 
> Diff: https://reviews.apache.org/r/70419/diff/2/
> 
> 
> Testing
> -------
> 
> make check
> 
> Benchmarking:
> Optimized build
> Ran QuotaParam/BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/2
> 
> tl;dr: minimal performance impact
> 
> ## Before
> Added 3000 agents in 85.844373ms
> Added 3000 frameworks in 19.713969615secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks
> Made 3500 allocations in 13.690538305secs
> Made 0 allocation in 9.76855825secs
> 
> ## After
> Added 3000 agents in 90.247786ms
> Added 3000 frameworks in 19.094854883secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with drf sorter
> Made 3500 allocations in 13.885463083secs
> Made 0 allocation in 10.165977763secs
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message