mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <bmah...@apache.org>
Subject Re: Review Request 70419: Refactor `Sorter::sorted()` to return `class SortedCandidates`.
Date Mon, 08 Apr 2019 20:07:55 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70419/#review214474
-----------------------------------------------------------



There is some unnecessary copying into the SortedCandidates vectors, perhaps that's what made
this slower as well? Can you re-run the benchmarks after those changes?


src/master/allocator/sorter/drf/sorter.hpp
Lines 445 (patched)
<https://reviews.apache.org/r/70419/#comment300684>

    This is copying, need to do:
    
    ```
    candidates(std::move(candidates_))
    ```



src/master/allocator/sorter/random/sorter.hpp
Lines 402 (patched)
<https://reviews.apache.org/r/70419/#comment300685>

    Ditto here, need to std::move



src/master/allocator/sorter/sorter.hpp
Line 139 (original), 143 (patched)
<https://reviews.apache.org/r/70419/#comment300683>

    Hm.. the unique_ptr seems cumbersome, can we just return `SortedClients` directly? And
if SortedCandidates needs to have a dynamic impl, we do that inside Sorted Candidates?
    
    ```
      virtual SortedClients sort() = 0;
    ```
    
    ```
    class Sorter::SortedClients
    {
    public:
      SortedClients(unique_ptr<SortedClientsImpl> impl);
      // ban copying, default should do that since unique_ptr member?
      // allow move construction, default should work since unique_ptr member?
      virtual ~SortedClients();
    
      // TODO(mzhu): Use `reference_wrapper` to avoid value copying.
      virtual Option<std::string> next();
    
    private:
      unique_ptr<SortedClientsImpl> impl;
    };
    ```
    
    That way, it acts the same as unique_ptr (caller can't copy it, can only move it) and
the caller doesn't have to have the pointer indirection.



src/master/allocator/sorter/sorter.hpp
Lines 155-156 (patched)
<https://reviews.apache.org/r/70419/#comment300682>

    Can you document why we don't just make this a forward iterator? Even if it's simply that
we need to investigate it more deeply to figure out if it's viable: https://en.cppreference.com/w/cpp/named_req/ForwardIterator



src/master/allocator/sorter/sorter.hpp
Lines 156 (patched)
<https://reviews.apache.org/r/70419/#comment300680>

    How about SortedClients? and referring to clients instead of candidates here and elsewhere?



src/master/allocator/sorter/sorter.hpp
Lines 162 (patched)
<https://reviews.apache.org/r/70419/#comment300681>

    Any reason not to in this patch?
    
    If there's some wrinkle in doing so, perhaps a `bool hasNext()` and `const string&
next()`?



src/tests/sorter_tests.cpp
Lines 57 (patched)
<https://reviews.apache.org/r/70419/#comment300679>

    Move it in? `push_back(*std::move(client))`


- Benjamin Mahler


On April 8, 2019, 5:15 p.m., Meng Zhu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70419/
> -----------------------------------------------------------
> 
> (Updated April 8, 2019, 5:15 p.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This patch refactors `Sorter::sorted` to return an abstract
> class `SortedCandidates` instead of a whole `vector<string>`.
> Callers can then use `SortedCandidates::next()` to get the
> next sorted candidate. This paves the way for sort optimization
> where sorting of the whole clients can be lazily done as callers
> ask for the next client.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87

>   src/master/allocator/sorter/drf/sorter.hpp 91a9d668b87079158f7072780dc86bb08865166e

>   src/master/allocator/sorter/drf/sorter.cpp 554ac84ee585d1d07048a58cf7d7d1e6586252ee

>   src/master/allocator/sorter/random/sorter.hpp 125ce84761e4c930370912151700ddda35d7b6c1

>   src/master/allocator/sorter/random/sorter.cpp bbe130dbf3b158ea14f9572bc5d14200fcd85127

>   src/master/allocator/sorter/sorter.hpp d56a1166a9e82b034564842ac071874ec2885004 
>   src/tests/sorter_tests.cpp 9d52a80eafb6f955386a6575875daacf5d4b4e9e 
> 
> 
> Diff: https://reviews.apache.org/r/70419/diff/1/
> 
> 
> Testing
> -------
> 
> make check
> 
> Benchmarking:
> Optimized build
> Ran QuotaParam/BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/2
> 
> tl;dr: ~10% slowdown for this setup
> 
> ## Before
> Added 3000 agents in 85.844373ms
> Added 3000 frameworks in 19.713969615secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks
> Made 3500 allocations in 13.690538305secs
> Made 0 allocation in 9.76855825secs
> 
> ## After
> Added 3000 agents in 94.645481ms
> Added 3000 frameworks in 19.142346619secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks
> Made 3500 allocations in 14.888207984secs
> Made 0 allocation in 10.869148916secs
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message