mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neil Conway <>
Subject Re: Review Request 44251: Improved allocator perf for labeled reservations and volumes.
Date Wed, 02 Mar 2016 19:17:04 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated March 2, 2016, 7:17 p.m.)

Review request for mesos and Joris Van Remoortere.


Tweak variable name.

Bugs: MESOS-4833

Repository: mesos


When the cluster contains many resources that have either labeled
reservations or persistent volumes, allocator performance can decrease
substantially because such metadata prevents `Resource` objects from
being merged together inside the allocator. As a result, the allocator
must manipulate `Resources` vectors that consist of many small
individual `Resource` values; since many `Resources` operations take
linear-time in the number of `Resource` values they contain, this can
cause very significant slowdowns.

As a short-term solution, this commit strips dynamic reservation and
persistent volume information from the `Resources` objects used
internally by the allocator, because they are not needed when
aggregating resource quantities together.

A long-term solution for this problem will be addressed as work on
refactoring the allocator more generally.

Diffs (updated)

  src/master/allocator/mesos/hierarchical.cpp 24fa50f62dec10ed43089297473cc386d6ba2f78 
  src/master/allocator/sorter/drf/sorter.hpp 46b2a9caf13b028a3aee6c1590679f885be90fd6 
  src/master/allocator/sorter/drf/sorter.cpp 9e863dd0ca5e2f2f0d517cb833687b757bed2c52 
  src/master/allocator/sorter/sorter.hpp ba91a38e47065718af87c9b3b7c5b74d25a258df 



make check


[ RUN      ] HierarchicalAllocator_BENCHMARK_Test.ResourceLabels
Using 2000 slaves and 200 frameworks
round 0 allocate took 2.458462secs to make 200 offers
round 1 allocate took 2.427941secs to make 200 offers
round 2 allocate took 2.460724secs to make 200 offers
round 3 allocate took 2.443408secs to make 200 offers
round 4 allocate took 2.464784secs to make 200 offers
round 5 allocate took 2.501429secs to make 200 offers
round 6 allocate took 2.468777secs to make 200 offers
round 7 allocate took 2.482268secs to make 200 offers
round 8 allocate took 2.479014secs to make 200 offers
round 9 allocate took 2.529951secs to make 200 offers
round 10 allocate took 2.460059secs to make 200 offers

Performance of `DeclineOffers` without labels is about ~2.1 seconds.


Neil Conway

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message