mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meng Zhu <m...@mesosphere.io>
Subject Review Request 70429: Avoided hierarchical shuffling in the random sorter.
Date Mon, 08 Apr 2019 22:30:40 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70429/
-----------------------------------------------------------

Review request for mesos and Benjamin Mahler.


Bugs: MESOS-9452
    https://issues.apache.org/jira/browse/MESOS-9452


Repository: mesos


Description
-------

Due to the presence of hierarchical weights, the random
sorter currently shuffles nodes level-by-level.
This patch avoids this by first calculating the relative
weights of each active leaf node and shuffle all of them
only once. This should improve the performance for
hierarchical roles while incurs a small overhead in the
case of flat roles. In addition, this paves the way for
random sampling where shuffling can be avoided altogether.


Diffs
-----

  src/master/allocator/sorter/random/sorter.cpp bbe130dbf3b158ea14f9572bc5d14200fcd85127 


Diff: https://reviews.apache.org/r/70429/diff/1/


Testing
-------

make check
Tests for hierarchical roles in subsequent patches

Benchmarking:
Optimized build with QuotaParam/BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/2

## Before:
Added 3000 agents in 85.844373ms
Added 3000 frameworks in 19.713969615secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks
Made 3500 allocations in 13.690538305secs
Made 0 allocation in 9.76855825secs

## After:
Added 3000 agents in 93.482508ms
Added 3000 frameworks in 19.193235872secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks
Made 3500 allocations in 13.530734654secs
Made 0 allocation in 10.338803223secs


Thanks,

Meng Zhu


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message