mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meng Zhu <>
Subject Re: Review Request 70750: Removed quota role sorter in the allocator.
Date Mon, 03 Jun 2019 00:15:55 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated June 2, 2019, 5:15 p.m.)

Review request for mesos, Andrei Sekretenko and Benjamin Mahler.


Updated benchmark result per Ben's suggestion.

Bugs: MESOS-9802

Repository: mesos


This patch removes the dedicated quota role sorter
in favor of using the same sorting between satisfying
guarantees and bursting above guarantees up to limits.
The dedicated quota role sorter is tech debt from when
a "quota role" was considered different from a "non-quota"
role. However, they are the same, one just has a default quota.

This helps to simplify the logic in the allocator.
Benchmark result shows negligible performance change for
clusters with small roles (e.g. 30 and 300 roles). For
large number of roles (e.g. 3000 roles), a 5% performance
degradation is observed.

The patch would result in some behavior change if a cluster is
using oversubscribed resources with quota under DRF.
Previously, in the quota allocation stage, revocable resources
are counted towards neither the total resource pool nor a role's
allocated resources when sorting with DRF. This is arguably the
right behavior. However, after this patch, all resources,
both revocable and non-revocable ones, will be counted when
calculating DRF shares in the quota allocation stage. This means,
for a quota role that also consumes a lot of revocable resources
but no-so-much non-revocable ones, previously it would be sorted
towards the head of the queue, now it is likely to be sorted towards
the tail of the queue.


  src/master/allocator/mesos/hierarchical.hpp 71a9656fb934bf9ac58e3165254ea49cb09efa8b 
  src/master/allocator/mesos/hierarchical.cpp 40c8363afddccdd5275ca06318a8cc2cc6fa21af 


Testing (updated)

make check

Benchmark using `NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/10`
with random sorter shows:

**50% performance slowdown** with 2000 agents, 1000 roles, 2000 frameworks, with random sorter


Meng Zhu

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message