mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Mann <g...@mesosphere.io>
Subject Review Request 67104: Fixed a race condition in the allocator metrics.
Date Sat, 12 May 2018 01:02:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67104/
-----------------------------------------------------------

Review request for mesos, Benjamin Mahler and Vinod Kone.


Bugs: MESOS-8904
    https://issues.apache.org/jira/browse/MESOS-8904


Repository: mesos


Description
-------

This patch addresses a race condition in which the removal of
a role from the allocator's quota sorter races with execution
of a callback tied to a `PullGauge`. The gauge's callback
assumed that the role would be present in the sorter, but it's
possible for the role to be removed before the callback is
executed.


Diffs
-----

  src/master/allocator/mesos/hierarchical.cpp 1000968be6a2935a4cac571414d7f06d7df7acf0 


Diff: https://reviews.apache.org/r/67104/diff/1/


Testing
-------

The test `MasterQuotaTest.RemoveSingleQuota` was modified to include a call to '/metrics/snapshot',
both with and without a framework registered in the role for which quota is set. It's not
easy to ensure that such a test provokes the race condition 100% of the time, but the test
would always expose it within a few iterations.

Before this patch, the test would crash reliably within several iterations. After this patch,
the test can be run many times with no failures.


Thanks,

Greg Mann


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message