mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Klues" <>
Subject Re: Review Request 42053: Add flags to set size of completed task/framework history.
Date Fri, 08 Jan 2016 17:47:57 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Jan. 8, 2016, 5:47 p.m.)

Review request for mesos and Ben Mahler.


This change doesn't resolve any issues and simply fixes a merge conflict due to added code
via a rebase.

Bugs: MESOS-3307

Repository: mesos


The default size of the buffers used to hold the state of completed
tasks/frameworks is very large. However, many frameworks don't care much
about this information when requesting a master's state. Moreover, if a
large number of frameworks request this state simultaneously, the
master can quickly become overwhelmed because the process of generating
this state both blocks the master and takes up a lot of cycles. By
allowing the master to configure the size of the buffers used to hold
this state, we give it the power to significantly reduce the amount of
state it needs to maintain.

This change allows the master to limit the size of this state via
command line flags.

This commit is based on a pull request generated by Felix Bechstein at:

Diffs (updated)

  src/master/constants.hpp f1624e1ed5091f08f57d4382b344cab8b05ba840 
  src/master/constants.cpp 589f2ee55a24d7e8b9352a4c8f94a3785fd1bef4 
  src/master/flags.hpp 9af6c68eef6bcf39d5776809fab6c66dc95da6b2 
  src/master/flags.cpp f864419a6276356c97a0a4fe29e5cfce6c4660c4 
  src/master/master.hpp f764915ff8bbf74b99a617e3c4735d38fc13e5f0 
  src/master/master.cpp 145c5932610bd019eb090947569b608df6815c3a 



On Darwin I launched a master with:

./bin/ --ip= --work_dir=/var/lib/mesos --max_completed_tasks_per_framework=2

and a slave with:

./bin/ --master=

and then ran a bunch of instances of:

./src/test-framework --master=

each of which runs 5 tasks to completion

I then ran:

curl http://localhost:5050/tasks

and verified that only 1 framework and 2 of its completed tasks were given back to me in the
json that was returned.  I repeated this for a number of other configurations with max_completed_frameworks=0..2
and max_completed_tasks_per_framework=0..5 and verified visually that the proper number of
tasks/frameworks were being returned by the /tasks endpoint.


Kevin Klues

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message