mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Klues" <>
Subject Re: Review Request 42053: Added flags to set size of completed task/framework history.
Date Wed, 13 Jan 2016 10:42:36 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Jan. 13, 2016, 10:42 a.m.)

Review request for mesos and Ben Mahler.


Missed one change from flags->masterFlags per Adam's previous comments

Bugs: MESOS-3307

Repository: mesos


The default size of the buffers used to hold the state of completed
tasks/frameworks is very large. However, many frameworks don't care much
about this information when requesting a master's state. Moreover, if a
large number of frameworks request this state simultaneously, the
master can quickly become overwhelmed because the process of generating
this state both blocks the master and takes up a lot of cycles. By
allowing the master to configure the size of the buffers used to hold
this state, we give it the power to significantly reduce the amount of
state it needs to maintain.

This change allows the master to limit the size of this state via
command line flags.

This commit is based on a pull request generated by Felix Bechstein at:


Diffs (updated)

  docs/ 7e0eb9584f3cb24a388c44d7bc81cd87dbb4072e 
  src/master/constants.hpp ebab341e58035d4b579828add752c1ee37efeb95 
  src/master/constants.cpp 77dd31430776e4f24e6e074c1470edcb19e58449 
  src/master/flags.hpp d923b1b0444d7e9023f1db4cbc4f7d4b84c20ff5 
  src/master/flags.cpp 88909590ff421421659e6faac7f3444bdc57b630 
  src/master/master.hpp f02d165874fa8023675e545793de699aeecae29b 
  src/master/master.cpp c122c30d943813fc3ce9e7025783c7231809b022 



On Darwin I launched a master with:

./bin/ --ip= --work_dir=/var/lib/mesos --max_completed_tasks_per_framework=2

and a slave with:

./bin/ --master=

and then ran a bunch of instances of:

./src/test-framework --master=

each of which runs 5 tasks to completion

I then ran:

curl http://localhost:5050/tasks

and verified that only 1 framework and 2 of its completed tasks were given back to me in the
json that was returned.  I repeated this for a number of other configurations with max_completed_frameworks=0..2
and max_completed_tasks_per_framework=0..5 and verified visually that the proper number of
tasks/frameworks were being returned by the /tasks endpoint.

There is alos now a unit test for this as reviewed here:


Kevin Klues

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message