mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Janco <jjanco....@gmail.com>
Subject Re: Review Request 51027: Track allocation candidates to bound allocator.
Date Thu, 01 Sep 2016 18:54:07 GMT


> On Aug. 26, 2016, 6:24 a.m., Klaus Ma wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 275
> > <https://reviews.apache.org/r/51027/diff/2/?file=1481820#file1481820line275>
> >
> >     I think we should use `insert(...)` instead of `=`; if the following event in
the queue, are we still going to get the correct result?
> >     ```
> >     batch -> addFramework(f1) -> addFramework(f2)
> >     ```
> 
> Jacob Janco wrote:
>     insert in a loop?
> 
> Klaus Ma wrote:
>     No; if multiple frameworks register at almost the same time, will there be multiple
addFramework events pending in the queue? Did not get a chance to test it, but did we consider
such kind of race condition?

If I understand this correctly: batch -> addFramework(f1) -> addFramework(f2) will ensure
an allocate() at batch over all known slaves. If the two addFramework events come in close
succession then the queue should look like 1: addFramework(f1) -> addFramework(f2) 2: addFramework(f2)
-> allocate() 3: allocate() There will be a pendingAllocation set for the first event with
the dispatched allocate() after addFramework(f2). I'm not sure what the race condition would
be as we can have many addFramework events queued.


- Jacob


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review146919
-----------------------------------------------------------


On Aug. 30, 2016, 4:53 p.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
> 
> (Updated Aug. 30, 2016, 4:53 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang
Yan Xu.
> 
> 
> Bugs: MESOS-3157
>     https://issues.apache.org/jira/browse/MESOS-3157
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Triggered allocations dispatch allocate() only
>   if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
>   cleared when enqueued allocations are processed.
> - Batched allocations are handled synchronously.
> 
> 	 	
> - Carrying over work from 
>   https://reviews.apache.org/r/41658/ and added 
>   the previous reviewers
> - Specifically, this patch introduces the boolean 
>   flag pendingAllocation, which when set on event 
>   triggered allocations, will prevent additional 
>   no-op allocations: the flag is cleared when the 
>   enqueued allocation is processed, subsequent 
>   event triggered allocations will update a set
>   of allocation candidates rather than dispatching 
>   an additional allocate().
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05

>   src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee

> 
> Diff: https://reviews.apache.org/r/51027/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> note: check without filters depends on https://reviews.apache.org/r/51028
> 
> With new benchmark https://reviews.apache.org/r/49617: 
> Sample output without 51027:
> [ RUN      ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
> Using 10000 agents and 3000 frameworks
> Added 3000 frameworks in 57251us
> Added 10000 agents in 3.21345353333333mins
> allocator settled after  1.61236038333333mins
> [       OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
(290578 ms)
> 
> Sample output with 51027:
> [ RUN      ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
> Using 10000 agents and 3000 frameworks
> Added 3000 frameworks in 39817us
> Added 10000 agents in 3.22860541666667mins
> allocator settled after  25.525654secs
> [       OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
(220137 ms)
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message