-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review146591
-----------------------------------------------------------
This patch really improved the performance a lot for allocator, I did some test with 6000
frameworks and 1000 agents (Add 6000 frameworks first then add 1000 agents), the result is
quite promising! Please take a look at the result of adding 1000 agents, the time decreased
from `4.6 mins` to `49 secs`! It would help a lot for master failover with thousands of frameworks
and agents.
Before fix:
```
[ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/7
Using 1000 agents and 6000 frameworks
Added 6000 frameworks in 702547us
Added 1000 agents in 4.60214636666667mins
```
After fix:
```
[ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.DeclineOffers/7
Using 1000 agents and 6000 frameworks
Added 6000 frameworks in 89707us
Added 1000 agents in 49.351196secs
```
- Guangya Liu
On 八月 23, 2016, 8:49 a.m., Jacob Janco wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
>
> (Updated 八月 23, 2016, 8:49 a.m.)
>
>
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang
Yan Xu.
>
>
> Bugs: MESOS-3157
> https://issues.apache.org/jira/browse/MESOS-3157
>
>
> Repository: mesos
>
>
> Description
> -------
>
> - Triggered allocations dispatch allocate() only
> if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
> cleared when enqueued allocations are processed.
> - Batched allocations are handled synchronously.
>
> - Carrying over work from https://reviews.apache.org/r/41658/ and added the previous
reviewers
> - Specifically, this patch introduces the boolean flag pendingAllocation, which when
set on event
> triggered allocations, will prevent additional no-op allocations: the flag is cleared
when
> the enqueued allocation is processed, subsequent event triggered allocations will update
a set
> of allocation candidates rather than dispatching an additional allocate().
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05
> src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee
>
> Diff: https://reviews.apache.org/r/51027/diff/
>
>
> Testing
> -------
>
> make check
>
> note: check without filters depends on https://reviews.apache.org/r/51028
>
> With new benchmark https://reviews.apache.org/r/49617:
> Sample output without 51027:
> [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
> Using 10000 agents and 3000 frameworks
> Added 3000 frameworks in 57251us
> Added 10000 agents in 3.21345353333333mins
> allocator settled after 1.61236038333333mins
> [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
(290578 ms)
>
> Sample output with 51027:
> [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
> Using 10000 agents and 3000 frameworks
> Added 3000 frameworks in 39817us
> Added 10000 agents in 3.22860541666667mins
> allocator settled after 25.525654secs
> [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
(220137 ms)
>
>
> Thanks,
>
> Jacob Janco
>
>
|