mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guangya Liu" <gyliu...@gmail.com>
Subject Re: Review Request 40632: Enabled oversubscribed resources for reservations in allocator.
Date Tue, 19 Jan 2016 07:03:29 GMT


> On 十二月 30, 2015, 12:42 p.m., Klaus Ma wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1321
> > <https://reviews.apache.org/r/40632/diff/8/?file=1178313#file1178313line1321>
> >
> >     As framework can not use ALLOCATION_SLACK mixed with other resources, allocator
need to check seperatly.
> 
> Guangya Liu wrote:
>     Yes, currently the validation logic in master is https://github.com/apache/mesos/blob/master/src/master/validation.cpp#L479-L488
which can make sure a task and its executor can either use non-revocable resources or revocable
resources of a given name but not both. After we introduced the allocation slack, we may want
to guard that a task and its executor can either use allocation slack or usage slack or non-revocable
resources but not all. I will upload a patch to handle this part.
> 
> Guangya Liu wrote:
>     Fixed in https://reviews.apache.org/r/41848/
> 
> Joseph Wu wrote:
>     Note: mixing resources is allowed.  Everything in a container is evicted at once,
including non-revocable resources mixed into a revocable one.
> 
> Klaus Ma wrote:
>     re 'Note: mixing resources is allowed.', my understanding is that: different tasks
can use different resources, but one task can NOT use mixing resources; for example, one task
can not use both `USAGE_SLACK` and `ALLOCATION_SLACK`.

It is an old issue, this problem also exist with usage slack. If the usage slack plus non
revocable resources, the offer will also be send. Will enhance `allocatable` to update it.


> On 十二月 30, 2015, 12:42 p.m., Klaus Ma wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1302
> > <https://reviews.apache.org/r/40632/diff/8/?file=1178313#file1178313line1302>
> >
> >     If framework has reserved.statless resources, allocator should not dispatch
it as ALLOCATION_SLACK.
> 
> Klaus Ma wrote:
>     @Joseph, I can not remenber where did you append your comments about this topic;
but it seems you're OK to offer both `stateless.reserved` & `ALLOCATION_SLACK` of the
same role, right? But how framework developer distinguish `stateless.reserved` & `ALLOCATION_SLACK`
of its? If it launch task in `ALLOCATION_SLACK` and `stateless.reserved`, the task in `ALLOCATION_SLACK`
was terminated immediately; it just increases overhead of cluster instead of resource utilization.
> 
> Joseph Wu wrote:
>     There's something about this in the design doc.
>     
>     The summary is: Meta-frameworks may benefit from receiving allocation slack from
itself.
> 
> Klaus Ma wrote:
>     OK, would you help to update the design doc? There's comments that:
>     
>     NOTE: We need to be careful not to offer the same resources (as reserved resources
and as revocable resources) to the same framework. This could presumably happen if a Lender
Framework also accepts revocable resources.
> 
> Guangya Liu wrote:
>     I've already handled this case here: When sending offers, the allocation slack resources
in the offer need to exclude all allocated stateless reserved resources and allocated allocation
slack resources.
> 
> Klaus Ma wrote:
>     I think current code did not handle this case; if a Lender framekwork accepts revocable
resources, the framework's `stateless.reserved` resource & allocation slack from that
`stateless.reserved` will send to that famework; so "offer the same resources (as reserved
resources and as revocable resources) to the same framework".

Filed a bug https://issues.apache.org/jira/browse/MESOS-4426


- Guangya


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40632/#review112309
-----------------------------------------------------------


On 一月 16, 2016, 7:51 a.m., Guangya Liu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40632/
> -----------------------------------------------------------
> 
> (Updated 一月 16, 2016, 7:51 a.m.)
> 
> 
> Review request for mesos, Ben Mahler, Artem Harutyunyan, Joris Van Remoortere, Joseph
Wu, Klaus Ma, and Jian Qiu.
> 
> 
> Bugs: MESOS-4145
>     https://issues.apache.org/jira/browse/MESOS-4145
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Enabled oversubscribed resources for reservations in allocator.
> The allocator part including 5 patches:
> 1) https://reviews.apache.org/r/40632 Enabled oversubscribed resources for reservations
in allocator
> 2) https://reviews.apache.org/r/41847 Updated allocation slack when slave was updated.
> 3) https://reviews.apache.org/r/41791 Updated allocation slack for dynamic reserve (1/3).
> 4) https://reviews.apache.org/r/42113 Handle unreserve logic for dynamic reservation
(2/3).
> 5) https://reviews.apache.org/r/42194 Handle unreserve logic for dynamic reservation
(3/3).
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.cpp 72e69a0f42dd724713f2a7a75f1b92ef16eb5569

>   src/tests/hierarchical_allocator_tests.cpp 9362dd306497ba01e0f387c3862456cdcac6f863

> 
> Diff: https://reviews.apache.org/r/40632/diff/
> 
> 
> Testing
> -------
> 
> make
> make check
> GLOG_v=2 ./bin/mesos-tests.sh  --gtest_filter="HierarchicalAllocatorTest.*" --verbose
> 
> 
> Thanks,
> 
> Guangya Liu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message