mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chun-Hung Hsiao <chhs...@apache.org>
Subject Re: Review Request 70132: Do not implicitly refuse speculatively converted resources.
Date Wed, 06 Mar 2019 04:41:33 GMT


> On March 6, 2019, 2:29 a.m., Meng Zhu wrote:
> > src/master/master.cpp
> > Lines 5921-5928 (original), 5928-5937 (patched)
> > <https://reviews.apache.org/r/70132/diff/1/?file=2129042#file2129042line5928>
> >
> >     hmm, I wonder if it is better to also not implicitly filter `unusedResources`
if `speculativeResources` is not empty.
> >     
> >     If a framework wants to consume the updated resources e.g. volume, it probably
also wants other resources such as cpu/mem (unless it can consume disk only resource). In
that case, the above code would not improve the situation because the framework still needs
to wait for the filtered cpu/mem. Also I am a little uncomfortable that, we now have *two*
recoverResources calls which could increase agent resource fragmentation (if an offer is sent
out in between).
> >     
> >     what do you think?

We have two problems here:

1. A framework may still wait for the unused resources which get filters.
This won't be a problem since the converted resources will enlarge the resource pool. As a
result, the "filtered" unused resources will come back along with the converted ones in the
next allocation.

2. `recoverResources(...converted..., None())` -> allocation -> `recoverResources(...unused...,
filters)` could lead to resource fragmentation.
How about doing the following?
```
allocator->pause();
allocator->recoverResources(...converted..., None());
allocator->recoverResources(...unused..., filters);
allocator->resume();
```


- Chun-Hung


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70132/#review213453
-----------------------------------------------------------


On March 6, 2019, 12:53 a.m., Chun-Hung Hsiao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70132/
> -----------------------------------------------------------
> 
> (Updated March 6, 2019, 12:53 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-9616
>     https://issues.apache.org/jira/browse/MESOS-9616
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Currently if a framework accepts an offer to perform pipelined
> operations, e.g., reserving resource, without a final consumer, the
> converted resources will be implicitly refused. This is an undesired
> behavior as the framework might want to reserve one resource first but
> launch a task later in the next allocation cycle. This patch fixes this
> behavior.
> 
> But, if the framework accepts an offers with multiple operations that
> cancel out each other, the resources consumed by these operations are
> still considered unused and will be refused.
> 
> 
> Diffs
> -----
> 
>   docs/scheduler-http-api.md 8384336bbecf2ca38a3cd203f9db28d931812d65 
>   src/master/master.cpp 015da54583448a8d102d8e401e48bd228baf6dd6 
>   src/tests/slave_tests.cpp 22a0295086ae4f4ec26df00a0e077eecfa27f1fb 
> 
> 
> Diff: https://reviews.apache.org/r/70132/diff/1/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message