mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anindya Sinha <anindya_si...@apple.com>
Subject Re: Review Request 51879: Determine disk size when not specified in static resources.
Date Mon, 26 Sep 2016 06:50:56 GMT


> On Sept. 22, 2016, 6:02 p.m., Jiang Yan Xu wrote:
> > src/slave/containerizer/containerizer.cpp, lines 275-280
> > <https://reviews.apache.org/r/51879/diff/4/?file=1506819#file1506819line275>
> >
> >     Mulling it over, I think we can achieve the consistency of "honoring explicitly
supplied values by the operator even if it's zero" by removing validation from the parsing
functions except for those that return `Resources`. i.e., 
> >     
> >     If `try<vector<Resource>> fromJSONString()` and `try<vector<Resource>>
fromSimpleString()` do not drop `Resource` objects that couldn't pass `Resources::validate(const
Resource& resource)` check, here we can identity the disk Resources that do not have the
value specified and fill in the auto-detected value. Eventually we will turn them into `Resources`
and the validation can happen there.
> >     
> >     If we do this, then the operator can specify the mount disk without the value
to have it auto-detected.
> >     
> >     ```
> >     {
> >       "name" : "disk",
> >       "type" : "SCALAR",
> >       "disk" : {
> >         "source" : {
> >           "type" : "MOUNT",
> >           "mount" : { "root" : "/mnt/data" }
> >         }
> >       }
> >     }
> >     ```
> >     
> >     (Frankly we can auto-detect the "type" as well but allowing it to be specified
is fine too)
> >     
> >     
> >     This allows us to fix the issue for other kinds of resources too (MESOS-6219).

SGTM. We have the following 3 cases now for each resource type (eg. `cpus`):
1) `cpus:0` => Do not include `cpus` in agent's resources.
2) `cpus` with no value specified (indicated by `cpus:` in text format) => Ask agent to
auto detect amount of `cpus`.
3) No `cpus` specified in the `flags.resources` => Ask agent to auto detect amount of `cpus`.

We should be able to extend this to all known resource types: `cpus`, `mem`, `disk` (root
disks and PATH/MOUNT disks) and `ports`. Still need to look at `gpus` a little more.

And ensured all existing call sites for `Resources::parse()` have the same expectation as
today, ie. returns a `Resources` object with no empty and invalid resources.


- Anindya


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51879/#review150030
-----------------------------------------------------------


On Sept. 21, 2016, 4:15 a.m., Anindya Sinha wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51879/
> -----------------------------------------------------------
> 
> (Updated Sept. 21, 2016, 4:15 a.m.)
> 
> 
> Review request for mesos and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6062
>     https://issues.apache.org/jira/browse/MESOS-6062
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> When static resources indicate disks with a positive size, we use that
> for the disk resources on the agent. However, --resources can include
> disks with size of 0, which indicates that mesos agent determine the
> size of those disks from the host and uses that information instead.
> Note that this is not allowed for PATH disks since PATH disks can be
> carved into smaller chunks and we cannot assume that the whole
> physical disk is available to the PATH disk.
> 
> With this change, JSON or textual representation for disk resources
> that specify scalar value of 0 would not result in an error, but those
> resources will not be accounted for until a valid size is determined
> for such resources. A scalar value of -1 in JSON or textual formats
> still results in an invalid resource.
> 
> 
> Diffs
> -----
> 
>   src/slave/containerizer/containerizer.cpp d46882baa904fd439bffb23c324828b777228f1c

> 
> Diff: https://reviews.apache.org/r/51879/diff/
> 
> 
> Testing
> -------
> 
> Tests passed.
> 
> 
> Thanks,
> 
> Anindya Sinha
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message