mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Mahler <benjamin.mah...@gmail.com>
Subject Re: Review Request 43884: Added allocator metrics for used quotas.
Date Thu, 17 Mar 2016 22:58:00 GMT


> On March 17, 2016, 9:07 p.m., Ben Mahler wrote:
> > docs/monitoring.md, lines 876-883
> > <https://reviews.apache.org/r/43884/diff/16/?file=1299847#file1299847line876>
> >
> >     Thanks! How about:
> >     
> >     ```
> >     <tr>
> >       <td>
> >       <code>allocator/mesos/quota/<role>/<resource>_allocated</code>
> >       </td>
> >       <td>Amount of resources considered allocated towards a role's quota
guarantee</td>
> >       <td>Gauge</td>
> >     </tr>
> >     ```
> >     
> >     Also, can you add the guarantee to this patch?
> >     
> >     ```
> >     <tr>
> >       <td>
> >       <code>allocator/mesos/quota/<role>/<resource>_guarantee</code>
> >       </td>
> >       <td>Amount of resources guaranteed for a role via quota</td>
> >       <td>Gauge</td>
> >     </tr>
> >     ```
> >     
> >     Then we can easily monitor and alert on the distance between the two, and you
can imagine future additions to enable better visibility into the system, for example:
> >     
> >     ```
> >     allocator/mesos/quota/role/cpus_guarantee
> >     allocator/mesos/quota/role/cpus_allocated
> >     allocator/mesos/quota/role/cpus_filtered
> >     allocator/mesos/quota/role/cpus_unsatisfiable
> >     ```
> 
> Ben Mahler wrote:
>     Sorry one more thought here. After chatting with vinod about our metric naming mistakes
we made in the past (e.g. master metrics), we're thinking the following would be better here:
>     
>     Per-role metrics:
>     `"allocator/mesos/quota/roles/<role>/resources/<resource>/guarantee"`
>     
>     Aggregated across-role metrics:
>     `"allocator/mesos/quota/resources/<resource>/guarantee"`
>     
>     For dynamically generated metrics, we now prefer using an explicit delimiter (like
`"roles/"`) to delimit metric names when there is a potential for additional metrics to be
added or aggregated versions are needed. Let's say we want to add a quota metric that isn't
tied to resources, we can easily do so. For example, if we added priorities we could easily
add `"allocator/mesos/quota/priority/1"` without the potential of `"priority"` colliding with
a role (thanks to the `"roles/"` and `"resources"` delimiter. For the aggregated case, clearly
we'll have a conflict with my original suggestion above if one were to set quota for a role
called `"cpus"`.
>     
>     Thoughts?

And another thought :)

I'd like to avoid using the word "allocated" here to indicate both offered and allocated,
because in the future (when the allocator understands offers), we will want to distinguish
the two. After chatting with Vinod about this one, we're thinking the following is the best
way to deal with this for now:

`"allocator/mesos/quota/roles/<role>/resources/<resource>/offered_or_allocated"`

Later, we'll remove this in favor of having two seperate metrics:

`"allocator/mesos/quota/roles/<role>/resources/<resource>/allocated"`
`"allocator/mesos/quota/roles/<role>/resources/<resource>/offered"`

If we added `"allocated"` now, splitting the metrics would be a backwards incompatible change
because we would be changing the meaning of `"allocated"`.


- Ben


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43884/#review124061
-----------------------------------------------------------


On March 15, 2016, 2:51 p.m., Benjamin Bannier wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43884/
> -----------------------------------------------------------
> 
> (Updated March 15, 2016, 2:51 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Ben Mahler.
> 
> 
> Bugs: MESOS-4723
>     https://issues.apache.org/jira/browse/MESOS-4723
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Added allocator metrics for used quotas.
> 
> 
> Diffs
> -----
> 
>   docs/monitoring.md 827f7073204fcf8575ca980a5571c8be4f5e4110 
>   src/master/allocator/mesos/hierarchical.hpp 52b3a9bfbe73d24968f7ddb0672aee1e05636ab0

>   src/master/allocator/mesos/hierarchical.cpp 70291075c00a9a557529c2562dedcfc6c6c3ec32

>   src/master/allocator/mesos/metrics.hpp d04e9a11c77b6fb2d522608e3085f81f8a6657ca 
>   src/master/allocator/mesos/metrics.cpp 46dd7f7b3d8d49c59b42e789fe5dcd49f1693ace 
>   src/tests/hierarchical_allocator_tests.cpp 459e02576f6d05abbbcc83ae5cabac5c66e93f05

> 
> Diff: https://reviews.apache.org/r/43884/diff/
> 
> 
> Testing
> -------
> 
> make check (OS X)
> 
> I confirmed that this does not lead to general performance regressions in the allocator;
this is partially expected since the added code only inserts metrics in the allocator while
the actual work is perform asynchronously. These tests where performed with `HierarchicalAllocator_BENCHMARK_Test.DeclineOffers`
on an optimized build under OS X using clang(trunk) as compiler.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message