mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiang Yan Xu <...@jxu.me>
Subject Re: Review Request 49571: Added a benchmark test for allocations and shared resources.
Date Tue, 19 Jul 2016 21:45:42 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49571/#review142771
-----------------------------------------------------------



Commented on the resources benchmarks. Also let's pull it out then we can hopefully commit
the arithmetic operations for shared resources patch first.

On the separate review could you post the numbers from cout instead of the total test time?

Overall I think this benchmark is sufficient for a first cut while we are thinking about more
sophisticated tests that evaluate the performance of shared persistent volumes fairly.

In the new review perhaps add klaus1982 as the review as they are writing benchmarks for regular
DiskInfo/Persistent Volumes (I pinged @klaus1982 on #allocator channel on slack)

If they haven't done so, we can add a benchmark for regular persistent volumes: add up (e.g.,)
5000 distinct persistent volumes together and see the performance. But then, what we can achieve
is to compare the performance of regular persistent volumes with and without the new patch:
to verify that the new patch doesn't lead to degradation of regular persistent volume arithmetic.


src/tests/resources_tests.cpp (lines 2592 - 2603)
<https://reviews.apache.org/r/49571/#comment208400>

    Can we use `createPersistentVolume()` to create the `disk` directly?


- Jiang Yan Xu


On July 18, 2016, 7:31 a.m., Anindya Sinha wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49571/
> -----------------------------------------------------------
> 
> (Updated July 18, 2016, 7:31 a.m.)
> 
> 
> Review request for mesos and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-5771
>     https://issues.apache.org/jira/browse/MESOS-5771
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Allocations test has the following configurations:
> (1) REGULAR: Offers from every slave have regular resources.
> (2) SHARED: Offers from every slave include a shared resource.
> (3) REGULAR: Offers from every alternate slave contain only regular
>     resources; and offers from every other alternate slave contains
>     a shared resource.
> 
> Added a case for testing arithmetic operations on shared resources.
> 
> 
> Diffs
> -----
> 
>   src/tests/hierarchical_allocator_tests.cpp ce5da6be490b6fce311286eb4018c91eef55067e

>   src/tests/resources_tests.cpp 40d290ac540d26373c5fb7c2a93d27d1aa61d722 
> 
> Diff: https://reviews.apache.org/r/49571/diff/
> 
> 
> Testing
> -------
> 
> All tests passed.
> 
> Resource arithmetic benchmark test results
> ==========================================
> Minimal impact seen in Resources arithmetic with the Resources refactor changes to incorporate
shared resources.
> 
> With shared resources patch (note that 4th test below is for shared resources for scalars)
> ------------------------------------------------------------------------------------------
> 
> [----------] 4 tests from ResourcesOperators/Resources_BENCHMARK_Test
> [ RUN      ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/0
> [       OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/0 (980 ms)
> [ RUN      ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/1
> [       OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/1 (17128 ms)
> [ RUN      ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2
> [       OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2 (9217 ms)
> [ RUN      ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/3
> [       OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/3 (1100 ms)
> [----------] 4 tests from ResourcesOperators/Resources_BENCHMARK_Test (28425 ms total)
> 
> HEAD
> ----
> 
> [----------] 3 tests from ResourcesOperators/Resources_BENCHMARK_Test
> [ RUN      ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/0
> [       OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/0 (866 ms)
> [ RUN      ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/1
> [       OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/1 (17563 ms)
> [ RUN      ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2
> [       OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2 (9218 ms)
> [----------] 3 tests from ResourcesOperators/Resources_BENCHMARK_Test (27647 ms total)
> 
> Allocations benchmark test results
> ==================================
> Support of shared resources has a small impact on runtime performance in allocations.
Also, there is no visible impact in performance when shared resources are added in the tests.
> 
> With the patch (and no shared resources)
> ----------------------------------------
> round 0 allocate took 3.19704secs to make 200 offers
> round 50 allocate took 3.240605secs to make 200 offers
> round 100 allocate took 3.227024secs to make 200 offers
> round 150 allocate took 3.225281secs to make 200 offers
> round 199 allocate took 3.26036secs to make 200 offers
> 
> With the patch (and shared resources on all agents)
> ---------------------------------------------------
> round 0 allocate took 3.279115secs to make 200 offers
> round 50 allocate took 3.273396secs to make 200 offers
> round 100 allocate took 3.278509secs to make 200 offers
> round 150 allocate took 3.275959secs to make 200 offers
> round 199 allocate took 3.278151secs to make 200 offers
> 
> With the patch (and shared resources on alternate agents)
> ---------------------------------------------------------
> round 0 allocate took 3.251739secs to make 200 offers
> round 50 allocate took 3.263777secs to make 200 offers
> round 100 allocate took 3.263079secs to make 200 offers
> round 150 allocate took 3.263114secs to make 200 offers
> round 199 allocate took 3.236228secs to make 200 offers
> 
> Based on HEAD, with all regular resources (no shared resources in HEAD supported)
> ---------------------------------------------------------------------------------
> round 0 allocate took 2.925681secs to make 200 offers
> round 50 allocate took 2.922036secs to make 200 offers
> round 100 allocate took 2.909337secs to make 200 offers
> round 150 allocate took 2.914093secs to make 200 offers
> round 199 allocate took 2.923762secs to make 200 offers
> 
> 
> Thanks,
> 
> Anindya Sinha
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message