lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [lucenenet] Shazwazza opened a new pull request #310: WIP Benchmarks between nuget versions
Date Wed, 15 Jul 2020 04:35:23 GMT

Shazwazza opened a new pull request #310:
URL: https://github.com/apache/lucenenet/pull/310


   This is a WIP for creating benchmarkdotnet tests to see performance (currently for testing
between nuget packages).
   
   I have asked about specifying different nuget package sources here https://github.com/dotnet/BenchmarkDotNet/pull/922#issuecomment-658519567
(I feel like I had done this in some way when implementing that feature but can't remember
now) since that will be needed to test the 'current' version. It is not possible to reference
the 'current' projects and then also use 'WithNuget' because that results in Nuget errors.
   
   Currently just added a single benchmark as an example, this is just the script from the
home page. There's manual iterations specified in there so that we don't get warnings from
BenchmarkDotNet about `MultimodalDistribution` or `MinIterationTime` since one less iterations
are too fast to measure correctly. 
   
   This example benchmark isn't showing much since the results are nearly identical between
versions, for example: 
   
   ``` ini
   
   BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18363.900 (1909/November2018Update/19H2)
   Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
   .NET Core SDK=3.1.400-preview-015151
     [Host]          : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64
RyuJIT
     4.8.0-beta00007 : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64
RyuJIT
     4.8.0-beta00008 : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64
RyuJIT
     4.8.0-beta00009 : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64
RyuJIT
   
   IterationCount=15  LaunchCount=2  WarmupCount=10  
   
   ```
   |         Method |             Job |                            NuGetReferences |     Mean
|   Error |  StdDev |      Gen 0 |     Gen 1 |     Gen 2 | Allocated |
   |--------------- |---------------- |------------------------------------------- |---------:|--------:|--------:|-----------:|----------:|----------:|----------:|
   | HomePageScript | 4.8.0-beta00007 | Lucene.Net.Analysis.Common 4.8.0-beta00007 | 207.5
ms | 3.94 ms | 5.65 ms | 63000.0000 | 6000.0000 | 1000.0000 | 249.37 MB |
   | HomePageScript | 4.8.0-beta00008 | Lucene.Net.Analysis.Common 4.8.0-beta00008 | 196.8
ms | 6.20 ms | 9.28 ms | 60000.0000 | 9000.0000 | 2000.0000 | 250.74 MB |
   | HomePageScript | 4.8.0-beta00009 | Lucene.Net.Analysis.Common 4.8.0-beta00009 | 208.8
ms | 7.15 ms | 9.54 ms | 62000.0000 | 7000.0000 | 2000.0000 | 250.77 MB |
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message