lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [lucenenet] NightOwl888 opened a new issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
Date Wed, 24 Jun 2020 18:12:43 GMT

NightOwl888 opened a new issue #295:
URL: https://github.com/apache/lucenenet/issues/295


   As part of #261, here is a list of known tests in the `Lucene.Net.Util` namespace that
are taking significantly longer in .NET than they did in Java. The percentage is the amount
of time that .NET takes compared to Java (for example, if the .NET test is running 10x longer
the percentage will be 1000% of the Java test).
   
   These tests need to be investigated to determine what is causing the slowness. Fixing these
issues will likely ripple across the entire project reducing the time the tests take to run
as well as impacting end users.
   
   ## Slow Running Tests
   
   - [ ] Lucene.Net.Util.TestDocIdBitSet.TestAgainstBitSet	(11583% - <b>Now 759%</b>)
   - [ ] Lucene.Net.Util.TestMathUtil.TestAcoshMethod	(7400% - <b>Now 900%</b>)
   - [x] Lucene.Net.Facet.Taxonomy.WriterCache.TestCharBlockArray.TestArray	(5498% - <b>Now
238%</b>)
   - [ ] Lucene.Net.Util.TestAttributeSource.TestCaptureState	(4850% - <b>Now 1150%</b>)
   - [ ] Lucene.Net.Util.TestBroadWord.TestPerfSelectAllBitsBroad	(4267% - <b>Now 1000%</b>)
   - [x] Lucene.Net.Util.TestFixedBitSet.TestAgainstBitSet	(2999% - <b>Now 130%</b>)
   - [ ] Lucene.Net.Util.Automaton.TestBasicOperations.TestEmptyLanguageConcatenate	(2750%)
   - [x] Lucene.Net.Util.TestRecyclingByteBlockAllocator.TestAllocate	(2333% - <b>Now
33%</b>)
   - [ ] Lucene.Net.Util.TestPackedInts.TestAppendingLongBuffer	(1597%)
   - [ ] Lucene.Net.Util.TestIdentityHashSet.TestCheck	(1176%)
   - [ ] Lucene.Net.Util.TestPagedBytes.TestDataInputOutput	(1015%)
   - [ ] Lucene.Net.Util.TestByteBlockPool.TestReadAndWrite	(1008%)
   - [ ] Lucene.Net.Util.TestWAH8DocIdSet.TestAgainstBitSet	(895%)
   - [ ] Lucene.Net.Util.TestPagedBytes.TestDataInputOutput2	(849%)
   - [ ] Lucene.Net.Util.TestBasicOperations.TestEmptySingletonConcatenate	(775%)
   - [ ] Lucene.Net.Util.TestBytesRefHash.TestSize	(762%)
   - [ ] Lucene.Net.Util.TestOpenBitSet.TestAgainstBitSet	(720%)
   - [ ] Lucene.Net.Util.TestNumericUtils.TestRandomSplit	(708%)
   - [ ] Lucene.Net.Util.Fst.TestBytesStore.TestRandom	(681%)
   - [ ] Lucene.Net.Util.Packed.TestEliasFanoSequence.TestAdvanceToAndBackToMultiples	(555%)
   - [x] Lucene.Net.Util.TestOfflineSorter.TestSmallRandom	(555% - <b>Now 99%</b>)
   - [ ] Lucene.Net.Util.TestBytesRefHash.TestSort	(414%)
   - [ ] Lucene.Net.Util.TestBytesRefHash.TestGet	(343%)
   - [ ] Lucene.Net.Util.TestBytesRefHash.TestAddByPoolOffset	(331%)
   - [ ] Lucene.Net.Util.TestUnicodeUtil.TestCodePointCount	(294%)
   - [ ] Lucene.Net.Util.Fst.TestFsts.TestRandomWords	(289%)
   - [ ] Lucene.Net.Util.TestBytesRefHash.TestFind	(284%)
   - [ ] Lucene.Net.Util.TestBytesRefHash.TestCompact	(273%)
   - [ ] Lucene.Net.Util.TestUnicodeUtil.TestUTF8toUTF32	(256%)
   - [x] Lucene.Net.Util.TestOfflineSorter.TestIntermediateMerges	(242% - <b>Now 76%</b>)
   - [x] Lucene.Net.Util.TestOpenBitSet.TestSmall	(188% - <b>Now 136%</b>)
   
   ---------------
   
   > These tests were not apples-apples comparisons, the Java tests were run on an older
and slower machine that was outclassed in RAM, CPU, and disk read/write speeds by the machine
that was used to test in .NET. The percentage is the approximate value that was measured by
the test frameworks, but is probably significantly lower than the difference using a proper
benchmark on the same machine.
   
   ### BitSet vs BitArray
   
   The `TestAgainstBitSet` test is slow in almost every test that utilizes it.
   
   One potential reason for this is that we are using the .NET `BitArray` class and in Java
Lucene was using a `BitSet`. While remapping to .NET APIs is the preferred way of porting,
in this case we have classes with completely different behaviors. `BitArray` is a fixed size,
and `BitSet` automatically expands as needed. To make `BitArray` expand, we need to reallocate
it, and those extra allocations may be contributing to these tests being slow.
   
   Note that J2N has [a C# implementation of `BitSet`](https://github.com/NightOwl888/J2N/blob/1ace2b3ad498501f9ea36bc277b353b54ddfc588/src/J2N/Collections/BitSet.cs)
that would be acceptable to use instead of `BitArray` if this is indeed one of the contributors.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message