lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [lucenenet] NightOwl888 edited a comment on pull request #325: Fix for LineFileDocs Bottleneck/Performance Improvements
Date Tue, 11 Aug 2020 18:21:47 GMT

NightOwl888 edited a comment on pull request #325:
URL: https://github.com/apache/lucenenet/pull/325#issuecomment-672154841


   I confirmed that the error exists in the master branch, and I got it to occur on `beta00006`
on the second try with a `[Repeat(5000)]` attribute. It does not occur in `beta00005` running
100,000 times.
   
   I also noticed that if the repeat attribute is changed to 10,000 times it will cause an
`OutOfMemoryException`.
   
   I also saw this error occur from the same test:
   
   ```
   
   Test Name:	TestMultiThreadedSnapshotting
   Test FullName:	Lucene.Net.Tests._I-J.Lucene.Net.Index.TestSnapshotDeletionPolicy.TestMultiThreadedSnapshotting
   Test Source:	F:\Projects\lucenenet\src\Lucene.Net.Tests\Index\TestSnapshotDeletionPolicy.cs
: line 353
   Test Outcome:	Failed
   Test Duration:	0:00:00
   
   Test Name:	TestMultiThreadedSnapshotting
   Test Outcome:	Failed
   Result StackTrace:	
   at J2N.Threading.ThreadJob.Join()
      at Lucene.Net.Index.TestSnapshotDeletionPolicy.TestMultiThreadedSnapshotting() in F:\Projects\lucenenet\src\Lucene.Net.Tests\Index\TestSnapshotDeletionPolicy.cs:line
375
   --DebugAssertException
   Result Message:	
   System.Exception : Method <method> failed with 'full flush buffer should be empty:
System.Collections.Generic.List`1[Lucene.Net.Index.DocumentsWriterPerThread]
   ', and was translated to Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException
to avoid terminating the process hosting the test.
   Data:
     OriginalMessage: System.Exception: Method <method> failed with 'full flush buffer
should be empty: System.Collections.Generic.List`1[Lucene.Net.Index.DocumentsWriterPerThread]
   ', and was translated to Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException
to avoid terminating the process hosting the test.
    ---> Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException: Method <method>
failed with 'full flush buffer should be empty: System.Collections.Generic.List`1[Lucene.Net.Index.DocumentsWriterPerThread]
   ', and was translated to Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException
to avoid terminating the process hosting the test.
   
      --- End of inner exception stack trace ---
      at Lucene.Net.Index.TestSnapshotDeletionPolicy.ThreadAnonymousInnerClassHelper2.Run()
in F:\Projects\lucenenet\src\Lucene.Net.Tests\Index\TestSnapshotDeletionPolicy.cs:line 420
      at J2N.Threading.ThreadJob.SafeRun(ThreadStart start)
   
     ----> Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException : Method <method>
failed with 'full flush buffer should be empty: System.Collections.Generic.List`1[Lucene.Net.Index.DocumentsWriterPerThread]
   ', and was translated to Microsoft.VisualStudio.TestPlatform.TestHost.DebugAssertException
to avoid terminating the process hosting the test.
   Result StandardOutput:	
   Culture: ru-KZ
   Time Zone: (UTC+04:00) Astrakhan, Ulyanovsk
   Default Codec: CheapBastard (CheapBastardCodec)
   Default Similarity: RandomSimilarityProvider(queryNorm=False,coord=crazy): {}
   Nightly: False
   Weekly: False
   Slow: True
   Awaits Fix: False
   Directory: random
   Verbose: True
   Random Multiplier: 1
   IFD 1 [11.08.2020 18:10:18; NonParallelWorker]: init: current segments file is ""; deletionPolicy=Lucene.Net.Index.SnapshotDeletionPolicy
   IFD 1 [11.08.2020 18:10:18; NonParallelWorker]: now checkpoint "" [0 segments ; isCommit
= False]
   IFD 1 [11.08.2020 18:10:18; NonParallelWorker]: 1 msec to checkpoint
   IW 1 [11.08.2020 18:10:18; NonParallelWorker]: init: create=True
   IW 1 [11.08.2020 18:10:18; NonParallelWorker]: 
   dir=MockDirectoryWrapper(RAMDirectory@548c1d lockFactory=Lucene.Net.Store.SingleInstanceLockFactory)
   index=
   version=4.8.0
   matchVersion=LUCENE_48
   analyzer=MockAnalyzer
   ramBufferSizeMB=16
   maxBufferedDocs=493
   maxBufferedDeleteTerms=-1
   mergedSegmentWarmer=
   readerTermsIndexDivisor=2
   termIndexInterval=58
   delPolicy=SnapshotDeletionPolicy
   commit=null
   openMode=CREATE_OR_APPEND
   similarity=RandomSimilarityProvider
   mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=1, maxMergeCount=2, mergeThreadPriority=-1
   default WRITE_LOCK_TIMEOUT=1000
   writeLockTimeout=1000
   codec=CheapBastard
   infoStream=ThreadNameFixingPrintStreamInfoStream
   mergePolicy=[LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=True, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8796093022207,999, noCFSRatio=1]
   indexerThreadPool=Lucene.Net.Index.ThreadAffinityDocumentsWriterThreadPool
   readerPooling=True
   perThreadHardLimitMB=1945
   useCompoundFile=True
   checkIntegrityAtMerge=False
   
   IW 1 [11.08.2020 18:10:18; t6]: commit: start
   IW 1 [11.08.2020 18:10:18; t5]: commit: start
   IW 1 [11.08.2020 18:10:18; t2]: commit: start
   IW 1 [11.08.2020 18:10:18; t6]: commit: enter lock
   IW 1 [11.08.2020 18:10:18; t1]: commit: start
   IW 1 [11.08.2020 18:10:18; t3]: commit: start
   IW 1 [11.08.2020 18:10:18; t6]: commit: now prepare
   IW 1 [11.08.2020 18:10:18; t9]: commit: start
   IW 1 [11.08.2020 18:10:18; t7]: commit: start
   IW 1 [11.08.2020 18:10:18; t8]: commit: start
   IW 1 [11.08.2020 18:10:18; t0]: commit: start
   IW 1 [11.08.2020 18:10:18; t4]: commit: start
   IW 1 [11.08.2020 18:10:18; t6]: prepareCommit: flush
   IW 1 [11.08.2020 18:10:18; t6]:   index before flush 
   DW 1 [11.08.2020 18:10:18; t6]: startFullFlush
   DW 1 [11.08.2020 18:10:18; t6]: anyChanges? numDocsInRam=10 deletes=False hasTickets:False
pendingChangesInFullFlush: False
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0,
segment=_0, aborting=False, numDocsInRAM=2, deleteQueue=DWDQ: [ generation: 0 ]]
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0,
segment=_1, aborting=False, numDocsInRAM=2, deleteQueue=DWDQ: [ generation: 0 ]]
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0,
segment=_6, aborting=False, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 0 ]]
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0,
segment=_2, aborting=False, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 0 ]]
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0,
segment=_3, aborting=False, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 0 ]]
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0,
segment=_4, aborting=False, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 0 ]]
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0,
segment=_5, aborting=False, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 0 ]]
   DWFC 1 [11.08.2020 18:10:18; t6]: addFlushableState Docume< Truncated >
   
   
   
   ```
   
   I have opened #326 to put a higher priority on this task - we need to get the "asserts"
working in Release mode to ensure that our test suite is testing everything that it is supposed
to so more issues like this don't creep in without us being aware of them.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message