lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shad Storhaug (Jira)" <j...@apache.org>
Subject [jira] [Updated] (LUCENENET-619) Known Failing Tests on Lucene.Net
Date Tue, 03 Sep 2019 21:05:00 GMT

     [ https://issues.apache.org/jira/browse/LUCENENET-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shad Storhaug updated LUCENENET-619:
------------------------------------
    Description: 
Up for grabs. Please contribute to Lucene.Net.

All 8000+ tests pass most of the time, but there are still a few that fail under the random
conditions we test them in. This is a complete list of the known tests that fail and what
is known about them. Note that these may not be all of the ways the tests can fail.
 # {color:#d04437}*-Lucene.Net.Search.TestSearchAfter::TestQueries()-*{color}
 ** Related to current culture
 ** Happens on netcoreapp2.1 on macOS
 ** Happens on netcoreapp1.0 on Linux
 # {color:#d04437}*-Lucene.Net.Util.TestVersionComparer::TestVersions()-*{color}
 ** Related to current culture
 ** Happens on netcoreapp2.1 on macOS
 # {color:#d04437}*Lucene.Net.Index.TestIndexWriter::TestTwoThreadsInterruptDeadlock()*{color}
 ** Happens on every target framework/OS
 # {color:#d04437}*Lucene.Net.Index.TestIndexWriter::TestThreadInterruptDeadlock()*{color}
 ** Happens on every target framework/OS
 ** Fails with message:
 {code:bash}FAILED; unexpected exception
 System.InvalidOperationException: cannot close: prepareCommit was already called with no
corresponding call to commit
 at Lucene.Net.Index.IndexWriter.CloseInternal(Boolean waitForMerges, Boolean doFlush) in
F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line 1157
 at Lucene.Net.Index.IndexWriter.Dispose(Boolean waitForMerges) in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line
1096
 at Lucene.Net.Index.IndexWriter.Dispose() in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line
1053
 at Lucene.Net.Index.TestIndexWriter.IndexerThreadInterrupt.Run() in F:\Projects\lucenenet\src\Lucene.Net.Tests\Index\TestIndexWriter.cs:line
1223{code}
 ** The error message doesn't occur when using {{lock (this}}} in {{IndexWriter.CommitInternal()}}
to make {{PrepareCommitInternal()}} and {{FinishCommit()}} atomic, but the test still fails
and the extra lock causes deadlocks in other tests. For some reason, the existing {{commitLock}}
isn't completely atomic in some (unknown) case.
 ** The test does not fail if both of the following lines are commented in {{TestIndexWriter.IndexerThreadInterrupt::Run()}}
 {{w.UpdateDocument(new Term("id", idField.GetStringValue()), doc);}}
 {{w.AddDocument(doc);}}
 ** The test does not fail if {{RAMDirectory}} is used instead of {{MockDirectoryWrapper}}
 ** It is quite possible that the problem is with {{MockDirectoryWrapper}} or one of its subcomponents,
although the locking behavior of {{IndexWriter}} is also suspicious
 # {color:#d04437}*Lucene.Net.Classification.KNearestNeighborClassifierTest::TestPerformance()*{color}
 ** Happens on netcoreapp2.1 on Windows
 ** Happens on netcoreapp1.0 on Windows
 ** Happens on net451 on Windows
 # {color:#d04437}*Lucene.Net.Analysis.NGram.EdgeNGramTokenizerTest::TestFullUTF8Range()*{color}
 ** Happens on netcoreapp1.0 on Linux
 # {color:#d04437}*Lucene.Net.Analysis.NGram.NGramTokenizerTest::TestFullUTF8Range()*{color}
 ** Happens on net451 on Windows
 ** Happens on netcoreapp1.0 on Linux
 ** Happens on netcoreapp1.0 on Windows
 ** Happens on netcoreapp1.0 on macOS
 ** Happens on netcoreapp2.1 on macOS
 # {color:#d04437}*Lucene.Net.Search.VectorHighlight.SimpleFragmentsBuilderTest::TestRandomDiscreteMultiValueHighlighting()*{color}
 ** Happens on Windows (I think on all target frameworks, but netcoreapp1.0 for sure)
 ** Not related to current culture
 ** Definitely a problem with the highlighter, not the test or test framework
 # {color:#d04437}*Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper::TestDateRange()*{color}
 ** Does not happen on Windows, but happens on both Linux and macOS
 ** Related to current culture - fails with {{new CultureInfo("ar")}} specifically (and others)
 ** Stack trace
 {code:bash}System.FormatException : String '1‏‏/1‏‏/2002' was not recognized as a
valid DateTime.
 at System.DateTimeParse.Parse(ReadOnlySpan`1 s, DateTimeFormatInfo dtfi, DateTimeStyles styles)
 at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.AssertDateRangeQueryEquals(StandardQueryParser
qp, String field, String startDate, String endDate, DateTime endDateInclusive, Resolution
resolution) in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line
837
 at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.TestDateRange() in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line
826{code}
 # {color:#d04437}*Lucene.Net.Expressions.TestExpressionSorts::TestQueries()*{color}
 ** Happens on net451 on Windows, x86, in Release mode only (if Release, x86, or different
target framework, does not fail)
 ** Does not seem to fail using {{dotnet vstest}}, only fails in Visual Studio 2017/2019

Note we now have an [{{azure-pipelines.yml}}|https://github.com/apache/lucenenet/blob/master/azure-pipelines.yml]
configuration file in our repository that anyone can use to setup a build pipeline to see
the tests run on Windows, macOS and Linux by setting up a [free Azure DevOps account|https://azure.microsoft.com/en-us/services/devops/].
If you create a public project to run the tests in, it will take roughly an hour to see the
test results (a private project will take significantly longer on the free subscription because
they only provide a single parallel job).

*NOTES*
 * {color:#205081}If you want to work on one of these issues, please open a new JIRA ticket
and make it "contained by" this one, so your efforts aren't duplicated by someone else. Assign
the issue to yourself. If you can't work out the issue, make sure that you unassign yourself
and comment on it below that it is still unresolved and up for grabs.{color}
 * {color:#205081}Sometimes problems can be spotted just by comparing the [Lucene 4.8.0 code|https://github.com/apache/lucene-solr/tree/8fdf89690404c0e65784b2c5477552b9dec58591/lucene]
against [Lucene.Net 4.8.0 code|https://github.com/apache/lucenenet].{color}
 * {color:#205081}The code should be checked to make sure there were no translation problems
from Java to C#. This may be easier than it sounds, as you can type phrases like "java HashMap
equivalent c#" into Google to find the answers easily.{color}
 * {color:#205081}We can change the code to properly match the behavior of Java, but no cheating
by changing the conditions of the test! Unless, of course, the test conditions were translated
to C# wrong, which has been known to happen.{color}
 * {color:#205081}Random failures can often be made to happen more frequently by adding a
{{RepeatAttribute}} to the top of the test. Try running 30 or 40 times and you will see the
failure much more often.{color}
 * {color:#205081}If you find a solution to make the test pass, please [open a PR on GitHub|https://github.com/apache/lucenenet/pulls]
or alternatively post the solution here so we can try it ourselves.{color}
 * {color:#205081}If you get the same warm fuzzy feeling we do when we make a test green,
feel free to fix another one.{color}

{color:#ff0000}Also, let us know if you find any failing test that is not on this list (aside
from the "file in use by another process" issue that is known to happen with multiple tests
on Linux and macOS - see{color} [LUCENENET-618|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-618]{color:#ff0000}
if you want to take a crack at that issue{color}{color:#707070}){color}

  was:
Up for grabs. Please contribute to Lucene.Net.

All 8000+ tests pass most of the time, but there are still a few that fail under the random
conditions we test them in. This is a complete list of the known tests that fail and what
is known about them. Note that these may not be all of the ways the tests can fail.
 # {color:#d04437}*-Lucene.Net.Search.TestSearchAfter::TestQueries()-*{color}
 ** Related to current culture
 ** Happens on netcoreapp2.1 on macOS
 ** Happens on netcoreapp1.0 on Linux
 # {color:#d04437}*-Lucene.Net.Util.TestVersionComparer::TestVersions()-*{color}
 ** Related to current culture
 ** Happens on netcoreapp2.1 on macOS
 # {color:#d04437}*Lucene.Net.Index.TestIndexWriter::TestTwoThreadsInterruptDeadlock()*{color}
 ** Happens on every target framework/OS
 # {color:#d04437}*Lucene.Net.Index.TestIndexWriter::TestThreadInterruptDeadlock()*{color}
 ** Happens on every target framework/OS
 ** Fails with message:
 FAILED; unexpected exception
 System.InvalidOperationException: cannot close: prepareCommit was already called with no
corresponding call to commit
 at Lucene.Net.Index.IndexWriter.CloseInternal(Boolean waitForMerges, Boolean doFlush) in
F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line 1157
 at Lucene.Net.Index.IndexWriter.Dispose(Boolean waitForMerges) in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line
1096
 at Lucene.Net.Index.IndexWriter.Dispose() in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line
1053
 at Lucene.Net.Index.TestIndexWriter.IndexerThreadInterrupt.Run() in F:\Projects\lucenenet\src\Lucene.Net.Tests\Index\TestIndexWriter.cs:line
1223
 ** The error message doesn't occur when using {{lock (this}}} in {{IndexWriter.CommitInternal()}}
to make {{PrepareCommitInternal()}} and {{FinishCommit()}} atomic, but the test still fails
and the extra lock causes deadlocks in other tests. For some reason, the existing {{commitLock}}
isn't completely atomic in some (unknown) case.
 ** The test does not fail if both of the following lines are commented in {{TestIndexWriter.IndexerThreadInterrupt::Run()}}
 {{w.UpdateDocument(new Term("id", idField.GetStringValue()), doc);}}
 {{w.AddDocument(doc);}}
 ** The test does not fail if {{RAMDirectory}} is used instead of {{MockDirectoryWrapper}}
 ** It is quite possible that the problem is with {{MockDirectoryWrapper}} or one of its subcomponents,
although the locking behavior of {{IndexWriter}} is also suspicious
 # {color:#d04437}*Lucene.Net.Classification.KNearestNeighborClassifierTest::TestPerformance()*{color}
 ** Happens on netcoreapp2.1 on Windows
 ** Happens on netcoreapp1.0 on Windows
 ** Happens on net451 on Windows
 # {color:#d04437}*Lucene.Net.Analysis.NGram.EdgeNGramTokenizerTest::TestFullUTF8Range()*{color}
 ** Happens on netcoreapp1.0 on Linux
 # {color:#d04437}*Lucene.Net.Analysis.NGram.NGramTokenizerTest::TestFullUTF8Range()*{color}
 ** Happens on net451 on Windows
 ** Happens on netcoreapp1.0 on Linux
 ** Happens on netcoreapp1.0 on Windows
 ** Happens on netcoreapp1.0 on macOS
 ** Happens on netcoreapp2.1 on macOS
 # {color:#d04437}*Lucene.Net.Search.VectorHighlight.SimpleFragmentsBuilderTest::TestRandomDiscreteMultiValueHighlighting()*{color}
 ** Happens on Windows (I think on all target frameworks, but netcoreapp1.0 for sure)
 ** Not related to current culture
 ** Definitely a problem with the highlighter, not the test or test framework
 # {color:#d04437}*Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper::TestDateRange()*{color}
 ** Does not happen on Windows, but happens on both Linux and macOS
 ** Related to current culture - fails with `ar` specifically (and others)
 ** Stack trace
 System.FormatException : String '1‏‏/1‏‏/2002' was not recognized as a valid DateTime.
 at System.DateTimeParse.Parse(ReadOnlySpan`1 s, DateTimeFormatInfo dtfi, DateTimeStyles styles)
 at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.AssertDateRangeQueryEquals(StandardQueryParser
qp, String field, String startDate, String endDate, DateTime endDateInclusive, Resolution
resolution) in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line
837
 at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.TestDateRange() in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line
826
 # {color:#d04437}*Lucene.Net.Expressions.TestExpressionSorts::TestQueries()*{color}
 ** Happens on net451 on Windows, x86, in Release mode only (if Release, x86, or different
target framework, does not fail)
 ** Does not seem to fail using {{dotnet vstest}}, only fails in Visual Studio 2017/2019

Note we now have an [{{azure-pipelines.yml}}|https://github.com/apache/lucenenet/blob/master/azure-pipelines.yml]
configuration file in our repository that anyone can use to setup a build pipeline to see
the tests run on Windows, macOS and Linux by setting up a [free Azure DevOps account|https://azure.microsoft.com/en-us/services/devops/].
If you create a public project to run the tests in, it will take roughly an hour to see the
test results (a private project will take significantly longer on the free subscription because
they only provide a single parallel job).

*NOTES*
 * {color:#205081}If you want to work on one of these issues, please open a new JIRA ticket
and make it "contained by" this one, so your efforts aren't duplicated by someone else. Assign
the issue to yourself. If you can't work out the issue, make sure that you unassign yourself
and comment on it below that it is still unresolved and up for grabs.{color}
 * {color:#205081}Sometimes problems can be spotted just by comparing the [Lucene 4.8.0 code|https://github.com/apache/lucene-solr/tree/8fdf89690404c0e65784b2c5477552b9dec58591/lucene]
against [Lucene.Net 4.8.0 code|https://github.com/apache/lucenenet].{color}
 * {color:#205081}The code should be checked to make sure there were no translation problems
from Java to C#. This may be easier than it sounds, as you can type phrases like "java HashMap
equivalent c#" into Google to find the answers easily.{color}
 * {color:#205081}We can change the code to properly match the behavior of Java, but no cheating
by changing the conditions of the test! Unless, of course, the test conditions were translated
to C# wrong, which has been known to happen.{color}
 * {color:#205081}Random failures can often be made to happen more frequently by adding a
{{RepeatAttribute}} to the top of the test. Try running 30 or 40 times and you will see the
failure much more often.{color}
 * {color:#205081}If you find a solution to make the test pass, please [open a PR on GitHub|https://github.com/apache/lucenenet/pulls]
or alternatively post the solution here so we can try it ourselves.{color}
 * {color:#205081}If you get the same warm fuzzy feeling we do when we make a test green,
feel free to fix another one.{color}

{color:#ff0000}Also, let us know if you find any failing test that is not on this list (aside
from the "file in use by another process" issue that is known to happen with multiple tests
on Linux and macOS - see{color} [LUCENENET-618|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-618]{color:#ff0000}
if you want to take a crack at that issue{color}{color:#707070}){color}


> Known Failing Tests on Lucene.Net
> ---------------------------------
>
>                 Key: LUCENENET-619
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-619
>             Project: Lucene.Net
>          Issue Type: Task
>          Components: Lucene.Net Core, Lucene.Net Test, Lucene.Net.Analysis.Common, Lucene.Net.Classification
>    Affects Versions: Lucene.Net 4.8.0
>            Reporter: Shad Storhaug
>            Priority: Major
>              Labels: test-fail, test-failure, up-for-grabs
>
> Up for grabs. Please contribute to Lucene.Net.
> All 8000+ tests pass most of the time, but there are still a few that fail under the
random conditions we test them in. This is a complete list of the known tests that fail and
what is known about them. Note that these may not be all of the ways the tests can fail.
>  # {color:#d04437}*-Lucene.Net.Search.TestSearchAfter::TestQueries()-*{color}
>  ** Related to current culture
>  ** Happens on netcoreapp2.1 on macOS
>  ** Happens on netcoreapp1.0 on Linux
>  # {color:#d04437}*-Lucene.Net.Util.TestVersionComparer::TestVersions()-*{color}
>  ** Related to current culture
>  ** Happens on netcoreapp2.1 on macOS
>  # {color:#d04437}*Lucene.Net.Index.TestIndexWriter::TestTwoThreadsInterruptDeadlock()*{color}
>  ** Happens on every target framework/OS
>  # {color:#d04437}*Lucene.Net.Index.TestIndexWriter::TestThreadInterruptDeadlock()*{color}
>  ** Happens on every target framework/OS
>  ** Fails with message:
>  {code:bash}FAILED; unexpected exception
>  System.InvalidOperationException: cannot close: prepareCommit was already called with
no corresponding call to commit
>  at Lucene.Net.Index.IndexWriter.CloseInternal(Boolean waitForMerges, Boolean doFlush)
in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line 1157
>  at Lucene.Net.Index.IndexWriter.Dispose(Boolean waitForMerges) in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line
1096
>  at Lucene.Net.Index.IndexWriter.Dispose() in F:\Projects\lucenenet\src\Lucene.Net\Index\IndexWriter.cs:line
1053
>  at Lucene.Net.Index.TestIndexWriter.IndexerThreadInterrupt.Run() in F:\Projects\lucenenet\src\Lucene.Net.Tests\Index\TestIndexWriter.cs:line
1223{code}
>  ** The error message doesn't occur when using {{lock (this}}} in {{IndexWriter.CommitInternal()}}
to make {{PrepareCommitInternal()}} and {{FinishCommit()}} atomic, but the test still fails
and the extra lock causes deadlocks in other tests. For some reason, the existing {{commitLock}}
isn't completely atomic in some (unknown) case.
>  ** The test does not fail if both of the following lines are commented in {{TestIndexWriter.IndexerThreadInterrupt::Run()}}
>  {{w.UpdateDocument(new Term("id", idField.GetStringValue()), doc);}}
>  {{w.AddDocument(doc);}}
>  ** The test does not fail if {{RAMDirectory}} is used instead of {{MockDirectoryWrapper}}
>  ** It is quite possible that the problem is with {{MockDirectoryWrapper}} or one of
its subcomponents, although the locking behavior of {{IndexWriter}} is also suspicious
>  # {color:#d04437}*Lucene.Net.Classification.KNearestNeighborClassifierTest::TestPerformance()*{color}
>  ** Happens on netcoreapp2.1 on Windows
>  ** Happens on netcoreapp1.0 on Windows
>  ** Happens on net451 on Windows
>  # {color:#d04437}*Lucene.Net.Analysis.NGram.EdgeNGramTokenizerTest::TestFullUTF8Range()*{color}
>  ** Happens on netcoreapp1.0 on Linux
>  # {color:#d04437}*Lucene.Net.Analysis.NGram.NGramTokenizerTest::TestFullUTF8Range()*{color}
>  ** Happens on net451 on Windows
>  ** Happens on netcoreapp1.0 on Linux
>  ** Happens on netcoreapp1.0 on Windows
>  ** Happens on netcoreapp1.0 on macOS
>  ** Happens on netcoreapp2.1 on macOS
>  # {color:#d04437}*Lucene.Net.Search.VectorHighlight.SimpleFragmentsBuilderTest::TestRandomDiscreteMultiValueHighlighting()*{color}
>  ** Happens on Windows (I think on all target frameworks, but netcoreapp1.0 for sure)
>  ** Not related to current culture
>  ** Definitely a problem with the highlighter, not the test or test framework
>  # {color:#d04437}*Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper::TestDateRange()*{color}
>  ** Does not happen on Windows, but happens on both Linux and macOS
>  ** Related to current culture - fails with {{new CultureInfo("ar")}} specifically (and
others)
>  ** Stack trace
>  {code:bash}System.FormatException : String '1‏‏/1‏‏/2002' was not recognized
as a valid DateTime.
>  at System.DateTimeParse.Parse(ReadOnlySpan`1 s, DateTimeFormatInfo dtfi, DateTimeStyles
styles)
>  at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.AssertDateRangeQueryEquals(StandardQueryParser
qp, String field, String startDate, String endDate, DateTime endDateInclusive, Resolution
resolution) in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line
837
>  at Lucene.Net.QueryParsers.Flexible.Standard.TestQPHelper.TestDateRange() in D:\a\1\s\src\Lucene.Net.Tests.QueryParser\Flexible\Standard\TestQPHelper.cs:line
826{code}
>  # {color:#d04437}*Lucene.Net.Expressions.TestExpressionSorts::TestQueries()*{color}
>  ** Happens on net451 on Windows, x86, in Release mode only (if Release, x86, or different
target framework, does not fail)
>  ** Does not seem to fail using {{dotnet vstest}}, only fails in Visual Studio 2017/2019
> Note we now have an [{{azure-pipelines.yml}}|https://github.com/apache/lucenenet/blob/master/azure-pipelines.yml]
configuration file in our repository that anyone can use to setup a build pipeline to see
the tests run on Windows, macOS and Linux by setting up a [free Azure DevOps account|https://azure.microsoft.com/en-us/services/devops/].
If you create a public project to run the tests in, it will take roughly an hour to see the
test results (a private project will take significantly longer on the free subscription because
they only provide a single parallel job).
> *NOTES*
>  * {color:#205081}If you want to work on one of these issues, please open a new JIRA
ticket and make it "contained by" this one, so your efforts aren't duplicated by someone else.
Assign the issue to yourself. If you can't work out the issue, make sure that you unassign
yourself and comment on it below that it is still unresolved and up for grabs.{color}
>  * {color:#205081}Sometimes problems can be spotted just by comparing the [Lucene 4.8.0
code|https://github.com/apache/lucene-solr/tree/8fdf89690404c0e65784b2c5477552b9dec58591/lucene]
against [Lucene.Net 4.8.0 code|https://github.com/apache/lucenenet].{color}
>  * {color:#205081}The code should be checked to make sure there were no translation problems
from Java to C#. This may be easier than it sounds, as you can type phrases like "java HashMap
equivalent c#" into Google to find the answers easily.{color}
>  * {color:#205081}We can change the code to properly match the behavior of Java, but
no cheating by changing the conditions of the test! Unless, of course, the test conditions
were translated to C# wrong, which has been known to happen.{color}
>  * {color:#205081}Random failures can often be made to happen more frequently by adding
a {{RepeatAttribute}} to the top of the test. Try running 30 or 40 times and you will see
the failure much more often.{color}
>  * {color:#205081}If you find a solution to make the test pass, please [open a PR on
GitHub|https://github.com/apache/lucenenet/pulls] or alternatively post the solution here
so we can try it ourselves.{color}
>  * {color:#205081}If you get the same warm fuzzy feeling we do when we make a test green,
feel free to fix another one.{color}
> {color:#ff0000}Also, let us know if you find any failing test that is not on this list
(aside from the "file in use by another process" issue that is known to happen with multiple
tests on Linux and macOS - see{color} [LUCENENET-618|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-618]{color:#ff0000}
if you want to take a crack at that issue{color}{color:#707070}){color}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message