lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shad Storhaug (Jira)" <j...@apache.org>
Subject [jira] [Created] (LUCENENET-627) Port RandomizedTesting Test Runner
Date Thu, 10 Oct 2019 18:40:00 GMT
Shad Storhaug created LUCENENET-627:
---------------------------------------

             Summary: Port RandomizedTesting Test Runner
                 Key: LUCENENET-627
                 URL: https://issues.apache.org/jira/browse/LUCENENET-627
             Project: Lucene.Net
          Issue Type: Task
          Components: Lucene.Net.TestFramework
    Affects Versions: Lucene.Net 4.8.0
            Reporter: Shad Storhaug


While the test framework can run with the standard NUnit test runner and we have managed to
make it work, it is missing the ability to reproduce a random test failure, which is crucial
for debugging some of the more complicated random tests. Lucene uses a custom JUnit test runner
called [RandomizedTesting|https://github.com/randomizedtesting/randomizedtesting] to accomplish
this.

It also includes some other nice features

# Ensure when a test fails, the random seeds are included in error messages and logs
# Code analysis to ensure the tests are setup properly
# Run tests in a random order to ensure they are not dependent upon each other

Preliminary analysis shows that the API for NUnit allows building custom test runners and
it is a close enough match to implement the functionality. Most likely, this will require
a custom adapter as well, so the test runner can integrate into Visual Studio and dotnet test/mstest.

Do note that without doing some pretty heavy refactoring on the Codec, Directory, and LuceneTestCase
classes, it is not possible to run tests in parallel within the same AppDomain because the
codecs use a static variable to turn codec impersonation on/off. For now, it would probably
be best to run tests serially.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message