lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shad Storhaug (JIRA)" <j...@apache.org>
Subject [jira] [Created] (LUCENENET-614) Make Lucene.Net.TestFramework functionality available to end users (as was done in Java)
Date Mon, 12 Aug 2019 03:01:00 GMT
Shad Storhaug created LUCENENET-614:
---------------------------------------

             Summary: Make Lucene.Net.TestFramework functionality available to end users (as
was done in Java)
                 Key: LUCENENET-614
                 URL: https://issues.apache.org/jira/browse/LUCENENET-614
             Project: Lucene.Net
          Issue Type: Test
          Components: Lucene.Net.TestFramework
    Affects Versions: Lucene.Net 4.8.0
            Reporter: Shad Storhaug


Early efforts on Lucene.Net failed to recognize that the test framework was not meant to be
an internal component only used by Lucene's tests, but it was also shipped to end users as
a Java package so they could use it to test their own custom extensions to Lucene.

We have had at least 2 people request that we make the TestFramework module public.

*Challenges:*
 # In Java JUnit is ubiquitous, but in .NET there are several test frameworks widely in use.
We would need to support at least NUnit, xUnit, and MSTest in order to have wide coverage.
 # Testing frameworks tend to use .NET attributes, and it is not possible to abstract attributes
in such a way we can use a single set of attributes to support every test framework.
 # Testing frameworks tend to use static methods to assert test failures, we need to ensure
wrapper abstractions are used consistently in {{Lucene.Net.TestFramework}} if we are to build
an abstraction over each 3rd party test framework's static methods.
 # The API of the test framework wasn't designed with end users in mind. It needs to be cleaned
up to use .NET conventions and best practices.
 # The test framework uses abstract classes with tests that are meant to be inherited by the
implementer of the subclass. In the past, this has had some strange behavior where the tests
were not showing up as belonging to the class library that owned them, but they were shown
as "external tests" in Visual Studio Test Explorer.
 # There are no tests that exist to verify the test framework works like it did in Java.

First of all, we should have 3rd party test framework specific adapter assemblies, i.e. {{Lucene.Net.TestFramework.NUnit}},
{{Lucene.Net.TestFramework.xUnit}}, {{Lucene.Net.TestFramework.MSTest}}.

I have done some analysis and it seems that the issue with calling static methods can easily
be overcome by consistently using the {{assertXXX}} methods that already exist. A wrapper
interface could be built to implement these methods, and then in each adapter assembly, an
implementation could be created to call into the 3rd-party test framework's static methods.

The existing Java-like methods could be made into internal methods and setup {{InternalsVisibleToAttribute}}
for all of our test projects so they can still access these methods. The thought is to keep
these Java-like so future tests are easy to port, but not to expose them to end users. End
users should be directed to use each 3rd party test framework's static methods directly.

For the attributes, there are at least 2 possible options. 
 # Quick and dirty: Add conditional compilation around the attributes (for example {{FEATURE_NUNIT}},
and use globbing patterns to decide which {{.cs}} files from the {{Lucene.Net.TestFramework}}
folder to include in each package.
 # Extensible:
 ## Rename each end-user-inheritable class in {{Lucene.Net.TestFramework}} with the suffix
{{Base}}.
 ## Create a subclass of each base class in each test framework adapter assembly with the
original class name, and add the test framework specific attributes to this class.

The latter approach has the advantage that any 3rd party can make an adapter for use with
another 3rd party test framework. The former approach has the advantage that we don't need
to ship a {{Lucene.Net.TestFramwork}} package in addition to the 3rd party framework adapter
packages, and is quicker to implement because we don't need to rename, move or create very
many files.

As far as the weird "external tests" behavior, that did not seem to happen with xUnit, and
seems to have inconsistent behavior between different versions of NUnit. For NUnit, we either
need to check for and patch compatibility with a wide range of versions of NUnit or document
that we only support specific versions.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message