lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shad Storhaug (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENENET-614) Make Lucene.Net.TestFramework functionality available to end users (as was done in Java)
Date Mon, 12 Aug 2019 03:08:00 GMT

     [ https://issues.apache.org/jira/browse/LUCENENET-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shad Storhaug updated LUCENENET-614:
------------------------------------
    Issue Type: Task  (was: Test)

> Make Lucene.Net.TestFramework functionality available to end users (as was done in Java)
> ----------------------------------------------------------------------------------------
>
>                 Key: LUCENENET-614
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-614
>             Project: Lucene.Net
>          Issue Type: Task
>          Components: Lucene.Net.TestFramework
>    Affects Versions: Lucene.Net 4.8.0
>            Reporter: Shad Storhaug
>            Priority: Minor
>
> Early efforts on Lucene.Net failed to recognize that the test framework was not meant
to be an internal component only used by Lucene's tests, but it was also shipped to end users
as a Java package so they could use it to test their own custom extensions to Lucene.
> We have had at least 2 people request that we make the TestFramework module public.
> *Challenges:*
>  # In Java JUnit is ubiquitous, but in .NET there are several test frameworks widely
in use. We would need to support at least NUnit, xUnit, and MSTest in order to have wide coverage.
>  # Testing frameworks tend to use .NET attributes, and it is not possible to abstract
attributes in such a way we can use a single set of attributes to support every test framework.
>  # Testing frameworks tend to use static methods to assert test failures, we need to
ensure wrapper abstractions are used consistently in {{Lucene.Net.TestFramework}} if we are
to build an abstraction over each 3rd party test framework's static methods.
>  # The API of the test framework wasn't designed with end users in mind. It needs to
be cleaned up to use .NET conventions and best practices.
>  # The test framework uses abstract classes with tests that are meant to be inherited
by the implementer of the subclass. In the past, this has had some strange behavior where
the tests were not showing up as belonging to the class library that owned them, but they
were shown as "external tests" in Visual Studio Test Explorer.
>  # There are no tests that exist to verify the test framework works like it did in Java.
> First of all, we should have 3rd party test framework specific adapter assemblies, i.e.
{{Lucene.Net.TestFramework.NUnit}}, {{Lucene.Net.TestFramework.xUnit}}, {{Lucene.Net.TestFramework.MSTest}}.
> I have done some analysis and it seems that the issue with calling static methods can
easily be overcome by consistently using the {{assertXXX}} methods that already exist. A wrapper
interface could be built to implement these methods, and then in each adapter assembly, an
implementation could be created to call into the 3rd-party test framework's static methods.
> The existing Java-like methods could be made into internal methods and setup {{InternalsVisibleToAttribute}}
for all of our test projects so they can still access these methods. The thought is to keep
these Java-like so future tests are easy to port, but not to expose them to end users. End
users should be directed to use each 3rd party test framework's static methods directly.
> For the attributes, there are at least 2 possible options. 
>  # Quick and dirty: Add conditional compilation around the attributes (for example {{FEATURE_NUNIT}},
and use globbing patterns to decide which {{.cs}} files from the {{Lucene.Net.TestFramework}}
folder to include in each package.
>  # Extensible:
>  ## Rename each end-user-inheritable class in {{Lucene.Net.TestFramework}} with the suffix
{{Base}}.
>  ## Create a subclass of each base class in each test framework adapter assembly with
the original class name, and add the test framework specific attributes to this class.
> The latter approach has the advantage that any 3rd party can make an adapter for use
with another 3rd party test framework. The former approach has the advantage that we don't
need to ship a {{Lucene.Net.TestFramwork}} package in addition to the 3rd party framework
adapter packages, and is quicker to implement because we don't need to rename, move or create
very many files.
> As far as the weird "external tests" behavior, that did not seem to happen with xUnit,
and seems to have inconsistent behavior between different versions of NUnit. For NUnit, we
either need to check for and patch compatibility with a wide range of versions of NUnit or
document that we only support specific versions.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message