lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shad Storhaug (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENENET-419) Tests should be categorized
Date Wed, 28 Jun 2017 17:49:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENENET-419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16066937#comment-16066937
] 

Shad Storhaug commented on LUCENENET-419:
-----------------------------------------

I am not sure I understand the problem that would be solved by making these changes.

{quote}The unit tests should always run. The integration tests should fail when things are
not set up right.

The testing env should help anyone looking to verify their testing setup. e.g. The test require
a temp directory as an Environment Variable. It would be helpful to put things like these
into their own tests to verify the setup and help developers figure out what they are missing.{quote}

In Lucene.Net 4.8.0, there are no requirements to have any environment variable setup for
the tests to run. As long as all of the prerequisites are in place (as specified on the readme
[https://github.com/apache/lucenenet#building-and-testing]), the tests will run.

Am I correct in assuming that this request was a workaround for the fact that Lucene.Net 3.0.3
didn't have the tests setup to "just run" (as they should always be setup)?

{quote}We should break down the tests into categories.

Possible Categories:
* ones that ensure the test env is set up correctly for any integration tests.
* integration tests
* unit tests (ones that use RAMDirectory, in memory objects, etc)
* any others that may be useful.

CI should run these by categories.{quote}

As for categorizing tests beyond "long running tests" and "lucene.net specific tests" which
we already have, I am not seeing the advantage there either. Nearly all of the tests are integration
tests (that is, tests that are setup to test things as a unit instead of one individual component
and a bunch of fakes). Putting 80-90% of the 7000+ tests in one big category doesn't seem
like it would gain anything. Also, I am not seeing what we gain by running the categories
separately in CI. Am I missing something?

> Tests should be categorized 
> ----------------------------
>
>                 Key: LUCENENET-419
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-419
>             Project: Lucene.Net
>          Issue Type: Task
>          Components: Build Automation, Lucene.Net Test
>    Affects Versions: Lucene.Net 3.0.3, Lucene.Net 4.8.0
>         Environment: All
>            Reporter: michael herndon
>            Assignee: michael herndon
>              Labels: test, testing
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>
> We should break down the tests into categories. 
> Possible Categories:
>  * ones that ensure the test env is set up correctly for any integration tests.
>  * integration tests
>  * unit tests (ones that use RAMDirectory, in memory objects, etc)
>  * any others that may be useful. 
> The unit tests should always run. The integration tests should fail when things are not
set up right.  
> CI should run these by categories.  
> The testing env should help anyone looking to verify their testing setup.  e.g. The test
require a temp directory as an Environment Variable.  It would be helpful to put things like
these into their own tests to verify the setup and help developers figure out what they are
missing.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message