lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prad Nelluru <prn...@microsoft.com>
Subject RE: Dealing with randomness in unit tests
Date Tue, 09 Sep 2014 21:16:38 GMT
Okay, one of my recent commits makes the tests deterministic. When we have them all passing,
we can see if they pass for all seeds.

-----Original Message-----
From: itamar.synhershko@gmail.com [mailto:itamar.synhershko@gmail.com] On Behalf Of Itamar
Syn-Hershko
Sent: Monday, September 8, 2014 5:06 PM
To: dev@lucenenet.apache.org
Subject: Re: Dealing with randomness in unit tests

Yes, this is a known problem.

Lucene's test randomization was contributed by CarrotSearch and specifically Dawid Weiss.
His talk on the subject is a must-watch really
http://vimeo.com/32087114

I've discussed this with him and it appears compiling CarrotSearch's tests randomization library
with IKVM is the way to go. I have received some integration classes they use internally to
connect this with NUnit. I have yet to do anything about it.

If we can disable the randomness for now by just using a static seed without changing any
code, that would be great. This way we could keep productive and introduce predictive randomness
using carrotsearch's excellent codebase when we get the time to do so.

Does that make sense?

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko> Freelance Developer
& Consultant Author of RavenDB in Action <http://manning.com/synhershko/>

On Tue, Sep 9, 2014 at 2:59 AM, Prad Nelluru <prnell@microsoft.com> wrote:

> Hey all,
>
> The Java Lucene code uses a test randomization framework. The 
> framework's random number methods use a new seed value each time the tests are run.
> Whenever there's a test failure, the seed is displayed so the 
> conditions for the failure can be replayed. As there is no equivalent 
> randomization framework in C#, we currently just use the .net Random 
> class whenever a random generator is required. Crucially, we don't 
> provide a seed or even record it. As the seeds can't be known, we have 
> no way of reproducing test failures. While I haven't seen an 
> intermittent failure yet, I've heard that Russell ran into them occasionally.
>
> We can either use a hardcoded fixed seed, or we can store a random 
> seed to show when a test fails. In any case, we should remove the 
> initializations of Random all over the place (replacing them with LuceneTestCase.Random()).
>
> What do you all think?
>
> -Prad
>
Mime
View raw message