lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Itamar Syn-Hershko <ita...@code972.com>
Subject Re: Dealing with randomness in unit tests
Date Tue, 09 Sep 2014 00:05:47 GMT
Yes, this is a known problem.

Lucene's test randomization was contributed by CarrotSearch and
specifically Dawid Weiss. His talk on the subject is a must-watch really
http://vimeo.com/32087114

I've discussed this with him and it appears compiling CarrotSearch's tests
randomization library with IKVM is the way to go. I have received some
integration classes they use internally to connect this with NUnit. I have
yet to do anything about it.

If we can disable the randomness for now by just using a static seed
without changing any code, that would be great. This way we could keep
productive and introduce predictive randomness using carrotsearch's
excellent codebase when we get the time to do so.

Does that make sense?

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Author of RavenDB in Action <http://manning.com/synhershko/>

On Tue, Sep 9, 2014 at 2:59 AM, Prad Nelluru <prnell@microsoft.com> wrote:

> Hey all,
>
> The Java Lucene code uses a test randomization framework. The framework's
> random number methods use a new seed value each time the tests are run.
> Whenever there's a test failure, the seed is displayed so the conditions
> for the failure can be replayed. As there is no equivalent randomization
> framework in C#, we currently just use the .net Random class whenever a
> random generator is required. Crucially, we don't provide a seed or even
> record it. As the seeds can't be known, we have no way of reproducing test
> failures. While I haven't seen an intermittent failure yet, I've heard that
> Russell ran into them occasionally.
>
> We can either use a hardcoded fixed seed, or we can store a random seed to
> show when a test fails. In any case, we should remove the initializations
> of Random all over the place (replacing them with LuceneTestCase.Random()).
>
> What do you all think?
>
> -Prad
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message