lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laimonas Simutis <lai...@gmail.com>
Subject Re: Reproducing random test failures
Date Mon, 18 May 2015 00:49:36 GMT
Wyatt,

I see the new options on TC, thanks for that. I still haven't thought about
how I will go about capturing the failures exactly, but will give you a
shout if I need some help with TC configuration just for those runs.

If you can reproduce any of those test failures locally, do you mind
running them in VERBOSE mode (debug build without any other changes will
do), and emailing the console output that you get? I might be too
optimistic, but perhaps something there will stand out.


Thanks again!

Laimis


On Sun, May 17, 2015 at 8:18 AM, Wyatt Barnett <wyatt.barnett@gmail.com>
wrote:

> For TestSort_2 -- It appears to be passing based on data at
>
> http://teamcity.codebetter.com/project.html?projectId=LuceneNet&testNameId=-8365680837810961892&tab=testDetails
> ; I am having locally reproducable problems on the others though.
>
> On Sun, May 17, 2015 at 7:55 AM, Wyatt Barnett <wyatt.barnett@gmail.com>
> wrote:
>
> > Done -- you should now see a run button when you visit
> > http://teamcity.codebetter.com/project.html?projectId=LuceneNet
> >
> > On Sat, May 16, 2015 at 8:02 PM, Laimonas Simutis <laimis@gmail.com>
> > wrote:
> >
> >> Wyatt,
> >>
> >> Could you add me to the lucene.net group on TC? I have a login there,
> >> username: laimis.
> >>
> >>
> >> Thanks!
> >>
> >> On Sat, May 16, 2015 at 6:15 PM, Wyatt Barnett <wyatt.barnett@gmail.com
> >
> >> wrote:
> >>
> >> > Sounds good Laimis. You will need to setup a login to the CodeBetter
> >> > teamcity server and get added to the lucene.net group if you haven't
> >> > already. Let me know if you need help there too.
> >> >
> >> > On Sat, May 16, 2015 at 4:52 PM Laimonas Simutis <laimis@gmail.com>
> >> wrote:
> >> >
> >> > > Wyatt,
> >> > >
> >> > > Sweet, I will let you know once I have a branch out with additional
> >> > logging
> >> > > and separate category for tests that you can configure to run.
> >> > >
> >> > > Re: release mode, tried that and was able to fix a few bugs after
> >> > switching
> >> > > to it. They were in that PR with debug.assert changes. Who knows,
> the
> >> > > remaining failures might still be related to that, but can't
> >> reproduce it
> >> > > locally.
> >> > >
> >> > > Laimis
> >> > > On May 16, 2015 4:34 PM, "Wyatt Barnett" <wyatt.barnett@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Sorry about the blank one -- getting used to google inbox here
> and I
> >> > > > misclicked.
> >> > > >
> >> > > > Anyhow, I have a repro or at least a rhyme and reason -- TeamCity
> is
> >> > > > running in release mode and I think we have difffering behavior
> >> there.
> >> > If
> >> > > > you switch your copy of visual studio to release mode you will
get
> >> the
> >> > > same
> >> > > > failures we are seeing in TeamCity. Does that help narrow it
down
> a
> >> > bit?
> >> > > >
> >> > > > On Sat, May 16, 2015 at 4:26 PM Wyatt Barnett <
> >> wyatt.barnett@gmail.com
> >> > >
> >> > > > wrote:
> >> > > >
> >> > > > >
> >> > > > >
> >> > > > > On Sat, May 16, 2015 at 3:22 PM Wyatt Barnett <
> >> > wyatt.barnett@gmail.com
> >> > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > >> I agree with Itamar -- it feels environmental. I'll
do some
> >> digging
> >> > > into
> >> > > > >> the teamcity output but I think the plan of setting
up some
> extra
> >> > > > verbose
> >> > > > >> logging here would make sense. I can set you up with
a separate
> >> > build
> >> > > > >> pointed at your fork if that helps -- it will keep the
feedback
> >> > cycle
> >> > > > >> tighter. The other thing we could do is categorize the
tests
> and
> >> > focus
> >> > > > that
> >> > > > >> build at running only that category so you don't need
to wait
> on
> >> the
> >> > > > whole
> >> > > > >> suite to get responses. Let me know if you want me to
proceed
> >> there.
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> On Sat, May 16, 2015 at 3:10 PM, Itamar Syn-Hershko
<
> >> > > itamar@code972.com
> >> > > > >
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >>> Yes, that would be the best way to do this. On Java
Lucene,
> the
> >> > > > >>> randomized
> >> > > > >>> tests framework allows you to re-use the random
seed
> associated
> >> > with
> >> > > > the
> >> > > > >>> failure, but we are not there yet. Either way, I
suspect this
> >> to be
> >> > > an
> >> > > > >>> environment issue rather than a code path one.
> >> > > > >>>
> >> > > > >>> --
> >> > > > >>>
> >> > > > >>> Itamar Syn-Hershko
> >> > > > >>> http://code972.com | @synhershko <
> >> https://twitter.com/synhershko>
> >> > > > >>> Freelance Developer & Consultant
> >> > > > >>> Lucene.NET committer and PMC member
> >> > > > >>>
> >> > > > >>> On Sat, May 16, 2015 at 10:06 PM, Laimonas Simutis
<
> >> > laimis@gmail.com
> >> > > >
> >> > > > >>> wrote:
> >> > > > >>>
> >> > > > >>> > There are three tests that consistently fail
on TC but no
> >> matter
> >> > > how
> >> > > > >>> many
> >> > > > >>> > times I try, I can't reproduce it locally.
These tests are:
> >> > > > >>> >
> >> > > > >>> > TestFuzzyQuery.TestTieBreaker
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> http://teamcity.codebetter.com/viewLog.html?buildId=191298&tab=buildResultsDiv&buildTypeId=LuceneNet_Core#testNameId-6371662534320583798
> >> > > > >>> >
> >> > > > >>> > TestSimpleExplanations.TestDMQ8
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> http://teamcity.codebetter.com/viewLog.html?buildId=191298&tab=buildResultsDiv&buildTypeId=LuceneNet_Core#testNameId5725706748293106127
> >> > > > >>> >
> >> > > > >>> > TestTopDocsMerge.TestSort_2
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> http://teamcity.codebetter.com/viewLog.html?buildId=191298&tab=buildResultsDiv&buildTypeId=LuceneNet_Core#testNameId-8365680837810961892
> >> > > > >>> >
> >> > > > >>> > I would fix them if I could reproduce it --
and I am running
> >> out
> >> > of
> >> > > > >>> ideas
> >> > > > >>> > how to do it. Even if I put them in a loop
running hundreds
> of
> >> > > > times, I
> >> > > > >>> > can't trigger the failure.
> >> > > > >>> >
> >> > > > >>> > Anyone have any ideas how to go about reproducing
it? I am
> >> > thinking
> >> > > > to
> >> > > > >>> push
> >> > > > >>> > very verbose code in a separate branch that
logs the input
> >> > values /
> >> > > > >>> random
> >> > > > >>> > values that are used and see what happens.
Checking if
> anyone
> >> has
> >> > > any
> >> > > > >>> other
> >> > > > >>> > suggestions.
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>> > Thanks,
> >> > > > >>> >
> >> > > > >>> > Laimis
> >> > > > >>> >
> >> > > > >>>
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message