lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Connie Yau <conn...@microsoft.com>
Subject RE: Lucene.NET to .NET Core
Date Fri, 24 Jun 2016 19:12:45 GMT
Hi,

Thank you for the input! I will add a Trait for the tests that have Timeouts for now so they
can be fixed to be more stable.  

One more question... While migrating the code to xUnit, I noticed that there are a lot of
`static` variables in LuceneTestCase and which would make it hard to run in parallel because
some test classes modify the static values. Is there a reason for keeping them static?

Thanks!
Connie

-----Original Message-----
From: Laimonas Simutis [mailto:laimis@gmail.com] 
Sent: Wednesday, June 22, 2016 2:54 PM
To: dev@lucenenet.apache.org
Subject: Re: Lucene.NET to .NET Core

Wyatt is right on. Certain tests under random circumstances could get into a bad state and
run "forever", potentially generate gigs of data on disk, etc and so timeout was added to
abort the test earlier in case that happened.

Our plan was to make a pass at those long running tests and fix the issues that cause them
to run so long once we are fully complete with the porting.

Perhaps you can add a category / trait on such tests in order to avoid them until they can
be properly investigated and underlying issues fixed?



On Wed, Jun 22, 2016 at 9:26 AM, Wyatt Barnett <wyatt.barnett@gmail.com>
wrote:

> I don't know the history on why the timeouts were added but I suspect 
> they were self-defensive in nature. I think in most cases the timeouts 
> are really back-stopping things so that if there is a race condition 
> the test suite bails and fails rather than runs forever. Is there a 
> way to backstop stuff like that with xunit?
>
>
> On Tue, Jun 21, 2016 at 8:24 PM Connie Yau <conniey@microsoft.com> wrote:
>
> > Hi,
> >
> > I've been looking into migrating existing tests from NUnit to xUnit 
> > and I think it's doable with the suggestions that were proposed.  
> > The only question I need answered from the community is:
> >
> > "How important are timeouts in your test cases?"
> > - xUnit runs tests asynchronously and parallel, so they removed the 
> > ability to have timeout because the ability to time these tests are 
> > not reliable.
> > - There are 4 timeouts that I see in the cases.
> >         - int.MaxValue
> >         - 5 minutes
> >         - 2.5 minutes
> >         - 40000 milliseconds
> > My current approach to dealing with timeouts is removing them 
> > because the tests run concurrently, so we aren’t waiting for a single test to
finish.
> >
> > Thanks!
> > Connie
> >
> > -----Original Message-----
> > From: Elizabeth Maher (NEWMAN) 
> > [mailto:Elizabeth.Maher@microsoft.com]
> > Sent: Monday, June 20, 2016 12:26 PM
> > To: dev@lucenenet.apache.org
> > Subject: RE: Lucene.NET to .NET Core
> >
> > Our goal with the tests was to keep it working with the existing 
> > system with a few changes as possible.  We thought there would be 
> > more changes than the community was comfortable with.  However, it 
> > sounds like the interest in xUnit is more that we realized.
> >
> > There are three items that concerned us in the xUnit move.
> > 1.  No categories - Traits can be used instead.  Hopefully just some 
> > simple cut and paste.
> > 2.  No timeout for test cases - This is the biggest issue as tc 
> > timeout
> is
> > frequently used in the test cases I saw.  We are not sure the effect 
> > of this or how to overcome any issues related to this.
> > 3.  TeamCity integration - Looks like Wyatt found a good article 
> > explaining how to do this.
> >
> > Perhaps Connie can look into the first two issues and Wyatt can look 
> > into the last one?  Connie will respond with an update by tomorrow 
> > evening and then we can decide together.
> >
> > On a sad note, I've been pulled off onto another project.  Connie 
> > will take point on this project and finish up necessary work.
> >
> > Thanks,
> > Elizabeth
> >
> >
> > -----Original Message-----
> > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > Sent: Monday, June 20, 2016 8:12 AM
> > To: dev@lucenenet.apache.org
> > Subject: Re: Lucene.NET to .NET Core
> >
> > Sorry for the delayed response, I was off at the beach last weekend.
> >
> > Not a whole lot of seat-time w/ xunit here. In terms of running it, 
> > there is a plugin, we can see if CodeBetter would be willing to 
> > install it. The hand-plumbed option doesn't seem to daunting either. 
> > In terms of
> categories
> > that is a better question but it seems like Xunit has a [Traits]
> attribute
> > that could be used -- see
> >
> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fstacko
> verflow.com%2fquestions%2f21791739%2fexecute-tests-based-on-xunit-filt
> ered-by-traits-in-teamcity&data=01%7c01%7cElizabeth.Maher%40microsoft.
> com%7c93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011d
> b47%7c1&sdata=SaJ%2b%2b5r2mjXE6s550uotyCpNxB9eTP5EH577DmSU8to%3d
> > for
> > an example.
> >
> > On Sun, Jun 19, 2016 at 5:49 AM Itamar Syn-Hershko 
> > <itamar@code972.com>
> > wrote:
> >
> > > "All these issues together lead us to the conclusion that we would 
> > > be doing a disservice to just blindly port to xUnit." - why do you 
> > > say that? if a move to xUnit benefits us on multiple occasions 
> > > (better API, .NET Core support, faster runs) this one-tine effort 
> > > would be
> worth
> > it - no?
> > >
> > > Wyatt, can you chime in perhaps - you have the most experience 
> > > getting the build system to run, and test categorisation
> > >
> > > Thanks
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fco
> > > de97
> > > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c93aa1f13228
> > > a414 
> > > 2f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Upe
> > > iUeT A6iiP7fV7dZ9Z9bKiMjgqo990K6xyt4D6LTs%3d | @synhershko 
> > > <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2f
> > > twit 
> > > ter.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.co
> > > m%7c 
> > > 93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db4
> > > 7%7c 1&sdata=kc%2fH3fEbR68ajKlN3pZ7vISmdPb%2bIRTHWrDTZ4FdplY%3d>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > > On Fri, Jun 17, 2016 at 7:06 AM, Elizabeth Maher (NEWMAN) < 
> > > Elizabeth.Maher@microsoft.com> wrote:
> > >
> > > > Hi all.  I have a quick update on our progress.
> > > >
> > > > We did a test of what it would be like to port Lucene.NET to use
> xUnit.
> > > > As Itamer guessed, it is quite a bit of work.  The basic work of 
> > > > renaming attributes and changing out package names and the like 
> > > > is a little
> > > tedious,
> > > > but it is doable.  The issues comes in because of the use of 
> > > > categories
> > > and
> > > > timeout attributes.  Creating categories in xUnit is possible, 
> > > > but rather complicated.  The bigger issue is that my research 
> > > > indicated that
> > > TeamCity
> > > > only has basic integration with xUnit and you have to run all 
> > > > the
> > tests.
> > > > Running only categories requires extra scripting.  The bigger 
> > > > issue was that there is no TimeOut attribute in xUnit so longer 
> > > > running tests are
> > > an
> > > > issue.  We would also have to re-arrange some the test fixture 
> > > > setup and teardown code into class constructors and dispose 
> > > > methods, but that work was doable.  All these issues together 
> > > > lead us to the conclusion that we would be doing a disservice to 
> > > > just blindly port
> to
> > xUnit.
> > > >
> > > > nUnit has just release an alpha package that runs on .NET Core RC2.
> > > > Connie is looking into getting working this week or next to so 
> > > > we can
> > > keep
> > > > the use of such features as testcase timeouts and categories.  
> > > > We'll send an update when we've tried out the alpha package.
> > > >
> > > > Thanks,
> > > > Elizabeth
> > > >
> > > > -----Original Message-----
> > > > From: Elizabeth Maher (NEWMAN)
> > > > [mailto:Elizabeth.Maher@microsoft.com]
> > > > Sent: Monday, May 23, 2016 4:40 PM
> > > > To: dev@lucenenet.apache.org
> > > > Subject: RE: Lucene.NET to .NET Core
> > > >
> > > > Thank you everyone for the feedback.
> > > >
> > > > It's sounds everyone is favor of the move to xUnit.  I've talked 
> > > > to a couple of my co-workers and they said the conversion can be 
> > > > a little tedious, but often worth the effort.  Give me a couple 
> > > > days to see if
> > > it’s
> > > > a rabbits hole.  I'll respond back if its more effort that I can
> > expend.
> > > >
> > > > Itamar,
> > > > Regarding your last question.  There are no more know tasks 
> > > > other than what I initially outlined.  Once I merge Connie's 
> > > > change in from master
> > > we
> > > > will have a building/ported assembly.  I just need to get the 
> > > > tests
> > > running
> > > > to verify everything works as expected.
> > > >
> > > > Thanks,
> > > > Elizabeth
> > > >
> > > > -----Original Message-----
> > > > From: itamar.synhershko@gmail.com 
> > > > [mailto:itamar.synhershko@gmail.com]
> > > On
> > > > Behalf Of Itamar Syn-Hershko
> > > > Sent: Friday, May 20, 2016 6:46 AM
> > > > To: dev@lucenenet.apache.org
> > > > Subject: Re: Lucene.NET to .NET Core
> > > >
> > > > Hey there,
> > > >
> > > > Thank you so much again for your great work here. Lots of good 
> > > > contributions we could definitely use.
> > > >
> > > > Moving to xUnit.Net was actually on my list of things to do - 
> > > > basically also because it's API is much better and more flexible 
> > > > (and less buggy!) and my experience with it was much better than 
> > > > with NUnit (which we use
> > > an
> > > > outdated version of). Some people claim xUnit is also faster, so 
> > > > hey
> > > > -
> > > why
> > > > not.
> > > >
> > > > Are you sure you can "easily move all tests to xUnit"? From what 
> > > > I can tell this will be a rather rigorous copy-paste job? While 
> > > > the JavaCompatibility bits we have (
> > > >
> > > >
> > > https://github.com/apache/lucenenet/tree/master/src/Lucene.Net.Tes
> > > tFra
> > > mework/JavaCompatibility
> > > > )
> > > > do contain centralized function like assertTrue() etc that save 
> > > > loads of time and effort porting tests, so that could be changed 
> > > > easily to use xUnit, there are many places where asserts etc 
> > > > where ported in full to
> > > the
> > > > NUnit API.
> > > >
> > > > I am definitely pro this change, but let's make sure we don't go 
> > > > down
> > > into
> > > > a rabbit hole first. Let me know what you think.
> > > >
> > > > With regards to the status update:
> > > >
> > > > 1. I made some last minor comments on the merge scheduler PR - 
> > > > we should be able to merge it within a few days now. Great work 
> > > > there
> > > >
> > > > 2. What is keeping us from merging your ICU etc changes? please 
> > > > note some subprojects are absolutely necessary for Lucene.NET to 
> > > > run on Core CLR as well - even though they are "sub" projects 
> > > > and not part of the Lucene
> > > core
> > > > itself. Queries and Analysis are probably the biggest ones 
> > > > (latter not fully ported yet, and it's the one with the ICU dep).
> > > >
> > > > 3. Other than what you posted - are there any other pending 
> > > > items to get this running on .NET Core?
> > > >
> > > > --
> > > >
> > > > Itamar Syn-Hershko
> > > >
> > > >
> > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fco
> > > de97
> > > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152df
> > > d451
> > > ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=HxE
> > > 52%2 bFAC6FJAN%2fifQ5V8v27tZZALzBuDw2yhbw%2bkc8%3d
> > > > | @synhershko <
> > > >
> > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2ft
> > > witt
> > > er.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.com
> > > %7c4
> > > 43f03152dfd451ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db47
> > > %7c1 &sdata=7iqRqwQwlcNbmwkckUjcS1MWjUPJi3cxLFfRdLc3BnQ%3d
> > > > >
> > > > Freelance Developer & Consultant Lucene.NET committer and PMC 
> > > > member
> > > >
> > > > On Fri, May 20, 2016 at 3:17 AM, Elizabeth Maher (NEWMAN) < 
> > > > Elizabeth.Maher@microsoft.com> wrote:
> > > >
> > > > > Lucene.NET Community,
> > > > >
> > > > > Connie and I continue to make progress on moving Lucene.NET to 
> > > > > .NET
> > > Core.
> > > > > The last challenge we face is to get the tests to successfully 
> > > > > run on .NET Core.  Currently the unit tests use NUnit.  There 
> > > > > is a .NET Core version called NUnitLite.  However, the issue 
> > > > > is there is some manual steps to get NUnitLite tests to work on .NET
Core.
> > > > > We would have to create console applications for each of the 
> > > > > test library, call the appropriate APIs to create a test 
> > > > > results file and then work with the CI system to manually upload
the results.
> > > > > Also, there is not Test Runner integration for NUnitLite.  
> > > > > While, this is all possible to do, I would like to propose a 
> > > > > different change.  I could easily move all the test to use 
> > > > > xUnit.  The would benefit Lucene.NET in the long run as there 
> > > > > are both full fx< 
> > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f
> > > > > %2fx
> > > > > unit
> > > > > .github.io%2fdocs%2fgetting-started-desktop.html&data=01%7c01%
> > > > > 7cEl
> > > > > izab
> > > > > eth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51eba%7
> > > > > c72f
> > > > > 988b
> > > > > f86f141af91ab2d7cd011db47%7c1&sdata=2orwADQfb%2fkpCykXkySekBp0
> > > > > Dxan
> > > > > w0WR
> > > > > rUupy%2fzRFTY%3d> and core fx< 
> > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f
> > > > > %2fx
> > > > > unit
> > > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html&data=01%7
> > > > > c01%
> > > > > 7cEl
> > > > > izabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51e
> > > > > ba%7
> > > > > c72f
> > > > > 988bf86f141af91ab2d7cd011db47%7c1&sdata=UGKx5nz6%2foaV2vt2LQDT
> > > > > qkpn
> > > > > 0g%2 bu2XXHzE4pDZ0K%2fhU%3d> versions of xUnit, there is a 
> > > > > TeamCity plugin that automatically works< 
> > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f
> > > > > %2fx
> > > > > unit
> > > > > .github.io%2fdocs%2fgetting-test-results-in-teamcity.html&data
> > > > > =01%
> > > > > 7c01
> > > > > %7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d38
> > > > > 0b51
> > > > > eba%
> > > > > 7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=an0p3MTPFYe24wQiP
> > > > > q3Ss F3Eb v3gwJwrr88TjoQ4DnM%3d>, as well as the ability to 
> > > > > run tests in Visual Studio.< 
> > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f
> > > > > %2fx
> > > > > unit
> > > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html%23run-tes
> > > > > ts-v
> > > > > s&da
> > > > > ta=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451
> > > > > ebb1
> > > > > c08d
> > > > > 380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BN%2f3X
> > > > > EmrU yKQu 8tdZtoghxSskpFBPP1N1YGxT7e45Vc%3d.>
> > > > > I believe the long-term maintenance cost for both full and 
> > > > > core fx tests would be lower using xUnit.  Is this an 
> > > > > acceptable direction for me to take the unit tests?  Please let me
know.
> > > > >
> > > > > Below is the progress of our migration work, for the curious.
> > > > >
> > > > > 1.       Remove use of SharpZipLib dependency - completed
> > > > >
> > > > > 2.       Update ICU4NET dependency for .NET Core - completed in dev
> > > > branch
> > > > >
> > > > > a.        In https://github.com/conniey/lucenenet/tree/move2dnx.
> > > > >
> > > > > 3.       Remove Appache.NMS dependency - completed
> > > > >
> > > > > 4.       Move Scheduler to use Tasks - PR submitted
> > > > >
> > > > > a.        Pull request at
> > https://github.com/apache/lucenenet/pull/171
> > > .
> > > > >
> > > > > 5.       Make portable libraries to build .NET Core binaries -
> > > completed
> > > > > in dev branch
> > > > >
> > > > > a.        In https://github.com/conniey/lucenenet/tree/move2dnx.
> > > > >
> > > > > 6.       Verify tests pass on both full framework and .Net Core.
-
> In
> > > > > progress
> > > > >
> > > > > a.        Need to get tests to run on .NET Core.
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Elizabeth
> > > > >
> > > >
> > >
> >
>
Mime
View raw message