lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wyatt Barnett <wyatt.barn...@gmail.com>
Subject Re: Lucene.NET to .NET Core
Date Wed, 22 Jun 2016 13:26:52 GMT
I don't know the history on why the timeouts were added but I suspect they
were self-defensive in nature. I think in most cases the timeouts are
really back-stopping things so that if there is a race condition the test
suite bails and fails rather than runs forever. Is there a way to backstop
stuff like that with xunit?


On Tue, Jun 21, 2016 at 8:24 PM Connie Yau <conniey@microsoft.com> wrote:

> Hi,
>
> I've been looking into migrating existing tests from NUnit to xUnit and I
> think it's doable with the suggestions that were proposed.  The only
> question I need answered from the community is:
>
> "How important are timeouts in your test cases?"
> - xUnit runs tests asynchronously and parallel, so they removed the
> ability to have timeout because the ability to time these tests are not
> reliable.
> - There are 4 timeouts that I see in the cases.
>         - int.MaxValue
>         - 5 minutes
>         - 2.5 minutes
>         - 40000 milliseconds
> My current approach to dealing with timeouts is removing them because the
> tests run concurrently, so we aren’t waiting for a single test to finish.
>
> Thanks!
> Connie
>
> -----Original Message-----
> From: Elizabeth Maher (NEWMAN) [mailto:Elizabeth.Maher@microsoft.com]
> Sent: Monday, June 20, 2016 12:26 PM
> To: dev@lucenenet.apache.org
> Subject: RE: Lucene.NET to .NET Core
>
> Our goal with the tests was to keep it working with the existing system
> with a few changes as possible.  We thought there would be more changes
> than the community was comfortable with.  However, it sounds like the
> interest in xUnit is more that we realized.
>
> There are three items that concerned us in the xUnit move.
> 1.  No categories - Traits can be used instead.  Hopefully just some
> simple cut and paste.
> 2.  No timeout for test cases - This is the biggest issue as tc timeout is
> frequently used in the test cases I saw.  We are not sure the effect of
> this or how to overcome any issues related to this.
> 3.  TeamCity integration - Looks like Wyatt found a good article
> explaining how to do this.
>
> Perhaps Connie can look into the first two issues and Wyatt can look into
> the last one?  Connie will respond with an update by tomorrow evening and
> then we can decide together.
>
> On a sad note, I've been pulled off onto another project.  Connie will
> take point on this project and finish up necessary work.
>
> Thanks,
> Elizabeth
>
>
> -----Original Message-----
> From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> Sent: Monday, June 20, 2016 8:12 AM
> To: dev@lucenenet.apache.org
> Subject: Re: Lucene.NET to .NET Core
>
> Sorry for the delayed response, I was off at the beach last weekend.
>
> Not a whole lot of seat-time w/ xunit here. In terms of running it, there
> is a plugin, we can see if CodeBetter would be willing to install it. The
> hand-plumbed option doesn't seem to daunting either. In terms of categories
> that is a better question but it seems like Xunit has a [Traits] attribute
> that could be used -- see
> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fstackoverflow.com%2fquestions%2f21791739%2fexecute-tests-based-on-xunit-filtered-by-traits-in-teamcity&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=SaJ%2b%2b5r2mjXE6s550uotyCpNxB9eTP5EH577DmSU8to%3d
> for
> an example.
>
> On Sun, Jun 19, 2016 at 5:49 AM Itamar Syn-Hershko <itamar@code972.com>
> wrote:
>
> > "All these issues together lead us to the conclusion that we would be
> > doing a disservice to just blindly port to xUnit." - why do you say
> > that? if a move to xUnit benefits us on multiple occasions (better
> > API, .NET Core support, faster runs) this one-tine effort would be worth
> it - no?
> >
> > Wyatt, can you chime in perhaps - you have the most experience getting
> > the build system to run, and test categorisation
> >
> > Thanks
> >
> > --
> >
> > Itamar Syn-Hershko
> > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcode97
> > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c93aa1f13228a414
> > 2f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=UpeiUeT
> > A6iiP7fV7dZ9Z9bKiMjgqo990K6xyt4D6LTs%3d | @synhershko
> > <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2ftwit
> > ter.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c
> > 93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db47%7c
> > 1&sdata=kc%2fH3fEbR68ajKlN3pZ7vISmdPb%2bIRTHWrDTZ4FdplY%3d>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Fri, Jun 17, 2016 at 7:06 AM, Elizabeth Maher (NEWMAN) <
> > Elizabeth.Maher@microsoft.com> wrote:
> >
> > > Hi all.  I have a quick update on our progress.
> > >
> > > We did a test of what it would be like to port Lucene.NET to use xUnit.
> > > As Itamer guessed, it is quite a bit of work.  The basic work of
> > > renaming attributes and changing out package names and the like is a
> > > little
> > tedious,
> > > but it is doable.  The issues comes in because of the use of
> > > categories
> > and
> > > timeout attributes.  Creating categories in xUnit is possible, but
> > > rather complicated.  The bigger issue is that my research indicated
> > > that
> > TeamCity
> > > only has basic integration with xUnit and you have to run all the
> tests.
> > > Running only categories requires extra scripting.  The bigger issue
> > > was that there is no TimeOut attribute in xUnit so longer running
> > > tests are
> > an
> > > issue.  We would also have to re-arrange some the test fixture setup
> > > and teardown code into class constructors and dispose methods, but
> > > that work was doable.  All these issues together lead us to the
> > > conclusion that we would be doing a disservice to just blindly port to
> xUnit.
> > >
> > > nUnit has just release an alpha package that runs on .NET Core RC2.
> > > Connie is looking into getting working this week or next to so we
> > > can
> > keep
> > > the use of such features as testcase timeouts and categories.  We'll
> > > send an update when we've tried out the alpha package.
> > >
> > > Thanks,
> > > Elizabeth
> > >
> > > -----Original Message-----
> > > From: Elizabeth Maher (NEWMAN)
> > > [mailto:Elizabeth.Maher@microsoft.com]
> > > Sent: Monday, May 23, 2016 4:40 PM
> > > To: dev@lucenenet.apache.org
> > > Subject: RE: Lucene.NET to .NET Core
> > >
> > > Thank you everyone for the feedback.
> > >
> > > It's sounds everyone is favor of the move to xUnit.  I've talked to
> > > a couple of my co-workers and they said the conversion can be a
> > > little tedious, but often worth the effort.  Give me a couple days
> > > to see if
> > it’s
> > > a rabbits hole.  I'll respond back if its more effort that I can
> expend.
> > >
> > > Itamar,
> > > Regarding your last question.  There are no more know tasks other
> > > than what I initially outlined.  Once I merge Connie's change in
> > > from master
> > we
> > > will have a building/ported assembly.  I just need to get the tests
> > running
> > > to verify everything works as expected.
> > >
> > > Thanks,
> > > Elizabeth
> > >
> > > -----Original Message-----
> > > From: itamar.synhershko@gmail.com
> > > [mailto:itamar.synhershko@gmail.com]
> > On
> > > Behalf Of Itamar Syn-Hershko
> > > Sent: Friday, May 20, 2016 6:46 AM
> > > To: dev@lucenenet.apache.org
> > > Subject: Re: Lucene.NET to .NET Core
> > >
> > > Hey there,
> > >
> > > Thank you so much again for your great work here. Lots of good
> > > contributions we could definitely use.
> > >
> > > Moving to xUnit.Net was actually on my list of things to do -
> > > basically also because it's API is much better and more flexible
> > > (and less buggy!) and my experience with it was much better than
> > > with NUnit (which we use
> > an
> > > outdated version of). Some people claim xUnit is also faster, so hey
> > > -
> > why
> > > not.
> > >
> > > Are you sure you can "easily move all tests to xUnit"? From what I
> > > can tell this will be a rather rigorous copy-paste job? While the
> > > JavaCompatibility bits we have (
> > >
> > >
> > https://github.com/apache/lucenenet/tree/master/src/Lucene.Net.TestFra
> > mework/JavaCompatibility
> > > )
> > > do contain centralized function like assertTrue() etc that save
> > > loads of time and effort porting tests, so that could be changed
> > > easily to use xUnit, there are many places where asserts etc where
> > > ported in full to
> > the
> > > NUnit API.
> > >
> > > I am definitely pro this change, but let's make sure we don't go
> > > down
> > into
> > > a rabbit hole first. Let me know what you think.
> > >
> > > With regards to the status update:
> > >
> > > 1. I made some last minor comments on the merge scheduler PR - we
> > > should be able to merge it within a few days now. Great work there
> > >
> > > 2. What is keeping us from merging your ICU etc changes? please note
> > > some subprojects are absolutely necessary for Lucene.NET to run on
> > > Core CLR as well - even though they are "sub" projects and not part
> > > of the Lucene
> > core
> > > itself. Queries and Analysis are probably the biggest ones (latter
> > > not fully ported yet, and it's the one with the ICU dep).
> > >
> > > 3. Other than what you posted - are there any other pending items to
> > > get this running on .NET Core?
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > >
> > >
> > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcode97
> > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451
> > ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=HxE52%2
> > bFAC6FJAN%2fifQ5V8v27tZZALzBuDw2yhbw%2bkc8%3d
> > > | @synhershko <
> > >
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2ftwitt
> > er.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c4
> > 43f03152dfd451ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1
> > &sdata=7iqRqwQwlcNbmwkckUjcS1MWjUPJi3cxLFfRdLc3BnQ%3d
> > > >
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > > On Fri, May 20, 2016 at 3:17 AM, Elizabeth Maher (NEWMAN) <
> > > Elizabeth.Maher@microsoft.com> wrote:
> > >
> > > > Lucene.NET Community,
> > > >
> > > > Connie and I continue to make progress on moving Lucene.NET to
> > > > .NET
> > Core.
> > > > The last challenge we face is to get the tests to successfully run
> > > > on .NET Core.  Currently the unit tests use NUnit.  There is a
> > > > .NET Core version called NUnitLite.  However, the issue is there
> > > > is some manual steps to get NUnitLite tests to work on .NET Core.
> > > > We would have to create console applications for each of the test
> > > > library, call the appropriate APIs to create a test results file
> > > > and then work with the CI system to manually upload the results.
> > > > Also, there is not Test Runner integration for NUnitLite.  While,
> > > > this is all possible to do, I would like to propose a different
> > > > change.  I could easily move all the test to use xUnit.  The would
> > > > benefit Lucene.NET in the long run as there are both full fx<
> > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fx
> > > > unit
> > > > .github.io%2fdocs%2fgetting-started-desktop.html&data=01%7c01%7cEl
> > > > izab
> > > > eth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51eba%7c72f
> > > > 988b
> > > > f86f141af91ab2d7cd011db47%7c1&sdata=2orwADQfb%2fkpCykXkySekBp0Dxan
> > > > w0WR
> > > > rUupy%2fzRFTY%3d> and core fx<
> > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fx
> > > > unit
> > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html&data=01%7c01%
> > > > 7cEl
> > > > izabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51eba%7
> > > > c72f
> > > > 988bf86f141af91ab2d7cd011db47%7c1&sdata=UGKx5nz6%2foaV2vt2LQDTqkpn
> > > > 0g%2 bu2XXHzE4pDZ0K%2fhU%3d> versions of xUnit, there is a
> > > > TeamCity plugin that automatically works<
> > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fx
> > > > unit
> > > > .github.io%2fdocs%2fgetting-test-results-in-teamcity.html&data=01%
> > > > 7c01
> > > > %7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51
> > > > eba%
> > > > 7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=an0p3MTPFYe24wQiPq3Ss
> > > > F3Eb v3gwJwrr88TjoQ4DnM%3d>, as well as the ability to run tests
> > > > in Visual Studio.<
> > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fx
> > > > unit
> > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html%23run-tests-v
> > > > s&da
> > > > ta=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1
> > > > c08d
> > > > 380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BN%2f3XEmrU
> > > > yKQu 8tdZtoghxSskpFBPP1N1YGxT7e45Vc%3d.>
> > > > I believe the long-term maintenance cost for both full and core fx
> > > > tests would be lower using xUnit.  Is this an acceptable direction
> > > > for me to take the unit tests?  Please let me know.
> > > >
> > > > Below is the progress of our migration work, for the curious.
> > > >
> > > > 1.       Remove use of SharpZipLib dependency - completed
> > > >
> > > > 2.       Update ICU4NET dependency for .NET Core - completed in dev
> > > branch
> > > >
> > > > a.        In https://github.com/conniey/lucenenet/tree/move2dnx.
> > > >
> > > > 3.       Remove Appache.NMS dependency - completed
> > > >
> > > > 4.       Move Scheduler to use Tasks - PR submitted
> > > >
> > > > a.        Pull request at
> https://github.com/apache/lucenenet/pull/171
> > .
> > > >
> > > > 5.       Make portable libraries to build .NET Core binaries -
> > completed
> > > > in dev branch
> > > >
> > > > a.        In https://github.com/conniey/lucenenet/tree/move2dnx.
> > > >
> > > > 6.       Verify tests pass on both full framework and .Net Core. - In
> > > > progress
> > > >
> > > > a.        Need to get tests to run on .NET Core.
> > > >
> > > >
> > > > Thanks,
> > > > Elizabeth
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message