lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Itamar Syn-Hershko <ita...@code972.com>
Subject Re: Lucene.NET to .NET Core
Date Thu, 07 Jul 2016 15:28:54 GMT
I don't have an easy answer then. The obvious answer would be to change
this to a Config object that's passed around or similar - or make them
immutable and somehow change that config in the test configuration. A good
place to start is to track the usage, and confirm the intended usage
pattern with the Java codebase just in case.

About both issues, timeouts and mutable configs - I trust your good
judgment here. What's important is faster running and more stable tests, we
can skip Java compatiblilty in the testsframework as long as it's
documented and gives us large benefits.

Thanks!

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Tue, Jun 28, 2016 at 2:42 AM, Connie Yau <conniey@microsoft.com> wrote:

> Yes, all of the statics are mutable. And some of the test cases make use
> of this... An example is: OLD_FORMAT_IMPERSONATION_IS_ACTIVE.  The old
> codec tests modify this field from true/false.
>
> About the Timeout issue, I like Laimonas' suggestion to use a Trait to
> ignore running this specific tests until they've been properly
> investigated. Brad points out in this issue that there is no way to
> accurate way to measure Timeouts because of the parallization in xUnit (
> https://github.com/xunit/xunit/issues/217).
>
> Thanks,
> Connie
>
> -----Original Message-----
> From: itamar.synhershko@gmail.com [mailto:itamar.synhershko@gmail.com] On
> Behalf Of Itamar Syn-Hershko
> Sent: Saturday, June 25, 2016 12:52 PM
> To: dev@lucenenet.apache.org
> Subject: Re: Lucene.NET to .NET Core
>
> I believe the statics are remnants of the Lucene Java port, and mostly
> readonly configurations. Are any of those statics mutable?
>
> Adding on the timeout thing - this indeed was a precaution. We should
> probably move from a attribute-based timeout to throwing a TimeoutException
> from the main test method, having the main loop check for the time passed
> since start and throw if it's above the timeout configured?
>
> --
>
> Itamar Syn-Hershko
>
> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcode972.com&data=01%7c01%7cconniey%40microsoft.com%7c7e667aa0b59d4958c19b08d39d3236bc%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=RfcfWtLeiAc%2b3JDIALSdcDTae5%2bA7PwdVbVQAAU%2fh1U%3d
> | @synhershko <
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2ftwitter.com%2fsynhershko&data=01%7c01%7cconniey%40microsoft.com%7c7e667aa0b59d4958c19b08d39d3236bc%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=%2fqmKXKltLCaz%2bpDdzAPJRhKlv5hRojyT41IydWUFD9o%3d
> >
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Fri, Jun 24, 2016 at 10:12 PM, Connie Yau <conniey@microsoft.com>
> wrote:
>
> > Hi,
> >
> > Thank you for the input! I will add a Trait for the tests that have
> > Timeouts for now so they can be fixed to be more stable.
> >
> > One more question... While migrating the code to xUnit, I noticed that
> > there are a lot of `static` variables in LuceneTestCase and which
> > would make it hard to run in parallel because some test classes modify
> > the static values. Is there a reason for keeping them static?
> >
> > Thanks!
> > Connie
> >
> > -----Original Message-----
> > From: Laimonas Simutis [mailto:laimis@gmail.com]
> > Sent: Wednesday, June 22, 2016 2:54 PM
> > To: dev@lucenenet.apache.org
> > Subject: Re: Lucene.NET to .NET Core
> >
> > Wyatt is right on. Certain tests under random circumstances could get
> > into a bad state and run "forever", potentially generate gigs of data
> > on disk, etc and so timeout was added to abort the test earlier in
> > case that happened.
> >
> > Our plan was to make a pass at those long running tests and fix the
> > issues that cause them to run so long once we are fully complete with
> the porting.
> >
> > Perhaps you can add a category / trait on such tests in order to avoid
> > them until they can be properly investigated and underlying issues fixed?
> >
> >
> >
> > On Wed, Jun 22, 2016 at 9:26 AM, Wyatt Barnett
> > <wyatt.barnett@gmail.com>
> > wrote:
> >
> > > I don't know the history on why the timeouts were added but I
> > > suspect they were self-defensive in nature. I think in most cases
> > > the timeouts are really back-stopping things so that if there is a
> > > race condition the test suite bails and fails rather than runs
> > > forever. Is there a way to backstop stuff like that with xunit?
> > >
> > >
> > > On Tue, Jun 21, 2016 at 8:24 PM Connie Yau <conniey@microsoft.com>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I've been looking into migrating existing tests from NUnit to
> > > > xUnit and I think it's doable with the suggestions that were
> proposed.
> > > > The only question I need answered from the community is:
> > > >
> > > > "How important are timeouts in your test cases?"
> > > > - xUnit runs tests asynchronously and parallel, so they removed
> > > > the ability to have timeout because the ability to time these
> > > > tests are not reliable.
> > > > - There are 4 timeouts that I see in the cases.
> > > >         - int.MaxValue
> > > >         - 5 minutes
> > > >         - 2.5 minutes
> > > >         - 40000 milliseconds
> > > > My current approach to dealing with timeouts is removing them
> > > > because the tests run concurrently, so we aren’t waiting for a
> > > > single
> > test to finish.
> > > >
> > > > Thanks!
> > > > Connie
> > > >
> > > > -----Original Message-----
> > > > From: Elizabeth Maher (NEWMAN)
> > > > [mailto:Elizabeth.Maher@microsoft.com]
> > > > Sent: Monday, June 20, 2016 12:26 PM
> > > > To: dev@lucenenet.apache.org
> > > > Subject: RE: Lucene.NET to .NET Core
> > > >
> > > > Our goal with the tests was to keep it working with the existing
> > > > system with a few changes as possible.  We thought there would be
> > > > more changes than the community was comfortable with.  However, it
> > > > sounds like the interest in xUnit is more that we realized.
> > > >
> > > > There are three items that concerned us in the xUnit move.
> > > > 1.  No categories - Traits can be used instead.  Hopefully just
> > > > some simple cut and paste.
> > > > 2.  No timeout for test cases - This is the biggest issue as tc
> > > > timeout
> > > is
> > > > frequently used in the test cases I saw.  We are not sure the
> > > > effect of this or how to overcome any issues related to this.
> > > > 3.  TeamCity integration - Looks like Wyatt found a good article
> > > > explaining how to do this.
> > > >
> > > > Perhaps Connie can look into the first two issues and Wyatt can
> > > > look into the last one?  Connie will respond with an update by
> > > > tomorrow evening and then we can decide together.
> > > >
> > > > On a sad note, I've been pulled off onto another project.  Connie
> > > > will take point on this project and finish up necessary work.
> > > >
> > > > Thanks,
> > > > Elizabeth
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > > > Sent: Monday, June 20, 2016 8:12 AM
> > > > To: dev@lucenenet.apache.org
> > > > Subject: Re: Lucene.NET to .NET Core
> > > >
> > > > Sorry for the delayed response, I was off at the beach last weekend.
> > > >
> > > > Not a whole lot of seat-time w/ xunit here. In terms of running
> > > > it, there is a plugin, we can see if CodeBetter would be willing
> > > > to install it. The hand-plumbed option doesn't seem to daunting
> either.
> > > > In terms of
> > > categories
> > > > that is a better question but it seems like Xunit has a [Traits]
> > > attribute
> > > > that could be used -- see
> > > >
> > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fstac
> > > ko
> > > verflow.com%2fquestions%2f21791739%2fexecute-tests-based-on-xunit-fi
> > > lt
> > > ered-by-traits-in-teamcity&data=01%7c01%7cElizabeth.Maher%40microsoft.
> > > com%7c93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd01
> > > 1d b47%7c1&sdata=SaJ%2b%2b5r2mjXE6s550uotyCpNxB9eTP5EH577DmSU8to%3d
> > > > for
> > > > an example.
> > > >
> > > > On Sun, Jun 19, 2016 at 5:49 AM Itamar Syn-Hershko
> > > > <itamar@code972.com>
> > > > wrote:
> > > >
> > > > > "All these issues together lead us to the conclusion that we
> > > > > would be doing a disservice to just blindly port to xUnit." -
> > > > > why do you say that? if a move to xUnit benefits us on multiple
> > > > > occasions (better API, .NET Core support, faster runs) this
> > > > > one-tine effort would be
> > > worth
> > > > it - no?
> > > > >
> > > > > Wyatt, can you chime in perhaps - you have the most experience
> > > > > getting the build system to run, and test categorisation
> > > > >
> > > > > Thanks
> > > > >
> > > > > --
> > > > >
> > > > > Itamar Syn-Hershko
> > > > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f
> > > > > co
> > > > > de97
> > > > > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c93aa1f132
> > > > > 28
> > > > > a414
> > > > > 2f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=U
> > > > > pe iUeT A6iiP7fV7dZ9Z9bKiMjgqo990K6xyt4D6LTs%3d | @synhershko
> > > > > <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%
> > > > > 2f
> > > > > twit
> > > > > ter.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.
> > > > > co
> > > > > m%7c
> > > > > 93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011d
> > > > > b4 7%7c
> > > > > 1&sdata=kc%2fH3fEbR68ajKlN3pZ7vISmdPb%2bIRTHWrDTZ4FdplY%3d>
> > > > > Freelance Developer & Consultant Lucene.NET committer and PMC
> > > > > member
> > > > >
> > > > > On Fri, Jun 17, 2016 at 7:06 AM, Elizabeth Maher (NEWMAN) <
> > > > > Elizabeth.Maher@microsoft.com> wrote:
> > > > >
> > > > > > Hi all.  I have a quick update on our progress.
> > > > > >
> > > > > > We did a test of what it would be like to port Lucene.NET to
> > > > > > use
> > > xUnit.
> > > > > > As Itamer guessed, it is quite a bit of work.  The basic work
> > > > > > of renaming attributes and changing out package names and the
> > > > > > like is a little
> > > > > tedious,
> > > > > > but it is doable.  The issues comes in because of the use of
> > > > > > categories
> > > > > and
> > > > > > timeout attributes.  Creating categories in xUnit is possible,
> > > > > > but rather complicated.  The bigger issue is that my research
> > > > > > indicated that
> > > > > TeamCity
> > > > > > only has basic integration with xUnit and you have to run all
> > > > > > the
> > > > tests.
> > > > > > Running only categories requires extra scripting.  The bigger
> > > > > > issue was that there is no TimeOut attribute in xUnit so
> > > > > > longer running tests are
> > > > > an
> > > > > > issue.  We would also have to re-arrange some the test fixture
> > > > > > setup and teardown code into class constructors and dispose
> > > > > > methods, but that work was doable.  All these issues together
> > > > > > lead us to the conclusion that we would be doing a disservice
> > > > > > to just blindly port
> > > to
> > > > xUnit.
> > > > > >
> > > > > > nUnit has just release an alpha package that runs on .NET Core
> RC2.
> > > > > > Connie is looking into getting working this week or next to
so
> > > > > > we can
> > > > > keep
> > > > > > the use of such features as testcase timeouts and categories.
> > > > > > We'll send an update when we've tried out the alpha package.
> > > > > >
> > > > > > Thanks,
> > > > > > Elizabeth
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Elizabeth Maher (NEWMAN)
> > > > > > [mailto:Elizabeth.Maher@microsoft.com]
> > > > > > Sent: Monday, May 23, 2016 4:40 PM
> > > > > > To: dev@lucenenet.apache.org
> > > > > > Subject: RE: Lucene.NET to .NET Core
> > > > > >
> > > > > > Thank you everyone for the feedback.
> > > > > >
> > > > > > It's sounds everyone is favor of the move to xUnit.  I've
> > > > > > talked to a couple of my co-workers and they said the
> > > > > > conversion can be a little tedious, but often worth the
> > > > > > effort.  Give me a couple days to see if
> > > > > it’s
> > > > > > a rabbits hole.  I'll respond back if its more effort that I
> > > > > > can
> > > > expend.
> > > > > >
> > > > > > Itamar,
> > > > > > Regarding your last question.  There are no more know tasks
> > > > > > other than what I initially outlined.  Once I merge Connie's
> > > > > > change in from master
> > > > > we
> > > > > > will have a building/ported assembly.  I just need to get the
> > > > > > tests
> > > > > running
> > > > > > to verify everything works as expected.
> > > > > >
> > > > > > Thanks,
> > > > > > Elizabeth
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: itamar.synhershko@gmail.com
> > > > > > [mailto:itamar.synhershko@gmail.com]
> > > > > On
> > > > > > Behalf Of Itamar Syn-Hershko
> > > > > > Sent: Friday, May 20, 2016 6:46 AM
> > > > > > To: dev@lucenenet.apache.org
> > > > > > Subject: Re: Lucene.NET to .NET Core
> > > > > >
> > > > > > Hey there,
> > > > > >
> > > > > > Thank you so much again for your great work here. Lots of good
> > > > > > contributions we could definitely use.
> > > > > >
> > > > > > Moving to xUnit.Net was actually on my list of things to do
-
> > > > > > basically also because it's API is much better and more
> > > > > > flexible (and less buggy!) and my experience with it was much
> > > > > > better than with NUnit (which we use
> > > > > an
> > > > > > outdated version of). Some people claim xUnit is also faster,
> > > > > > so hey
> > > > > > -
> > > > > why
> > > > > > not.
> > > > > >
> > > > > > Are you sure you can "easily move all tests to xUnit"? From
> > > > > > what I can tell this will be a rather rigorous copy-paste job?
> > > > > > While the JavaCompatibility bits we have (
> > > > > >
> > > > > >
> > > > > https://github.com/apache/lucenenet/tree/master/src/Lucene.Net.T
> > > > > es
> > > > > tFra
> > > > > mework/JavaCompatibility
> > > > > > )
> > > > > > do contain centralized function like assertTrue() etc that
> > > > > > save loads of time and effort porting tests, so that could be
> > > > > > changed easily to use xUnit, there are many places where
> > > > > > asserts etc where ported in full to
> > > > > the
> > > > > > NUnit API.
> > > > > >
> > > > > > I am definitely pro this change, but let's make sure we don't
> > > > > > go down
> > > > > into
> > > > > > a rabbit hole first. Let me know what you think.
> > > > > >
> > > > > > With regards to the status update:
> > > > > >
> > > > > > 1. I made some last minor comments on the merge scheduler PR
-
> > > > > > we should be able to merge it within a few days now. Great
> > > > > > work there
> > > > > >
> > > > > > 2. What is keeping us from merging your ICU etc changes?
> > > > > > please note some subprojects are absolutely necessary for
> > > > > > Lucene.NET to run on Core CLR as well - even though they are
> > > > > > "sub" projects and not part of the Lucene
> > > > > core
> > > > > > itself. Queries and Analysis are probably the biggest ones
> > > > > > (latter not fully ported yet, and it's the one with the ICU
dep).
> > > > > >
> > > > > > 3. Other than what you posted - are there any other pending
> > > > > > items to get this running on .NET Core?
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Itamar Syn-Hershko
> > > > > >
> > > > > >
> > > > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f
> > > > > co
> > > > > de97
> > > > > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152
> > > > > df
> > > > > d451
> > > > > ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=H
> > > > > xE
> > > > > 52%2 bFAC6FJAN%2fifQ5V8v27tZZALzBuDw2yhbw%2bkc8%3d
> > > > > > | @synhershko <
> > > > > >
> > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2
> > > > > ft
> > > > > witt
> > > > > er.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.c
> > > > > om
> > > > > %7c4
> > > > > 43f03152dfd451ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db
> > > > > 47
> > > > > %7c1 &sdata=7iqRqwQwlcNbmwkckUjcS1MWjUPJi3cxLFfRdLc3BnQ%3d
> > > > > > >
> > > > > > Freelance Developer & Consultant Lucene.NET committer and
PMC
> > > > > > member
> > > > > >
> > > > > > On Fri, May 20, 2016 at 3:17 AM, Elizabeth Maher (NEWMAN) <
> > > > > > Elizabeth.Maher@microsoft.com> wrote:
> > > > > >
> > > > > > > Lucene.NET Community,
> > > > > > >
> > > > > > > Connie and I continue to make progress on moving Lucene.NET
> > > > > > > to .NET
> > > > > Core.
> > > > > > > The last challenge we face is to get the tests to
> > > > > > > successfully run on .NET Core.  Currently the unit tests
use
> > > > > > > NUnit.  There is a .NET Core version called NUnitLite.
> > > > > > > However, the issue is there is some manual steps to get
> > > > > > > NUnitLite tests to work on
> > .NET Core.
> > > > > > > We would have to create console applications for each of
the
> > > > > > > test library, call the appropriate APIs to create a test
> > > > > > > results file and then work with the CI system to manually
> > > > > > > upload
> > the results.
> > > > > > > Also, there is not Test Runner integration for NUnitLite.
> > > > > > > While, this is all possible to do, I would like to propose
a
> > > > > > > different change.  I could easily move all the test to
use
> > > > > > > xUnit.  The would benefit Lucene.NET in the long run as
> > > > > > > there are both full fx<
> > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%
> > > > > > > 2f
> > > > > > > %2fx
> > > > > > > unit
> > > > > > > .github.io%2fdocs%2fgetting-started-desktop.html&data=01%7c0
> > > > > > > 1%
> > > > > > > 7cEl
> > > > > > > izab
> > > > > > > eth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51eba
> > > > > > > %7
> > > > > > > c72f
> > > > > > > 988b
> > > > > > > f86f141af91ab2d7cd011db47%7c1&sdata=2orwADQfb%2fkpCykXkySekB
> > > > > > > p0
> > > > > > > Dxan
> > > > > > > w0WR
> > > > > > > rUupy%2fzRFTY%3d> and core fx<
> > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%
> > > > > > > 2f
> > > > > > > %2fx
> > > > > > > unit
> > > > > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html&data=01
> > > > > > > %7
> > > > > > > c01%
> > > > > > > 7cEl
> > > > > > > izabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b5
> > > > > > > 1e
> > > > > > > ba%7
> > > > > > > c72f
> > > > > > > 988bf86f141af91ab2d7cd011db47%7c1&sdata=UGKx5nz6%2foaV2vt2LQ
> > > > > > > DT
> > > > > > > qkpn
> > > > > > > 0g%2 bu2XXHzE4pDZ0K%2fhU%3d> versions of xUnit, there
is a
> > > > > > > TeamCity plugin that automatically works<
> > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%
> > > > > > > 2f
> > > > > > > %2fx
> > > > > > > unit
> > > > > > > .github.io%2fdocs%2fgetting-test-results-in-teamcity.html&da
> > > > > > > ta
> > > > > > > =01%
> > > > > > > 7c01
> > > > > > > %7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d
> > > > > > > 38
> > > > > > > 0b51
> > > > > > > eba%
> > > > > > > 7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=an0p3MTPFYe24wQ
> > > > > > > iP q3Ss F3Eb v3gwJwrr88TjoQ4DnM%3d>, as well as the
ability
> > > > > > > to run tests in Visual Studio.<
> > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%
> > > > > > > 2f
> > > > > > > %2fx
> > > > > > > unit
> > > > > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html%23run-t
> > > > > > > es
> > > > > > > ts-v
> > > > > > > s&da
> > > > > > > ta=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152dfd4
> > > > > > > 51
> > > > > > > ebb1
> > > > > > > c08d
> > > > > > > 380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BN%2f
> > > > > > > 3X EmrU yKQu 8tdZtoghxSskpFBPP1N1YGxT7e45Vc%3d.>
> > > > > > > I believe the long-term maintenance cost for both full
and
> > > > > > > core fx tests would be lower using xUnit.  Is this an
> > > > > > > acceptable direction for me to take the unit tests?  Please
> > > > > > > let
> > me know.
> > > > > > >
> > > > > > > Below is the progress of our migration work, for the curious.
> > > > > > >
> > > > > > > 1.       Remove use of SharpZipLib dependency - completed
> > > > > > >
> > > > > > > 2.       Update ICU4NET dependency for .NET Core - completed
in
> > dev
> > > > > > branch
> > > > > > >
> > > > > > > a.        In
> https://github.com/conniey/lucenenet/tree/move2dnx.
> > > > > > >
> > > > > > > 3.       Remove Appache.NMS dependency - completed
> > > > > > >
> > > > > > > 4.       Move Scheduler to use Tasks - PR submitted
> > > > > > >
> > > > > > > a.        Pull request at
> > > > https://github.com/apache/lucenenet/pull/171
> > > > > .
> > > > > > >
> > > > > > > 5.       Make portable libraries to build .NET Core binaries
-
> > > > > completed
> > > > > > > in dev branch
> > > > > > >
> > > > > > > a.        In
> https://github.com/conniey/lucenenet/tree/move2dnx.
> > > > > > >
> > > > > > > 6.       Verify tests pass on both full framework and .Net
> Core.
> > -
> > > In
> > > > > > > progress
> > > > > > >
> > > > > > > a.        Need to get tests to run on .NET Core.
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Elizabeth
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message