lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wyatt Barnett <wyatt.barn...@gmail.com>
Subject Re: API Work/Stabilization Update
Date Sat, 01 Apr 2017 00:52:28 GMT
I wanted to walk back to one thing you asked about the tests Shad:

"Worst case, we can just turn off verbosity on the build, but it might be
helpful to leave it on if something goes wrong"

I *think* I'm turning the verbosity off in the tests -- they should be
running in release mode. They would not even run in debug if I recall
correctly. Let me know how the switch works and I'll make sure I'm throwing
it on the build server side.

On Fri, Mar 31, 2017 at 6:56 PM Shad Storhaug <shad@shadstorhaug.com> wrote:

> I don't necessarily need ownership access, but he did give me ownership to
> the https://www.nuget.org/packages/Spatial4n.Core/ package.
> Alternatively, Itamar can enter his keys on
> https://www.myget.org/feed/Security/spatial4n - he already has ownership.
> Once the keys are added there, I will be able to push.
>
> -----Original Message-----
> From: Prescott Nasser [mailto:geobmx540@hotmail.com]
> Sent: Saturday, April 1, 2017 5:49 AM
> To: dev@lucenenet.apache.org
> Subject: RE: API Work/Stabilization Update
>
> Access like ownership access? Paging Itamar..
>
>
>
> -----Original Message-----
> From: Shad Storhaug [mailto:shad@shadstorhaug.com]
> Sent: Friday, March 31, 2017 3:38 PM
> To: dev@lucenenet.apache.org
> Subject: RE: API Work/Stabilization Update
>
> BTW - the contrib NuGet package is now dead - the functionality has been
> moved into new sub-projects just like in Lucene.
>
> But, that reminds me - I still don't have access to
> https://www.nuget.org/packages/Spatial4n.Core.NTS/, so I have been unable
> to update it to the same version that
> https://www.nuget.org/packages/Spatial4n.Core/ is on. NTS is not required
> by Lucene.Net, but it is required for anyone who needs to run the tests.
>
> -----Original Message-----
> From: Shad Storhaug [mailto:shad@shadstorhaug.com]
> Sent: Saturday, April 1, 2017 5:31 AM
> To: dev@lucenenet.apache.org
> Subject: RE: API Work/Stabilization Update
>
> Wyatt,
>
> Great. Yea, actually after I sent my last email I was starting to wonder
> if Connie ever tested the build on a box without Visual Studio installed.
> We might need to determine what the prerequisites for the build are so we
> know what to put on the build server.
>
> I haven't yet had a chance to determine what tests are causing the build
> to crash (although, I know I can reliably reproduce it). I plan on getting
> that straightened out shortly. Worst case, we can just turn off verbosity
> on the build, but it might be helpful to leave it on if something goes
> wrong. I'll also look into the build dependencies and the script itself.
>
> I am just going through now and deleting all of the old 3.x source files
> that are now just causing noise in the project and resurrecting the tests
> for the support classes. But I am nearly finished. If you are available, I
> would like to try to get the beta2 release done over the weekend (at least
> to the point where it is on MyGet). We need to get the README and
> CONTRIBUTING files updated with the latest status (it would be helpful to
> sync the wiki page as well:
> https://cwiki.apache.org/confluence/display/LUCENENET/Current+Status - I
> don't think I have access for that).
>
> Several issues regarding the main readme need to be addressed:
>
> 1. There is no link to the wiki
> 2. There is no link to the issue tracker 3. It is not clear what the
> status is 4. There are no build instructions 5. There is no mention that we
> need help to finish 6. There is no link to the license 7. The documentation
> links are out of date 8. The top part of the repo already lists the files,
> do we really need to describe them again?
>
> Itamar mentioned he would be working on a new web site - I am not sure
> what the status of that is, but either way these issues are causing
> friction with anyone who is willing to help, but can't navigate through
> these hurdles. People who are used to working with GitHub, its issue
> tracker, and wiki don't find it natural to go looking somewhere else to
> these tools. For beta testing, we definitely need to make it crystal clear
> how to report bugs.
>
> Also, the "known issues" list is getting short enough so I can start
> adding them to the issue tracker.
>
> I have been using the downtime while running tests to fix up the
> documentation comments in Lucene.Net.Core, but it's still only about 50%
> done. I could use some help getting them fixed so they are at least visible
> in Visual Studio intellisense. See the latest status on #203. And then of
> course we can use them to generate new documents.
>
>
> Thanks,
> Shad Storhaug (NightOwl888)
>
>
> -----Original Message-----
> From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> Sent: Saturday, April 1, 2017 4:22 AM
> To: dev@lucenenet.apache.org
> Subject: Re: API Work/Stabilization Update
>
> Shad -- definitely makes sense.
>
> Json files are fine -- functionally this is a bit too fancy to use
> Teamcity's automagic nuget package generation so as long as we've got a
> file to edit we are fine.
>
> Myget -> nuget works for me but that doesn't solve the key problem. I
> don't have it, maybe Prescott or Itamar know where it is kept but I can't
> claim to have ever seen it. I joined this party after the last nuget push.
>
> It is a bit foggy but I think I ran into the nunit-console issue with the
> Build.ps1 script. Remember that with build servers the pre-requisites
> often need to be embedded in the project for things to work properly.
>
> Anyhow, let me know when you are in a good place with your branch to start
> slogging through getting the new build working. In the interests of full
> disclosure I'm working an event the last week and a half of April and will
> be completely out of pocket then. But I'm about otherwise.
>
> On Sat, Mar 25, 2017 at 10:04 AM Shad Storhaug <shad@shadstorhaug.com>
> wrote:
>
> > Wyatt,
> >
> > > We will probably want to build out .nuspec files to get all the
> > > nuget
> > stuff right for these projects -- I don't think the generation will
> > work for us to get things quite right.
> >
> > Connie has set us up to use .json files instead of .nuspec files to
> > generate the NuGet packages (the new way instead of the old way). The
> > build script Build.ps1 does it all (it even has help documentation),
> > but it is missing an option to override versioning. Ideally we would
> > be able to override the version that is in the .json file with an
> > environment variable (which you can pass from TeamCity), and be able
> > to override that on the CLI for local "one-off" builds.  See the build
> instructions on #191:
> > https://github.com/apache/lucenenet/pull/191
> >
> > > Regarding the nuget key -- that plan works for me, the trick is I
> > > don't
> > have the key to add to myget.
> >
> > I don't know what order the infrastructure was setup in on your end,
> > but my thought was that if someone had previously pushed from MyGet to
> > NuGet the key is probably already configured there. But yea, you would
> > need access to MyGet to confirm.
> >
> > > I would love to start beating on that a bit but the .net core
> > > version
> > seems to want NUnit 3.5+ which needs to be added to the project to run.
> >
> > I will take a look at your pull request, but I think this is a symptom
> > of trying to run using the older tooling. The Build.ps1 script already
> > has the ability to test, and all of the tooling is there to do it (I
> > think - maybe I should do a fresh clone to be sure). It does have some
> > prerequisites, though (see #191). It builds both the .NET Framework
> > and .NET Core versions and packages them into NuGet.
> >
> > Per #191: Hopefully Lucene.Net.sln can be removed in the future
> > because the .NET Core projects compile for .NET 4.5.1 already.
> >
> > So I think the aim is to eventually eliminate those .csproj files (and
> > for that matter .nuspec files) and use strictly .json files for
> > project configuration going forward.
> >
> >
> > Thanks,
> > Shad Storhaug (NightOwl888)
> >
> > -----Original Message-----
> > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > Sent: Saturday, March 25, 2017 5:00 AM
> > To: dev@lucenenet.apache.org
> > Subject: Re: API Work/Stabilization Update
> >
> > Shad -- the overall plan sounds good. We will probably want to build
> > out .nuspec files to get all the nuget stuff right for these projects
> > -- I don't think the generation will work for us to get things quite
> right.
> >
> > Regarding the nuget key -- that plan works for me, the trick is I
> > don't have the key to add to myget. Come to think of it I don't think
> > I have the proverbial keys to the myget page either but I think Martin
> > can help us out there.
> >
> > Buffers could be the issue on the tests -- I've long suspected that or
> > I/O causing the meltdown, I just haven't been able to reproduce. I
> > would love to start beating on that a bit but the .net core version
> > seems to want NUnit 3.5+ which needs to be added to the project to
> > run. If you get that added I can start beating on the test problems a
> bit more.
> >
> > Thanks for all your hard work putting this together, let me know how I
> > can help you get it out the proverbial door.
> >
> > On Fri, Mar 24, 2017 at 9:34 AM Shad Storhaug <shad@shadstorhaug.com>
> > wrote:
> >
> > > Wyatt,
> > >
> > > Thanks. Actually, I was thinking this should go in a few steps
> > > instead of
> > > one:
> > >
> > > 1. Merge #203.
> > > 2. Change the pre-release label to "beta2" and work out any issues
> > > to build/push to MyGet (might take a few tries) 3. Update the README
> > > and CONTRIBUTING pages 4. Push the package to NuGet
> > >
> > > I have always just used the control panel at MyGet to push upstream
> > > to NuGet, and it is capable of storing someone's key so the person
> > > who pushes it doesn't actually need it.
> > >
> > > As far as the tests burning down are concerned, I discovered that
> > > some of them write so much "verbose" data that they overflow NUnit's
> > > buffer and cause it to crash (sometimes this even causes Visual
> > > Studio to crash locally). I think I have found all of the tests in
> > > the Core that were causing this and hard-coded them to set verbose
> > > off (with the ability to manually override), but I noticed that
> > > there are still tests in Analysis.Common that can cause it to crash.
> > > I haven't investigated if there is a setting in NUnit to increase
> > > the buffer size, which might be a better fix, but I could probably
> > > track down the rest of the tests that are causing this before the
> merge.
> > >
> > > Thanks,
> > > Shad Storhaug (NightOwl888)
> > >
> > > -----Original Message-----
> > > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > > Sent: Friday, March 24, 2017 8:14 PM
> > > To: dev@lucenenet.apache.org
> > > Subject: Re: API Work/Stabilization Update
> > >
> > > I'm about and happy to update when you are ready.
> > >
> > > I think the build should start working for 203 if you add the
> > > nunit-console nuget package. At least work in the sense of the build
> > > will start. I'm still chasing those build time bugbears I haven't
> > > been able to slay yet.
> > >
> > > As for getting to nuget.org -- who has the key? I've never had
> > > access to it so I'm not sure we can update what is out there.
> > >
> > >
> > > On Fri, Mar 24, 2017 at 09:10 Shad Storhaug <shad@shadstorhaug.com>
> > wrote:
> > >
> > > > Wyatt, Prescott, Itamar, anybody? We've almost got all of the
> > > > tests passing on #203 and would like to merge to master and
> > > > release it with the new pre-release label "beta2".
> > > >
> > > > If there is nobody available to get the build running after
> > > > merging, could someone give me access to TeamCity to work on it?
> > > >
> > > > Thanks,
> > > > Shad Storhaug (NightOwl888)
> > > >
> > > > From: Shad Storhaug
> > > > Sent: Tuesday, March 21, 2017 5:25 AM
> > > > To: 'dev@lucenenet.apache.org'
> > > > Subject: API Work/Stabilization Update
> > > >
> > > > I am getting very close to getting #203 merged. I wouldn't go so
> > > > far as to say that the API is finished, but the most significant
> > > > of the breaking API changes are now behind us.
> > > >
> > > > BUILD/VERSIONING
> > > >
> > > > I just wanted to be sure there is someone available to help get
> > > > the build working after the merge. I think it would be appropriate
> > > > to change the pre-release label from "beta" to "beta2" (without
> > > > resetting the build number, since that is actually what NuGet uses).
> > > > This would be primarily because of a major breaking API change,
> > > > but also to indicate another advancement toward release.
> > > >
> > > > We should probably also get this onto NuGet as soon as possible to
> > > > (hopefully) make it easier to recruit help to stabilize and create
> > > > some integration packages for popular Microsoft frameworks.
> > > >
> > > > KNOWN ISSUES
> > > >
> > > >
> > > > 1.       The QueryParser.Flexible custom localized message
> > functionality
> > > > is currently not implemented for .NET core, so those tests are now
> > > failing.
> > > >
> > > > 2.       The implementation of Lucene.Net.Expressions currently reads
> > > data
> > > > from the configuration file. This is not how modern libraries are
> > > > supposed to be built - instead we want any configuration to be
> > > > pushed in from the application that uses Lucene.Net. Reading from
> > > > the configuration file directly means no opportunity to use
> > > > dependency injection. There is also a namespace
> > > > Support/Configuration that can and should be removed after the
> > > > implementation is refactored to be DI-friendly (see
> > > > http://blog.ploeh.dk/2014/05/19/di-friendly-framework/). I haven't
> > > > yet worked out how the implementation was done in .NET - in Java,
> > > > the defaults were read from an embedded resource file and could be
> > > > overridden by passing in a ClassLoader (similar to .NET's Assembly
> > > class) - if anyone has any information on how the "auto generated"
> > > C# code was generated, please share.
> > > >
> > > > 3.       The Collation functionality in Analysis.Common doesn't work
> > with
> > > > icu-dotnet, and has been excluded from compilation using the
> > > > constant FEATURE_COLLATION. I am now convinced after reading the
> > > > docs that it would be better to port the similar functionality
> > > > from Analysis.ICU because it was designed to work with icu4j and
> > > > is therefore more likely to work with icu-dotnet.
> > > >
> > > > 4.       The Highlighter PostingsHighlight and VectorHighlight
> > > > functionality relies on icu-dotnet, which doesn't have a close
> > > > match for the BreakIterator in the JRE, so there are likely some
> > > > big differences in the functionality. Several hacks were put in to
> > > > make the tests pass, but these are not likely to fix all of the
> > > > issues in the
> > > wild.
> > > >
> > > > 5.       There are several namespaces in Lucene.Net.Core and
> > > > Lucene.Net.Codecs that have broken documentation comments.
> > > >
> > > > 6.       There are some concurrency and performance issues (as
> pointed
> > > out
> > > > by Vincent Van Den Burghe):
> > > > http://git.net/ml/general/2017-02/msg00168.html
> > > >
> > > > 7.       We have around 2 dozen tests that fail during randomization
> > > > (averaging about 17 broken per run), and 8 tests that fail
> > > > all/most of the time.
> > > >
> > > > RESOLVED ISSUES (in addition to API refactoring)
> > > >
> > > >
> > > > 1.       Finished implementing the randomization of Codecs, Culture,
> > Time
> > > > Zone, and InfoStream in the TestFramework.
> > > >
> > > > 2.       Added factories for Codec, DocValuesFormat, and
> PostingsFormat
> > > so
> > > > custom implementations can be provided via dependency injection
> > > > instead of using the Java-ish NamedSPILoader class. The name must
> > > > now be provided by an attribute (or by class naming convention)
> > > > rather than via constructor, so it can be read without creating an
> > > > instance of
> > > the class.
> > > >
> > > > 3.       Fixed several of the codecs in Lucene.Net.Codecs that were
> > still
> > > > not functioning (and not being tested because of the unfinished
> > > > RandomCodec class and test mocks).
> > > >
> > > > 4.       Reviewed all catch blocks in Lucene.Net.Core to ensure the
> > right
> > > > type of exceptions are being caught and the right type re-thrown.
> > > >
> > > > 5.       Fixed culture-sensitive comparison and sort order issues
> when
> > > > using strings in Lucene.Net.Core and Lucene.Net.Codecs.
> > > >
> > > > 6.       Merged similar functionality in Support into the same class
> > and
> > > > deleted several unused Support classes.
> > > >
> > > > 7.       Made the API CLS compliant, so it now works with all .NET
> > > > languages.
> > > >
> > > >
> > > > Shad Storhaug (NightOwl888)
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message