lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prescott Nasser <geobmx...@hotmail.com>
Subject RE: API Work/Stabilization Update
Date Fri, 31 Mar 2017 21:34:48 GMT
In the past I think we agreed PMC members could be added as an owner if they wanted. If we
wanted something past that we should probably talk about it.


-----Original Message-----
From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com] 
Sent: Friday, March 31, 2017 2:31 PM
To: dev@lucenenet.apache.org
Subject: Re: API Work/Stabilization Update

Thanks -- got it and accepted.

We will probably need to add a few more, is there a "right" way to do that so the ownership
stays clean?

On Fri, Mar 31, 2017 at 5:30 PM Prescott Nasser <geobmx540@hotmail.com>
wrote:

> Invited you to the 4 Lucene.Net packages we have up there
>
> -----Original Message-----
> From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> Sent: Friday, March 31, 2017 2:28 PM
> To: dev@lucenenet.apache.org
> Subject: Re: API Work/Stabilization Update
>
> Thanks Prescott -- that clears up that mystery. Username is wwb, email 
> is wyatt.barnett@gmail.com if it matters.
>
> On Fri, Mar 31, 2017 at 5:25 PM Prescott Nasser 
> <geobmx540@hotmail.com>
> wrote:
>
> > All PMC members should have ownership of the nuget packages - if you 
> > have a username I can add you as ownership... or I can just get you 
> > the keys
> >
> > -----Original Message-----
> > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > Sent: Friday, March 31, 2017 2:22 PM
> > To: dev@lucenenet.apache.org
> > Subject: Re: API Work/Stabilization Update
> >
> > Shad -- definitely makes sense.
> >
> > Json files are fine -- functionally this is a bit too fancy to use 
> > Teamcity's automagic nuget package generation so as long as we've 
> > got a file to edit we are fine.
> >
> > Myget -> nuget works for me but that doesn't solve the key problem. 
> > I don't have it, maybe Prescott or Itamar know where it is kept but 
> > I can't claim to have ever seen it. I joined this party after the 
> > last
> nuget push.
> >
> > It is a bit foggy but I think I ran into the nunit-console issue 
> > with the
> > Build.ps1 script. Remember that with build servers the 
> > pre-requisites often need to be embedded in the project for things to work properly.
> >
> > Anyhow, let me know when you are in a good place with your branch to 
> > start slogging through getting the new build working. In the 
> > interests of full disclosure I'm working an event the last week and 
> > a half of April and will be completely out of pocket then. But I'm about otherwise.
> >
> > On Sat, Mar 25, 2017 at 10:04 AM Shad Storhaug 
> > <shad@shadstorhaug.com>
> > wrote:
> >
> > > Wyatt,
> > >
> > > > We will probably want to build out .nuspec files to get all the 
> > > > nuget
> > > stuff right for these projects -- I don't think the generation 
> > > will work for us to get things quite right.
> > >
> > > Connie has set us up to use .json files instead of .nuspec files 
> > > to generate the NuGet packages (the new way instead of the old way).
> > > The build script Build.ps1 does it all (it even has help 
> > > documentation), but it is missing an option to override versioning.
> > > Ideally we would be able to override the version that is in the 
> > > .json file with an environment variable (which you can pass from 
> > > TeamCity), and be able to override that on the CLI for local 
> > > "one-off" builds.  See the build
> > instructions on #191:
> > > https://github.com/apache/lucenenet/pull/191
> > >
> > > > Regarding the nuget key -- that plan works for me, the trick is 
> > > > I don't
> > > have the key to add to myget.
> > >
> > > I don't know what order the infrastructure was setup in on your 
> > > end, but my thought was that if someone had previously pushed from 
> > > MyGet to NuGet the key is probably already configured there. But 
> > > yea, you would need access to MyGet to confirm.
> > >
> > > > I would love to start beating on that a bit but the .net core 
> > > > version
> > > seems to want NUnit 3.5+ which needs to be added to the project to run.
> > >
> > > I will take a look at your pull request, but I think this is a 
> > > symptom of trying to run using the older tooling. The Build.ps1 
> > > script already has the ability to test, and all of the tooling is 
> > > there to do it (I think - maybe I should do a fresh clone to be 
> > > sure). It does have some prerequisites, though (see #191). It 
> > > builds both the .NET Framework and .NET Core versions and packages 
> > > them into
> NuGet.
> > >
> > > Per #191: Hopefully Lucene.Net.sln can be removed in the future 
> > > because the .NET Core projects compile for .NET 4.5.1 already.
> > >
> > > So I think the aim is to eventually eliminate those .csproj files 
> > > (and for that matter .nuspec files) and use strictly .json files 
> > > for project configuration going forward.
> > >
> > >
> > > Thanks,
> > > Shad Storhaug (NightOwl888)
> > >
> > > -----Original Message-----
> > > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > > Sent: Saturday, March 25, 2017 5:00 AM
> > > To: dev@lucenenet.apache.org
> > > Subject: Re: API Work/Stabilization Update
> > >
> > > Shad -- the overall plan sounds good. We will probably want to 
> > > build out .nuspec files to get all the nuget stuff right for these 
> > > projects
> > > -- I don't think the generation will work for us to get things 
> > > quite
> > right.
> > >
> > > Regarding the nuget key -- that plan works for me, the trick is I 
> > > don't have the key to add to myget. Come to think of it I don't 
> > > think I have the proverbial keys to the myget page either but I 
> > > think Martin can help us out there.
> > >
> > > Buffers could be the issue on the tests -- I've long suspected 
> > > that or I/O causing the meltdown, I just haven't been able to reproduce.
> > > I would love to start beating on that a bit but the .net core 
> > > version seems to want NUnit 3.5+ which needs to be added to the 
> > > project to run. If you get that added I can start beating on the 
> > > test problems a
> > bit more.
> > >
> > > Thanks for all your hard work putting this together, let me know 
> > > how I can help you get it out the proverbial door.
> > >
> > > On Fri, Mar 24, 2017 at 9:34 AM Shad Storhaug 
> > > <shad@shadstorhaug.com>
> > > wrote:
> > >
> > > > Wyatt,
> > > >
> > > > Thanks. Actually, I was thinking this should go in a few steps 
> > > > instead of
> > > > one:
> > > >
> > > > 1. Merge #203.
> > > > 2. Change the pre-release label to "beta2" and work out any 
> > > > issues to build/push to MyGet (might take a few tries) 3. Update 
> > > > the README and CONTRIBUTING pages 4. Push the package to NuGet
> > > >
> > > > I have always just used the control panel at MyGet to push 
> > > > upstream to NuGet, and it is capable of storing someone's key so 
> > > > the person who pushes it doesn't actually need it.
> > > >
> > > > As far as the tests burning down are concerned, I discovered 
> > > > that some of them write so much "verbose" data that they 
> > > > overflow NUnit's buffer and cause it to crash (sometimes this 
> > > > even causes Visual Studio to crash locally). I think I have 
> > > > found all of the tests in the Core that were causing this and 
> > > > hard-coded them to set verbose off (with the ability to manually 
> > > > override), but I noticed that there are still tests in 
> > > > Analysis.Common that can cause
> it to crash.
> > > > I haven't investigated if there is a setting in NUnit to 
> > > > increase the buffer size, which might be a better fix, but I 
> > > > could probably track down the rest of the tests that are causing 
> > > > this before the
> > merge.
> > > >
> > > > Thanks,
> > > > Shad Storhaug (NightOwl888)
> > > >
> > > > -----Original Message-----
> > > > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > > > Sent: Friday, March 24, 2017 8:14 PM
> > > > To: dev@lucenenet.apache.org
> > > > Subject: Re: API Work/Stabilization Update
> > > >
> > > > I'm about and happy to update when you are ready.
> > > >
> > > > I think the build should start working for 203 if you add the 
> > > > nunit-console nuget package. At least work in the sense of the 
> > > > build will start. I'm still chasing those build time bugbears I 
> > > > haven't been able to slay yet.
> > > >
> > > > As for getting to nuget.org -- who has the key? I've never had 
> > > > access to it so I'm not sure we can update what is out there.
> > > >
> > > >
> > > > On Fri, Mar 24, 2017 at 09:10 Shad Storhaug 
> > > > <shad@shadstorhaug.com>
> > > wrote:
> > > >
> > > > > Wyatt, Prescott, Itamar, anybody? We've almost got all of the 
> > > > > tests passing on #203 and would like to merge to master and 
> > > > > release it with the new pre-release label "beta2".
> > > > >
> > > > > If there is nobody available to get the build running after 
> > > > > merging, could someone give me access to TeamCity to work on it?
> > > > >
> > > > > Thanks,
> > > > > Shad Storhaug (NightOwl888)
> > > > >
> > > > > From: Shad Storhaug
> > > > > Sent: Tuesday, March 21, 2017 5:25 AM
> > > > > To: 'dev@lucenenet.apache.org'
> > > > > Subject: API Work/Stabilization Update
> > > > >
> > > > > I am getting very close to getting #203 merged. I wouldn't go 
> > > > > so far as to say that the API is finished, but the most 
> > > > > significant of the breaking API changes are now behind us.
> > > > >
> > > > > BUILD/VERSIONING
> > > > >
> > > > > I just wanted to be sure there is someone available to help 
> > > > > get the build working after the merge. I think it would be 
> > > > > appropriate to change the pre-release label from "beta" to 
> > > > > "beta2" (without resetting the build number, since that is
> actually what NuGet uses).
> > > > > This would be primarily because of a major breaking API 
> > > > > change, but also to indicate another advancement toward release.
> > > > >
> > > > > We should probably also get this onto NuGet as soon as 
> > > > > possible to
> > > > > (hopefully) make it easier to recruit help to stabilize and 
> > > > > create some integration packages for popular Microsoft frameworks.
> > > > >
> > > > > KNOWN ISSUES
> > > > >
> > > > >
> > > > > 1.       The QueryParser.Flexible custom localized message
> > > functionality
> > > > > is currently not implemented for .NET core, so those tests are 
> > > > > now
> > > > failing.
> > > > >
> > > > > 2.       The implementation of Lucene.Net.Expressions currently
> reads
> > > > data
> > > > > from the configuration file. This is not how modern libraries 
> > > > > are supposed to be built - instead we want any configuration 
> > > > > to be pushed in from the application that uses Lucene.Net. 
> > > > > Reading from the configuration file directly means no 
> > > > > opportunity to use dependency injection. There is also a 
> > > > > namespace Support/Configuration that can and should be removed 
> > > > > after the implementation is refactored to be DI-friendly (see 
> > > > > http://blog.ploeh.dk/2014/05/19/di-friendly-framework/). I 
> > > > > haven't yet worked out how the implementation was done in .NET 
> > > > > - in Java, the defaults were read from an embedded resource 
> > > > > file and could be overridden by passing in a ClassLoader 
> > > > > (similar to .NET's Assembly
> > > > class) - if anyone has any information on how the "auto generated"
> > > > C# code was generated, please share.
> > > > >
> > > > > 3.       The Collation functionality in Analysis.Common doesn't
> work
> > > with
> > > > > icu-dotnet, and has been excluded from compilation using the 
> > > > > constant FEATURE_COLLATION. I am now convinced after reading 
> > > > > the docs that it would be better to port the similar 
> > > > > functionality from Analysis.ICU because it was designed to 
> > > > > work with icu4j and is therefore more likely to work with icu-dotnet.
> > > > >
> > > > > 4.       The Highlighter PostingsHighlight and VectorHighlight
> > > > > functionality relies on icu-dotnet, which doesn't have a close 
> > > > > match for the BreakIterator in the JRE, so there are likely 
> > > > > some big differences in the functionality. Several hacks were 
> > > > > put in to make the tests pass, but these are not likely to fix 
> > > > > all of the issues in the
> > > > wild.
> > > > >
> > > > > 5.       There are several namespaces in Lucene.Net.Core and
> > > > > Lucene.Net.Codecs that have broken documentation comments.
> > > > >
> > > > > 6.       There are some concurrency and performance issues (as
> > pointed
> > > > out
> > > > > by Vincent Van Den Burghe):
> > > > > http://git.net/ml/general/2017-02/msg00168.html
> > > > >
> > > > > 7.       We have around 2 dozen tests that fail during
> randomization
> > > > > (averaging about 17 broken per run), and 8 tests that fail 
> > > > > all/most of the time.
> > > > >
> > > > > RESOLVED ISSUES (in addition to API refactoring)
> > > > >
> > > > >
> > > > > 1.       Finished implementing the randomization of Codecs,
> Culture,
> > > Time
> > > > > Zone, and InfoStream in the TestFramework.
> > > > >
> > > > > 2.       Added factories for Codec, DocValuesFormat, and
> > PostingsFormat
> > > > so
> > > > > custom implementations can be provided via dependency 
> > > > > injection instead of using the Java-ish NamedSPILoader class. 
> > > > > The name must now be provided by an attribute (or by class 
> > > > > naming
> > > > > convention) rather than via constructor, so it can be read 
> > > > > without creating an instance of
> > > > the class.
> > > > >
> > > > > 3.       Fixed several of the codecs in Lucene.Net.Codecs that were
> > > still
> > > > > not functioning (and not being tested because of the 
> > > > > unfinished RandomCodec class and test mocks).
> > > > >
> > > > > 4.       Reviewed all catch blocks in Lucene.Net.Core to ensure the
> > > right
> > > > > type of exceptions are being caught and the right type re-thrown.
> > > > >
> > > > > 5.       Fixed culture-sensitive comparison and sort order issues
> > when
> > > > > using strings in Lucene.Net.Core and Lucene.Net.Codecs.
> > > > >
> > > > > 6.       Merged similar functionality in Support into the same
> class
> > > and
> > > > > deleted several unused Support classes.
> > > > >
> > > > > 7.       Made the API CLS compliant, so it now works with all .NET
> > > > > languages.
> > > > >
> > > > >
> > > > > Shad Storhaug (NightOwl888)
> > > > >
> > > >
> > >
> >
>
Mime
View raw message