lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wyatt Barnett <wyatt.barn...@gmail.com>
Subject Re: API Work/Stabilization Update
Date Fri, 31 Mar 2017 21:22:08 GMT
Shad -- definitely makes sense.

Json files are fine -- functionally this is a bit too fancy to use
Teamcity's automagic nuget package generation so as long as we've got a
file to edit we are fine.

Myget -> nuget works for me but that doesn't solve the key problem. I don't
have it, maybe Prescott or Itamar know where it is kept but I can't claim
to have ever seen it. I joined this party after the last nuget push.

It is a bit foggy but I think I ran into the nunit-console issue with the
Build.ps1 script. Remember that with build servers the pre-requisites often
need to be embedded in the project for things to work properly.

Anyhow, let me know when you are in a good place with your branch to start
slogging through getting the new build working. In the interests of full
disclosure I'm working an event the last week and a half of April and will
be completely out of pocket then. But I'm about otherwise.

On Sat, Mar 25, 2017 at 10:04 AM Shad Storhaug <shad@shadstorhaug.com>
wrote:

> Wyatt,
>
> > We will probably want to build out .nuspec files to get all the nuget
> stuff right for these projects -- I don't think the generation will work
> for us to get things quite right.
>
> Connie has set us up to use .json files instead of .nuspec files to
> generate the NuGet packages (the new way instead of the old way). The build
> script Build.ps1 does it all (it even has help documentation), but it is
> missing an option to override versioning. Ideally we would be able to
> override the version that is in the .json file with an environment variable
> (which you can pass from TeamCity), and be able to override that on the CLI
> for local "one-off" builds.  See the build instructions on #191:
> https://github.com/apache/lucenenet/pull/191
>
> > Regarding the nuget key -- that plan works for me, the trick is I don't
> have the key to add to myget.
>
> I don't know what order the infrastructure was setup in on your end, but
> my thought was that if someone had previously pushed from MyGet to NuGet
> the key is probably already configured there. But yea, you would need
> access to MyGet to confirm.
>
> > I would love to start beating on that a bit but the .net core version
> seems to want NUnit 3.5+ which needs to be added to the project to run.
>
> I will take a look at your pull request, but I think this is a symptom of
> trying to run using the older tooling. The Build.ps1 script already has the
> ability to test, and all of the tooling is there to do it (I think - maybe
> I should do a fresh clone to be sure). It does have some prerequisites,
> though (see #191). It builds both the .NET Framework and .NET Core versions
> and packages them into NuGet.
>
> Per #191: Hopefully Lucene.Net.sln can be removed in the future because
> the .NET Core projects compile for .NET 4.5.1 already.
>
> So I think the aim is to eventually eliminate those .csproj files (and for
> that matter .nuspec files) and use strictly .json files for project
> configuration going forward.
>
>
> Thanks,
> Shad Storhaug (NightOwl888)
>
> -----Original Message-----
> From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> Sent: Saturday, March 25, 2017 5:00 AM
> To: dev@lucenenet.apache.org
> Subject: Re: API Work/Stabilization Update
>
> Shad -- the overall plan sounds good. We will probably want to build out
> .nuspec files to get all the nuget stuff right for these projects -- I
> don't think the generation will work for us to get things quite right.
>
> Regarding the nuget key -- that plan works for me, the trick is I don't
> have the key to add to myget. Come to think of it I don't think I have the
> proverbial keys to the myget page either but I think Martin can help us out
> there.
>
> Buffers could be the issue on the tests -- I've long suspected that or I/O
> causing the meltdown, I just haven't been able to reproduce. I would love
> to start beating on that a bit but the .net core version seems to want
> NUnit 3.5+ which needs to be added to the project to run. If you get that
> added I can start beating on the test problems a bit more.
>
> Thanks for all your hard work putting this together, let me know how I can
> help you get it out the proverbial door.
>
> On Fri, Mar 24, 2017 at 9:34 AM Shad Storhaug <shad@shadstorhaug.com>
> wrote:
>
> > Wyatt,
> >
> > Thanks. Actually, I was thinking this should go in a few steps instead
> > of
> > one:
> >
> > 1. Merge #203.
> > 2. Change the pre-release label to "beta2" and work out any issues to
> > build/push to MyGet (might take a few tries) 3. Update the README and
> > CONTRIBUTING pages 4. Push the package to NuGet
> >
> > I have always just used the control panel at MyGet to push upstream to
> > NuGet, and it is capable of storing someone's key so the person who
> > pushes it doesn't actually need it.
> >
> > As far as the tests burning down are concerned, I discovered that some
> > of them write so much "verbose" data that they overflow NUnit's buffer
> > and cause it to crash (sometimes this even causes Visual Studio to
> > crash locally). I think I have found all of the tests in the Core that
> > were causing this and hard-coded them to set verbose off (with the
> > ability to manually override), but I noticed that there are still
> > tests in Analysis.Common that can cause it to crash. I haven't
> > investigated if there is a setting in NUnit to increase the buffer
> > size, which might be a better fix, but I could probably track down the
> > rest of the tests that are causing this before the merge.
> >
> > Thanks,
> > Shad Storhaug (NightOwl888)
> >
> > -----Original Message-----
> > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > Sent: Friday, March 24, 2017 8:14 PM
> > To: dev@lucenenet.apache.org
> > Subject: Re: API Work/Stabilization Update
> >
> > I'm about and happy to update when you are ready.
> >
> > I think the build should start working for 203 if you add the
> > nunit-console nuget package. At least work in the sense of the build
> > will start. I'm still chasing those build time bugbears I haven't been
> > able to slay yet.
> >
> > As for getting to nuget.org -- who has the key? I've never had access
> > to it so I'm not sure we can update what is out there.
> >
> >
> > On Fri, Mar 24, 2017 at 09:10 Shad Storhaug <shad@shadstorhaug.com>
> wrote:
> >
> > > Wyatt, Prescott, Itamar, anybody? We've almost got all of the tests
> > > passing on #203 and would like to merge to master and release it
> > > with the new pre-release label "beta2".
> > >
> > > If there is nobody available to get the build running after merging,
> > > could someone give me access to TeamCity to work on it?
> > >
> > > Thanks,
> > > Shad Storhaug (NightOwl888)
> > >
> > > From: Shad Storhaug
> > > Sent: Tuesday, March 21, 2017 5:25 AM
> > > To: 'dev@lucenenet.apache.org'
> > > Subject: API Work/Stabilization Update
> > >
> > > I am getting very close to getting #203 merged. I wouldn't go so far
> > > as to say that the API is finished, but the most significant of the
> > > breaking API changes are now behind us.
> > >
> > > BUILD/VERSIONING
> > >
> > > I just wanted to be sure there is someone available to help get the
> > > build working after the merge. I think it would be appropriate to
> > > change the pre-release label from "beta" to "beta2" (without
> > > resetting the build number, since that is actually what NuGet uses).
> > > This would be primarily because of a major breaking API change, but
> > > also to indicate another advancement toward release.
> > >
> > > We should probably also get this onto NuGet as soon as possible to
> > > (hopefully) make it easier to recruit help to stabilize and create
> > > some integration packages for popular Microsoft frameworks.
> > >
> > > KNOWN ISSUES
> > >
> > >
> > > 1.       The QueryParser.Flexible custom localized message
> functionality
> > > is currently not implemented for .NET core, so those tests are now
> > failing.
> > >
> > > 2.       The implementation of Lucene.Net.Expressions currently reads
> > data
> > > from the configuration file. This is not how modern libraries are
> > > supposed to be built - instead we want any configuration to be
> > > pushed in from the application that uses Lucene.Net. Reading from
> > > the configuration file directly means no opportunity to use
> > > dependency injection. There is also a namespace
> > > Support/Configuration that can and should be removed after the
> > > implementation is refactored to be DI-friendly (see
> > > http://blog.ploeh.dk/2014/05/19/di-friendly-framework/). I haven't
> > > yet worked out how the implementation was done in .NET - in Java,
> > > the defaults were read from an embedded resource file and could be
> > > overridden by passing in a ClassLoader (similar to .NET's Assembly
> > class) - if anyone has any information on how the "auto generated" C#
> > code was generated, please share.
> > >
> > > 3.       The Collation functionality in Analysis.Common doesn't work
> with
> > > icu-dotnet, and has been excluded from compilation using the
> > > constant FEATURE_COLLATION. I am now convinced after reading the
> > > docs that it would be better to port the similar functionality from
> > > Analysis.ICU because it was designed to work with icu4j and is
> > > therefore more likely to work with icu-dotnet.
> > >
> > > 4.       The Highlighter PostingsHighlight and VectorHighlight
> > > functionality relies on icu-dotnet, which doesn't have a close match
> > > for the BreakIterator in the JRE, so there are likely some big
> > > differences in the functionality. Several hacks were put in to make
> > > the tests pass, but these are not likely to fix all of the issues in
> > > the
> > wild.
> > >
> > > 5.       There are several namespaces in Lucene.Net.Core and
> > > Lucene.Net.Codecs that have broken documentation comments.
> > >
> > > 6.       There are some concurrency and performance issues (as pointed
> > out
> > > by Vincent Van Den Burghe):
> > > http://git.net/ml/general/2017-02/msg00168.html
> > >
> > > 7.       We have around 2 dozen tests that fail during randomization
> > > (averaging about 17 broken per run), and 8 tests that fail all/most
> > > of the time.
> > >
> > > RESOLVED ISSUES (in addition to API refactoring)
> > >
> > >
> > > 1.       Finished implementing the randomization of Codecs, Culture,
> Time
> > > Zone, and InfoStream in the TestFramework.
> > >
> > > 2.       Added factories for Codec, DocValuesFormat, and PostingsFormat
> > so
> > > custom implementations can be provided via dependency injection
> > > instead of using the Java-ish NamedSPILoader class. The name must
> > > now be provided by an attribute (or by class naming convention)
> > > rather than via constructor, so it can be read without creating an
> > > instance of
> > the class.
> > >
> > > 3.       Fixed several of the codecs in Lucene.Net.Codecs that were
> still
> > > not functioning (and not being tested because of the unfinished
> > > RandomCodec class and test mocks).
> > >
> > > 4.       Reviewed all catch blocks in Lucene.Net.Core to ensure the
> right
> > > type of exceptions are being caught and the right type re-thrown.
> > >
> > > 5.       Fixed culture-sensitive comparison and sort order issues when
> > > using strings in Lucene.Net.Core and Lucene.Net.Codecs.
> > >
> > > 6.       Merged similar functionality in Support into the same class
> and
> > > deleted several unused Support classes.
> > >
> > > 7.       Made the API CLS compliant, so it now works with all .NET
> > > languages.
> > >
> > >
> > > Shad Storhaug (NightOwl888)
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message