lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shad Storhaug <s...@shadstorhaug.com>
Subject RE: API Work/Stabilization Update
Date Sat, 01 Apr 2017 01:03:19 GMT
Wyatt,

When the DEBUG compilation symbol is passed (which happens by default when the project is
set to Debug mode), VERBOSE is on. If you pass don't pass an empty string for DefineConstants
verbose will be turned off regardless of whether you actually run in Debug or Release mode.
 Of course, this only matters for the Lucene.Net.TestFramework project, since that is the
one that sets the VERBOSE setting.

Thanks,
Shad

-----Original Message-----
From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com] 
Sent: Saturday, April 1, 2017 7:52 AM
To: dev@lucenenet.apache.org
Subject: Re: API Work/Stabilization Update

I wanted to walk back to one thing you asked about the tests Shad:

"Worst case, we can just turn off verbosity on the build, but it might be helpful to leave
it on if something goes wrong"

I *think* I'm turning the verbosity off in the tests -- they should be running in release
mode. They would not even run in debug if I recall correctly. Let me know how the switch works
and I'll make sure I'm throwing it on the build server side.

On Fri, Mar 31, 2017 at 6:56 PM Shad Storhaug <shad@shadstorhaug.com> wrote:

> I don't necessarily need ownership access, but he did give me 
> ownership to the https://www.nuget.org/packages/Spatial4n.Core/ package.
> Alternatively, Itamar can enter his keys on 
> https://www.myget.org/feed/Security/spatial4n - he already has ownership.
> Once the keys are added there, I will be able to push.
>
> -----Original Message-----
> From: Prescott Nasser [mailto:geobmx540@hotmail.com]
> Sent: Saturday, April 1, 2017 5:49 AM
> To: dev@lucenenet.apache.org
> Subject: RE: API Work/Stabilization Update
>
> Access like ownership access? Paging Itamar..
>
>
>
> -----Original Message-----
> From: Shad Storhaug [mailto:shad@shadstorhaug.com]
> Sent: Friday, March 31, 2017 3:38 PM
> To: dev@lucenenet.apache.org
> Subject: RE: API Work/Stabilization Update
>
> BTW - the contrib NuGet package is now dead - the functionality has 
> been moved into new sub-projects just like in Lucene.
>
> But, that reminds me - I still don't have access to 
> https://www.nuget.org/packages/Spatial4n.Core.NTS/, so I have been 
> unable to update it to the same version that 
> https://www.nuget.org/packages/Spatial4n.Core/ is on. NTS is not 
> required by Lucene.Net, but it is required for anyone who needs to run the tests.
>
> -----Original Message-----
> From: Shad Storhaug [mailto:shad@shadstorhaug.com]
> Sent: Saturday, April 1, 2017 5:31 AM
> To: dev@lucenenet.apache.org
> Subject: RE: API Work/Stabilization Update
>
> Wyatt,
>
> Great. Yea, actually after I sent my last email I was starting to 
> wonder if Connie ever tested the build on a box without Visual Studio installed.
> We might need to determine what the prerequisites for the build are so 
> we know what to put on the build server.
>
> I haven't yet had a chance to determine what tests are causing the 
> build to crash (although, I know I can reliably reproduce it). I plan 
> on getting that straightened out shortly. Worst case, we can just turn 
> off verbosity on the build, but it might be helpful to leave it on if 
> something goes wrong. I'll also look into the build dependencies and the script itself.
>
> I am just going through now and deleting all of the old 3.x source 
> files that are now just causing noise in the project and resurrecting 
> the tests for the support classes. But I am nearly finished. If you 
> are available, I would like to try to get the beta2 release done over 
> the weekend (at least to the point where it is on MyGet). We need to 
> get the README and CONTRIBUTING files updated with the latest status 
> (it would be helpful to sync the wiki page as well:
> https://cwiki.apache.org/confluence/display/LUCENENET/Current+Status - 
> I don't think I have access for that).
>
> Several issues regarding the main readme need to be addressed:
>
> 1. There is no link to the wiki
> 2. There is no link to the issue tracker 3. It is not clear what the 
> status is 4. There are no build instructions 5. There is no mention 
> that we need help to finish 6. There is no link to the license 7. The 
> documentation links are out of date 8. The top part of the repo 
> already lists the files, do we really need to describe them again?
>
> Itamar mentioned he would be working on a new web site - I am not sure 
> what the status of that is, but either way these issues are causing 
> friction with anyone who is willing to help, but can't navigate 
> through these hurdles. People who are used to working with GitHub, its 
> issue tracker, and wiki don't find it natural to go looking somewhere 
> else to these tools. For beta testing, we definitely need to make it 
> crystal clear how to report bugs.
>
> Also, the "known issues" list is getting short enough so I can start 
> adding them to the issue tracker.
>
> I have been using the downtime while running tests to fix up the 
> documentation comments in Lucene.Net.Core, but it's still only about 
> 50% done. I could use some help getting them fixed so they are at 
> least visible in Visual Studio intellisense. See the latest status on 
> #203. And then of course we can use them to generate new documents.
>
>
> Thanks,
> Shad Storhaug (NightOwl888)
>
>
> -----Original Message-----
> From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> Sent: Saturday, April 1, 2017 4:22 AM
> To: dev@lucenenet.apache.org
> Subject: Re: API Work/Stabilization Update
>
> Shad -- definitely makes sense.
>
> Json files are fine -- functionally this is a bit too fancy to use 
> Teamcity's automagic nuget package generation so as long as we've got 
> a file to edit we are fine.
>
> Myget -> nuget works for me but that doesn't solve the key problem. I 
> don't have it, maybe Prescott or Itamar know where it is kept but I 
> can't claim to have ever seen it. I joined this party after the last nuget push.
>
> It is a bit foggy but I think I ran into the nunit-console issue with 
> the
> Build.ps1 script. Remember that with build servers the pre-requisites 
> often need to be embedded in the project for things to work properly.
>
> Anyhow, let me know when you are in a good place with your branch to 
> start slogging through getting the new build working. In the interests 
> of full disclosure I'm working an event the last week and a half of 
> April and will be completely out of pocket then. But I'm about otherwise.
>
> On Sat, Mar 25, 2017 at 10:04 AM Shad Storhaug <shad@shadstorhaug.com>
> wrote:
>
> > Wyatt,
> >
> > > We will probably want to build out .nuspec files to get all the 
> > > nuget
> > stuff right for these projects -- I don't think the generation will 
> > work for us to get things quite right.
> >
> > Connie has set us up to use .json files instead of .nuspec files to 
> > generate the NuGet packages (the new way instead of the old way). 
> > The build script Build.ps1 does it all (it even has help 
> > documentation), but it is missing an option to override versioning. 
> > Ideally we would be able to override the version that is in the 
> > .json file with an environment variable (which you can pass from 
> > TeamCity), and be able to override that on the CLI for local 
> > "one-off" builds.  See the build
> instructions on #191:
> > https://github.com/apache/lucenenet/pull/191
> >
> > > Regarding the nuget key -- that plan works for me, the trick is I 
> > > don't
> > have the key to add to myget.
> >
> > I don't know what order the infrastructure was setup in on your end, 
> > but my thought was that if someone had previously pushed from MyGet 
> > to NuGet the key is probably already configured there. But yea, you 
> > would need access to MyGet to confirm.
> >
> > > I would love to start beating on that a bit but the .net core 
> > > version
> > seems to want NUnit 3.5+ which needs to be added to the project to run.
> >
> > I will take a look at your pull request, but I think this is a 
> > symptom of trying to run using the older tooling. The Build.ps1 
> > script already has the ability to test, and all of the tooling is 
> > there to do it (I think - maybe I should do a fresh clone to be 
> > sure). It does have some prerequisites, though (see #191). It builds 
> > both the .NET Framework and .NET Core versions and packages them into NuGet.
> >
> > Per #191: Hopefully Lucene.Net.sln can be removed in the future 
> > because the .NET Core projects compile for .NET 4.5.1 already.
> >
> > So I think the aim is to eventually eliminate those .csproj files 
> > (and for that matter .nuspec files) and use strictly .json files for 
> > project configuration going forward.
> >
> >
> > Thanks,
> > Shad Storhaug (NightOwl888)
> >
> > -----Original Message-----
> > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > Sent: Saturday, March 25, 2017 5:00 AM
> > To: dev@lucenenet.apache.org
> > Subject: Re: API Work/Stabilization Update
> >
> > Shad -- the overall plan sounds good. We will probably want to build 
> > out .nuspec files to get all the nuget stuff right for these 
> > projects
> > -- I don't think the generation will work for us to get things quite
> right.
> >
> > Regarding the nuget key -- that plan works for me, the trick is I 
> > don't have the key to add to myget. Come to think of it I don't 
> > think I have the proverbial keys to the myget page either but I 
> > think Martin can help us out there.
> >
> > Buffers could be the issue on the tests -- I've long suspected that 
> > or I/O causing the meltdown, I just haven't been able to reproduce. 
> > I would love to start beating on that a bit but the .net core 
> > version seems to want NUnit 3.5+ which needs to be added to the 
> > project to run. If you get that added I can start beating on the 
> > test problems a
> bit more.
> >
> > Thanks for all your hard work putting this together, let me know how 
> > I can help you get it out the proverbial door.
> >
> > On Fri, Mar 24, 2017 at 9:34 AM Shad Storhaug 
> > <shad@shadstorhaug.com>
> > wrote:
> >
> > > Wyatt,
> > >
> > > Thanks. Actually, I was thinking this should go in a few steps 
> > > instead of
> > > one:
> > >
> > > 1. Merge #203.
> > > 2. Change the pre-release label to "beta2" and work out any issues 
> > > to build/push to MyGet (might take a few tries) 3. Update the 
> > > README and CONTRIBUTING pages 4. Push the package to NuGet
> > >
> > > I have always just used the control panel at MyGet to push 
> > > upstream to NuGet, and it is capable of storing someone's key so 
> > > the person who pushes it doesn't actually need it.
> > >
> > > As far as the tests burning down are concerned, I discovered that 
> > > some of them write so much "verbose" data that they overflow 
> > > NUnit's buffer and cause it to crash (sometimes this even causes 
> > > Visual Studio to crash locally). I think I have found all of the 
> > > tests in the Core that were causing this and hard-coded them to 
> > > set verbose off (with the ability to manually override), but I 
> > > noticed that there are still tests in Analysis.Common that can cause it to
crash.
> > > I haven't investigated if there is a setting in NUnit to increase 
> > > the buffer size, which might be a better fix, but I could probably 
> > > track down the rest of the tests that are causing this before the
> merge.
> > >
> > > Thanks,
> > > Shad Storhaug (NightOwl888)
> > >
> > > -----Original Message-----
> > > From: Wyatt Barnett [mailto:wyatt.barnett@gmail.com]
> > > Sent: Friday, March 24, 2017 8:14 PM
> > > To: dev@lucenenet.apache.org
> > > Subject: Re: API Work/Stabilization Update
> > >
> > > I'm about and happy to update when you are ready.
> > >
> > > I think the build should start working for 203 if you add the 
> > > nunit-console nuget package. At least work in the sense of the 
> > > build will start. I'm still chasing those build time bugbears I 
> > > haven't been able to slay yet.
> > >
> > > As for getting to nuget.org -- who has the key? I've never had 
> > > access to it so I'm not sure we can update what is out there.
> > >
> > >
> > > On Fri, Mar 24, 2017 at 09:10 Shad Storhaug 
> > > <shad@shadstorhaug.com>
> > wrote:
> > >
> > > > Wyatt, Prescott, Itamar, anybody? We've almost got all of the 
> > > > tests passing on #203 and would like to merge to master and 
> > > > release it with the new pre-release label "beta2".
> > > >
> > > > If there is nobody available to get the build running after 
> > > > merging, could someone give me access to TeamCity to work on it?
> > > >
> > > > Thanks,
> > > > Shad Storhaug (NightOwl888)
> > > >
> > > > From: Shad Storhaug
> > > > Sent: Tuesday, March 21, 2017 5:25 AM
> > > > To: 'dev@lucenenet.apache.org'
> > > > Subject: API Work/Stabilization Update
> > > >
> > > > I am getting very close to getting #203 merged. I wouldn't go so 
> > > > far as to say that the API is finished, but the most significant 
> > > > of the breaking API changes are now behind us.
> > > >
> > > > BUILD/VERSIONING
> > > >
> > > > I just wanted to be sure there is someone available to help get 
> > > > the build working after the merge. I think it would be 
> > > > appropriate to change the pre-release label from "beta" to 
> > > > "beta2" (without resetting the build number, since that is actually what
NuGet uses).
> > > > This would be primarily because of a major breaking API change, 
> > > > but also to indicate another advancement toward release.
> > > >
> > > > We should probably also get this onto NuGet as soon as possible 
> > > > to
> > > > (hopefully) make it easier to recruit help to stabilize and 
> > > > create some integration packages for popular Microsoft frameworks.
> > > >
> > > > KNOWN ISSUES
> > > >
> > > >
> > > > 1.       The QueryParser.Flexible custom localized message
> > functionality
> > > > is currently not implemented for .NET core, so those tests are 
> > > > now
> > > failing.
> > > >
> > > > 2.       The implementation of Lucene.Net.Expressions currently reads
> > > data
> > > > from the configuration file. This is not how modern libraries 
> > > > are supposed to be built - instead we want any configuration to 
> > > > be pushed in from the application that uses Lucene.Net. Reading 
> > > > from the configuration file directly means no opportunity to use 
> > > > dependency injection. There is also a namespace 
> > > > Support/Configuration that can and should be removed after the 
> > > > implementation is refactored to be DI-friendly (see 
> > > > http://blog.ploeh.dk/2014/05/19/di-friendly-framework/). I 
> > > > haven't yet worked out how the implementation was done in .NET - 
> > > > in Java, the defaults were read from an embedded resource file 
> > > > and could be overridden by passing in a ClassLoader (similar to 
> > > > .NET's Assembly
> > > class) - if anyone has any information on how the "auto generated"
> > > C# code was generated, please share.
> > > >
> > > > 3.       The Collation functionality in Analysis.Common doesn't work
> > with
> > > > icu-dotnet, and has been excluded from compilation using the 
> > > > constant FEATURE_COLLATION. I am now convinced after reading the 
> > > > docs that it would be better to port the similar functionality 
> > > > from Analysis.ICU because it was designed to work with icu4j and 
> > > > is therefore more likely to work with icu-dotnet.
> > > >
> > > > 4.       The Highlighter PostingsHighlight and VectorHighlight
> > > > functionality relies on icu-dotnet, which doesn't have a close 
> > > > match for the BreakIterator in the JRE, so there are likely some 
> > > > big differences in the functionality. Several hacks were put in 
> > > > to make the tests pass, but these are not likely to fix all of 
> > > > the issues in the
> > > wild.
> > > >
> > > > 5.       There are several namespaces in Lucene.Net.Core and
> > > > Lucene.Net.Codecs that have broken documentation comments.
> > > >
> > > > 6.       There are some concurrency and performance issues (as
> pointed
> > > out
> > > > by Vincent Van Den Burghe):
> > > > http://git.net/ml/general/2017-02/msg00168.html
> > > >
> > > > 7.       We have around 2 dozen tests that fail during randomization
> > > > (averaging about 17 broken per run), and 8 tests that fail 
> > > > all/most of the time.
> > > >
> > > > RESOLVED ISSUES (in addition to API refactoring)
> > > >
> > > >
> > > > 1.       Finished implementing the randomization of Codecs, Culture,
> > Time
> > > > Zone, and InfoStream in the TestFramework.
> > > >
> > > > 2.       Added factories for Codec, DocValuesFormat, and
> PostingsFormat
> > > so
> > > > custom implementations can be provided via dependency injection 
> > > > instead of using the Java-ish NamedSPILoader class. The name 
> > > > must now be provided by an attribute (or by class naming 
> > > > convention) rather than via constructor, so it can be read 
> > > > without creating an instance of
> > > the class.
> > > >
> > > > 3.       Fixed several of the codecs in Lucene.Net.Codecs that were
> > still
> > > > not functioning (and not being tested because of the unfinished 
> > > > RandomCodec class and test mocks).
> > > >
> > > > 4.       Reviewed all catch blocks in Lucene.Net.Core to ensure the
> > right
> > > > type of exceptions are being caught and the right type re-thrown.
> > > >
> > > > 5.       Fixed culture-sensitive comparison and sort order issues
> when
> > > > using strings in Lucene.Net.Core and Lucene.Net.Codecs.
> > > >
> > > > 6.       Merged similar functionality in Support into the same class
> > and
> > > > deleted several unused Support classes.
> > > >
> > > > 7.       Made the API CLS compliant, so it now works with all .NET
> > > > languages.
> > > >
> > > >
> > > > Shad Storhaug (NightOwl888)
> > > >
> > >
> >
>
Mime
View raw message