lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rory Plaire <codekai...@gmail.com>
Subject Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?
Date Fri, 01 Jul 2011 19:35:51 GMT
@Michael -

Should that list be in JIRA? It would be easier to manage, I think...

If yes, I'll happily do it.

-r

On Fri, Jul 1, 2011 at 8:04 AM, Michael Herndon <mherndon@wickedsoftware.net
> wrote:

> * need to document what the build script does.  whut grammerz?
>
> On Fri, Jul 1, 2011 at 10:52 AM, Michael Herndon <
> mherndon@wickedsoftware.net> wrote:
>
> > @Rory, @All,
> >
> > The only tickets I currently have for those is LUCENE-419, LUCENE-418
> >
> > 418, I should be able to push into the 2.9.4g branch tonight.    419 is a
> > long term goal and not as important as getting the tests fixed, of have
> the
> > tests broken down into what is actually a unit test, functional test,
> perf
> > or long running test. I can get into more why it needs to be done.
> >
> > I'll also need to make document the what build script currently does on
> the
> > wiki & and make a few notes about testing, like using the RAMDirectory,
> > etc.
> >
> > Things that need to get done or even be discussed.
> >  * There needs to be a running list of things to do/not to do with
> testing.
> > I don't know if this goes in a jira or do we keep a running list on the
> wiki
> > or site for people to pick up and  help with.
> >  * Tests need to run on mono and not Fail (there is a good deal of
> failing
> > tests on mono, mostly due to the temp directory have the C:\ in the
> path).
> >  * Assert.Throw<ExceptionType>() needs to be used instead of Try/Catch
> > Assert.Fail.  **
> >  * File & Path combines to the temp directory need helper methods,
> >      * e,g, having this in a hundred places is bad   new
> >
> System.IO.FileInfo(System.IO.Path.Combine(Support.AppSettings.Get("tempDir",
> > ""), "testIndex"));
> >  * We should still be testing deprecated methods, but we need to use
> #pragma
> > warning disable/enable 0618  for testing those. otherwise compiler
> warnings
> > are too numerous to be anywhere near helpful.
> >  * We should only be using deprecated methods in places where they are
> > being explicitly tested, other tests that need that functionality in
> order
> > to validate those tests should be re factored to use methods that are not
> > deprecated.
> >  * Identify code that could be abstracted into test utility classes.
> >  * Infrastructure Validation tests need to be made, anything that seems
> > like infrastructure.  e.g. does the temp directory exist, does the
> folders
> > that the tests use inside the temp directory exist, can we read/write to
> > those folders. (if a ton of tests fail due to the file system, we should
> be
> > able to point out that it was due to permissions or missing folders,
> files,
> > etc).
> >  * Identify what classes need an interface, abstract class or inherited
> in
> > order to create testing mocks. (once those classes are created, they
> should
> > be documented in the wiki).
> >
> >
> >
> > ** Asset.Throws needs to replace stuff like the following. We should also
> > be checking the messages for exceptions and make sure they make sense and
> > can help users fix isses if the exceptions are aimed at the library
> users.
> > try
> > {
> > d = DateTools.StringToDate("97"); // no date
> >  Assert.Fail();
> > }
> > catch (System.FormatException e)
> >  {
> > /* expected exception */
> > }
> >
> > On Thu, Jun 30, 2011 at 11:48 PM, Rory Plaire <codekaizen@gmail.com
> >wrote:
> >
> >> So, veering towards action - are there concrete tasks written up
> anywhere
> >> for the unit tests? If a poor schlep like me wanted to dig in and start
> to
> >> improve them, where would I get the understanding of what is good and
> what
> >> needs help?
> >>
> >> -r
> >>
> >> On Thu, Jun 30, 2011 at 3:29 PM, Digy <digydigy@gmail.com> wrote:
> >>
> >> > I can not say I like this approach, but till we find an automated
> >> way(with
> >> > good results), it seems to be the only way we can use.
> >> >
> >> > DIGY
> >> >
> >> > -----Original Message-----
> >> > From: Troy Howard [mailto:thoward37@gmail.com]
> >> > Sent: Friday, July 01, 2011 12:43 AM
> >> > To: lucene-net-dev@lucene.apache.org
> >> > Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port
> needed?
> >> >
> >> > Scott -
> >> >
> >> > The idea of the automated port is still worth doing. Perhaps it makes
> >> sense
> >> > for someone more passionate about the line-by-line idea to do that
> work?
> >> >
> >> > I would say, focus on what makes sense to you. Being productive,
> >> regardless
> >> > of the specific direction, is what will be most valuable. Once you
> >> start,
> >> > others will join and momentum will build. That is how these things
> work.
> >> >
> >> > I like DIGY's approach too, but the problem with it is that it is a
> >> > never-ending manual task. The theory behind the automated port is that
> >> it
> >> > may reduce the manual work. It is complicated, but once it's built and
> >> > works, it will save a lot of future development hours. If it's built
> in
> >> a
> >> > sufficiently general manner, it could be useful for other project like
> >> > Lucene.Net that want to automate a port from Java to C#.
> >> >
> >> > It might make sense for that to be a separate project from Lucene.Net
> >> > though.
> >> >
> >> > -T
> >> >
> >> >
> >> > On Thu, Jun 30, 2011 at 2:13 PM, Scott Lombard <
> lombardenator@gmail.com
> >> > >wrote:
> >> >
> >> > > Ok I think I asked the wrong question.  I am trying to figure out
> >> where
> >> > to
> >> > > put my time.  I was thinking about working on the automated porting
> >> > system,
> >> > > but when I saw the response to the .NET 4.0 discussions I started
to
> >> > > question if that is the right direction.  The community seemed to
be
> >> more
> >> > > interested in the .NET features.
> >> > >
> >> > > The complexity of the automated tool is going to become very high
> and
> >> > will
> >> > > probably end up with a line-for-line style port.  So I keep asking
> my
> >> > self
> >> > > is the automated tool worth it.  I don't think it is.
> >> > >
> >> > > I like the method has been Digy is using for porting the code.  So
I
> >> > guess
> >> > > for me the real question is Digy where did you see 2.9.4g going next
> >> and
> >> > > what do you need help on?
> >> > >
> >> > > Scott
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > > -----Original Message-----
> >> > > > From: Digy [mailto:digydigy@gmail.com]
> >> > > > Sent: Thursday, June 30, 2011 4:20 PM
> >> > > > To: lucene-net-dev@lucene.apache.org
> >> > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port
> >> > needed?
> >> > > >
> >> > > > Michael,
> >> > > > You interpret the report as "whoever commits code wins"? But
when
> I
> >> > look
> >> > > > at it, I see "a lof of talk, no work". .Net community is not
> >> interested
> >> > > in
> >> > > > contributing.
> >> > > > I really don't understand what hinders people to work on
> Lucene.Net.
> >> As
> >> > I
> >> > > > did for 2.9.4g, grab the code, do whatever you want on it and
> submit
> >> > > back.
> >> > > > If it doesn't fit to the project's direction it can still find
a
> >> place
> >> > in
> >> > > > contrib or in branch. All of the approaches can live side by
side
> >> > happily
> >> > > > in the Lucene.Net repository.
> >> > > >
> >> > > > Troy,
> >> > > > I also don't understand why do you wait for 2.9.4g? It is a
> *branch*
> >> > and
> >> > > > has nothing to do with the trunk. It need not be an offical
> release
> >> and
> >> > > > can live in branch as a PoC.
> >> > > >
> >> > > >
> >> > > > As a result, I got bored to listen to "this should be done that
> >> way".
> >> > > What
> >> > > > I want to see is "I did it that way, should we continue with
> this".
> >> > > >
> >> > > > DIGY
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > -----Original Message-----
> >> > > > From: Troy Howard [mailto:thoward37@gmail.com]
> >> > > > Sent: Thursday, June 30, 2011 10:47 PM
> >> > > > To: lucene-net-dev@lucene.apache.org
> >> > > > Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port
> >> > needed?
> >> > > >
> >> > > > Michael,
> >> > > >
> >> > > > I agree with everything you said. My point in saying "whoever
> >> commits
> >> > > code
> >> > > > wins" was to illustrate the reality of how and why the project
has
> >> the
> >> > > > current form.
> >> > > >
> >> > > > Building consensus is difficult. It is an essential first step
> >> before
> >> > we
> >> > > > can
> >> > > > do something like make a list of bit-sized pieces of work that
> >> others
> >> > can
> >> > > > work on.
> >> > > >
> >> > > > This is why my real message of "Let's find a way to accommodate
> >> both"
> >> > is
> >> > > > so
> >> > > > important. It allows us to build consensus, so that we can settle
> on
> >> a
> >> > > > direction and structure our work.
> >> > > >
> >> > > > Until we accomplish that, it really is "whoever commits code
> wins",
> >> and
> >> > > > that
> >> > > > is an unhealthy and unmaintainable way to operate.
> >> > > >
> >> > > > From a technical perspective, your statements about the unit
tests
> >> are
> >> > > > completely accurate. They really need a LOT of reworking. That's
> the
> >> > very
> >> > > > first step before making any significant changes. Part of the
> >> problem
> >> > is
> >> > > > that the tests themselves are not well written. The other part
is
> >> that
> >> > > the
> >> > > > Lucene object model was not designed for testability, and it
makes
> >> > > writing
> >> > > > good tests more difficult, and certain tests might not be
> possible.
> >> It
> >> > > > will
> >> > > > be difficult to write good unit tests without re-structuring.
The
> >> > biggest
> >> > > > issue is the use of abstract classes with base behaviour vs
> >> interfaces
> >> > or
> >> > > > fully abstracted classes. Makes mocking tough. This is the
> direction
> >> I
> >> > > was
> >> > > > going when I started the Lucere project. I'd like to start in
on
> >> that
> >> > > work
> >> > > > after the 2.9.4g release.
> >> > > >
> >> > > > Thanks,
> >> > > > Troy
> >> > > >
> >> > > >
> >> > > > On Thu, Jun 30, 2011 at 12:04 PM, Michael Herndon <
> >> > > > mherndon@wickedsoftware.net> wrote:
> >> > > >
> >> > > > > I'd say that is all the more reasons that we need to work
> smarter
> >> and
> >> > > > not
> >> > > > > harder. I'd also say thats a good reason to make sure we
build
> >> > > consensus
> >> > > > > rather than just saying whoever commits code wins.
> >> > > > >
> >> > > > > And its a damn good reason to focus on the effort of growing
the
> >> > number
> >> > > > of
> >> > > > > contributors and lowing the barrier to submitting patches,
> >> breaking
> >> > > > things
> >> > > > > down into pieces that people would feel confident to work
on
> >> without
> >> > > > > being overwhelmed by the complexity of Lucene.Net.
> >> > > > >
> >> > > > > There is a pretty big gap between Lucene 2.9.x to Lucene
4.0 and
> >> the
> >> > > > > internals and index formats are significantly different
> including
> >> > > nixing
> >> > > > > the
> >> > > > > current vint file format and using byte[] array slices for
Terms
> >> > > instead
> >> > > > of
> >> > > > > char[].
> >> > > > >
> >> > > > > So while porting 1 to 1 while require less knowledge or
thought,
> >> its
> >> > > > most
> >> > > > > likely going to require more hours of work. And Its definitely
> not
> >> > > going
> >> > > > to
> >> > > > > guarantee the stability of the code or that its great code.
> >> > > > >
> >> > > > > I'd have to say that its not as stable as most would believe
at
> >> the
> >> > > > moment.
> >> > > > >
> >> > > > > Most of the tests avoid anything that remotely looks like
it
> knows
> >> > > about
> >> > > > > the
> >> > > > > DRY principle and there is a static constructor in the core
test
> >> case
> >> > > > that
> >> > > > > throws an exception if it doesn't find an environment variable
> >> "TEMP"
> >> > > > which
> >> > > > > will fail 90% of the tests and nunit will be unable to give
you
> a
> >> > clear
> >> > > > > reason why.  Just to name a few issues I came across working
> >> towards
> >> > > > > getting
> >> > > > > Lucene.Net into CI.  I haven't even started really digging
in
> >> under
> >> > the
> >> > > > > covers of the code yet.
> >> > > > >
> >> > > > > So my suggestion is to chew on this a bit more and build
> >> consensus,
> >> > > > avoid
> >> > > > > fracturing people into sides.  Be open to reservations and
> >> concerns
> >> > > that
> >> > > > > others have and continue to address them.
> >> > > > >
> >> > > > > - Michael
> >> > > > >
> >> > > > >
> >> > > > > On Thu, Jun 30, 2011 at 2:10 PM, Digy <digydigy@gmail.com>
> wrote:
> >> > > > >
> >> > > > > > Although there are a lot of people using Lucene.Net,
this is
> our
> >> > > > > > contribution report for the past 5 years.
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/secure/ConfigureReport.jspa?atl_token=A5KQ-
> >> > > > 2Q
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> AV-T4JA-FDED|3204f7e696067a386144705075c074e991db2a2b|lin&versionId=-
> >> > > > 1&issue
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> Status=all&selectedProjectId=12310290&reportKey=com.sourcelabs.jira.plugin
> >> > > > .r
> >> > > > > > eport.contributions%3Acontributionreport&Next=Next
> >> > > > > >
> >> > > > > >
> >> > > > > > DIGY
> >> > > > > >
> >> > > > > > -----Original Message-----
> >> > > > > > From: Ayende Rahien [mailto:ayende@ayende.com]
> >> > > > > > Sent: Thursday, June 30, 2011 8:16 PM
> >> > > > > > To: Rory Plaire; lucene-net-dev@lucene.apache.org
> >> > > > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line
Jave
> port
> >> > > > needed?
> >> > > > > >
> >> > > > > > As someone from the nhibernate project
> >> > > > > > We stopped following hibernate a while ago, and haven't
> >> regretted
> >> > it
> >> > > > > > We have mire features, less bugs and better code base
> >> > > > > >
> >> > > > > > Sent from my Windows Phone From: Rory Plaire
> >> > > > > > Sent: Thursday, June 30, 2011 19:58
> >> > > > > > To: lucene-net-dev@lucene.apache.org
> >> > > > > > Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line
Jave
> port
> >> > > > needed?
> >> > > > > > I don't want to drag this out much longer, but I am
curious
> with
> >> > > > people
> >> > > > > who
> >> > > > > > hold the "line-by-line" sentiment - are you NHibernate
users?
> >> > > > > >
> >> > > > > > -r
> >> > > > > >
> >> > > > > > On Thu, Jun 30, 2011 at 2:39 AM, Noel Lysaght <
> >> > lysaghtn@hotmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Can I just plug in my bit and say I agree 100%
with what
> Moray
> >> > has
> >> > > > > > outlined
> >> > > > > > > below.
> >> > > > > > >
> >> > > > > > > If we move away from the line by line port then
over time
> >> we'll
> >> > > > loose
> >> > > > > out
> >> > > > > > > on the momentum that is Lucene and the improvements
that
> they
> >> > make.
> >> > > > > > > It is only if the Lucene.NET community has expertise
in
> >> search,
> >> >  a
> >> > > > >  deep
> >> > > > > > > knowledge of the project and the community can
guarantee
> that
> >> the
> >> > > > > > knowledge
> >> > > > > > > will survive members coming and going should such
a
> >> consideration
> >> > > be
> >> > > > > > give.
> >> > > > > > >
> >> > > > > > > When Lucene.NET has stood on it's feet for a number
of years
> >> > after
> >> > > > it
> >> > > > > has
> >> > > > > > > moved out of Apache incubation should consideration
be given
> >> to
> >> > > > > > abandoning
> >> > > > > > a
> >> > > > > > > line by line port.
> >> > > > > > > By all means extend and wrap the libraries in
.NET
> equivalents
> >> > and
> >> > > > .NET
> >> > > > > > > goodness like LINQ (we do this internally in our
company at
> >> the
> >> > > > > moment);
> >> > > > > > but
> >> > > > > > > leave the core of the project on a line by line
port.
> >> > > > > > >
> >> > > > > > > Just my tu-pence worth.
> >> > > > > > >
> >> > > > > > > Kind Regards
> >> > > > > > > Noel
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > -----Original Message----- From: Moray McConnachie
> >> > > > > > > Sent: Thursday, June 30, 2011 10:25 AM
> >> > > > > > >
> >> > > > > > > To: lucene-net-user@lucene.apache.**org<
> >> > > > > > lucene-net-user@lucene.apache.org>
> >> > > > > > > Cc:
> >> > > > > > lucene-net-dev@incubator.**apache.org<
> >> > > > > lucene-net-dev@incubator.apache.org>
> >> > > > > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line
Jave
> >> port
> >> > > > > needed?
> >> > > > > > >
> >> > > > > > > I don't think I'm as hard core on this as Neal,
but
> remember:
> >> the
> >> > > > > > > history of the Lucene.NET project is that all
the
> intellectual
> >> > > work,
> >> > > > > all
> >> > > > > > > the understanding of search, all the new features
come from
> >> the
> >> > > > Lucene
> >> > > > > > > Java folks. Theirs is an immensely respected project,
and I
> >> trust
> >> > > > them
> >> > > > > > > to add new features that will be well-tested and
> >> well-researched,
> >> > > > and
> >> > > > > to
> >> > > > > > > have a decent roadmap which I can trust they will
execute
> on.
> >> > > > > > >
> >> > > > > > > Now I know there's been an influx of capable developers
to
> >> > > > Lucene.NET
> >> > > > > > > who are ready, willing and (I'm going to assume)
able to add
> a
> >> > lot
> >> > > > more
> >> > > > > > > value in a generic .NET implementation as they
change it.
> But
> >> > it'll
> >> > > > > take
> >> > > > > > > a while before I trust a .NET dedicated framework
which is
> >> > > > > significantly
> >> > > > > > > diverged from Java in the way I do the line-by-line
version.
> >> And
> >> > at
> >> > > > > what
> >> > > > > > > stage is it not just not a line-by-line port,
but not a port
> >> at
> >> > > all?
> >> > > > > > >
> >> > > > > > > At the same time, I recognise that if this project
is going
> to
> >> > > > > continue,
> >> > > > > > > and attract good developers, it has to change
in this
> >> direction.
> >> > > > > > >
> >> > > > > > > So that said, I can see why a line-by-line port
might not be
> >> > > > > > > sustainable. And most people don't need it. But
most of us
> >> using
> >> > > > Lucene
> >> > > > > > > in production systems do need a system that we
can trust and
> >> rely
> >> > > > on.
> >> > > > > So
> >> > > > > > > let me chime in with someone else's plea, to keep
the
> general
> >> > > > structure
> >> > > > > > > close to Lucene, to keep the same general objects
and
> >> inheritance
> >> > > > > > > set-up, and to keep the same method names, even
if you add
> >> other
> >> > > > > methods
> >> > > > > > > and classes to provide additional functionality.
ABSOLUTELY
> >> the
> >> > > same
> >> > > > > > > file formats. End users benefit a lot from a high
degree of
> >> > > > similarity,
> >> > > > > > > with good documentation and help being available
from the
> Java
> >> > > > > > > community.
> >> > > > > > >
> >> > > > > > > Yours,
> >> > > > > > > Moray
> >> > > > > > > ------------------------------**-------
> >> > > > > > > Moray McConnachie
> >> > > > > > > Director of IT    +44 1865 261 600
> >> > > > > > > Oxford Analytica  http://www.oxan.com
> >> > > > > > >
> >> > > > > > > -----Original Message-----
> >> > > > > > > From: Granroth, Neal V.
> >> > > > > >
> >> > > > [mailto:neal.granroth@**thermofisher.com<
> >> > neal.granroth@thermofisher.com>
> >> > > > > > > ]
> >> > > > > > > Sent: 29 June 2011 20:47
> >> > > > > > > To: lucene-net-user@lucene.apache.**org<
> >> > > > > > lucene-net-user@lucene.apache.org>
> >> > > > > > > Cc:
> >> > > > > > lucene-net-dev@incubator.**apache.org<
> >> > > > > lucene-net-dev@incubator.apache.org>
> >> > > > > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line
Jave
> >> port
> >> > > > > needed?
> >> > > > > > >
> >> > > > > > > This is has been discussed many times.
> >> > > > > > > Lucene.NET is not valid, the code cannot be trusted,
if it
> is
> >> not
> >> > a
> >> > > > > > > line-by-line port.  It ceases to be Lucene.
> >> > > > > > >
> >> > > > > > > - Neal
> >> > > > > > >
> >> > > > > > > -----Original Message-----
> >> > > > > > > From: Scott Lombard
> >> > > > > > [mailto:lombardenator@gmail.**com<lombardenator@gmail.com>
> >> > > > > > > ]
> >> > > > > > > Sent: Wednesday, June 29, 2011 1:58 PM
> >> > > > > > > To: lucene-net-dev@lucene.apache.**org <
> >> > > > > lucene-net-dev@lucene.apache.org
> >> > > > > > >;
> >> > > > > > > lucene-net-user@lucene.apache.**org <lucene-net-
> >> > > > user@lucene.apache.org
> >> > > > > >
> >> > > > > > > Subject: [Lucene.Net] Is a Lucene.Net Line-by-Line
Jave port
> >> > > needed?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > After the large community response about moving
the code
> base
> >> > from
> >> > > > .Net
> >> > > > > > > 2.0 to Net 4.0 I am trying to figure out what
is the need
> for
> >> a
> >> > > > > > > line-by-line port.  Starting with Digy's excellent
work on
> the
> >> > > > > > > conversion to generics a priority of the 2.9.4g
release is
> the
> >> 2
> >> > > > > > > packages would not be interchangeable.  So faster
turnaround
> >> from
> >> > a
> >> > > > > java
> >> > > > > > > release won't matter to non line-by-line users
they will
> have
> >> to
> >> > > > wait
> >> > > > > > > until the updates are made to the non line-by-line
code
> base.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > My question is there really a user base for the
line-by-line
> >> > port?
> >> > > > > > > Anyone have a comment?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Scott
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > ------------------------------**---------------------------
> >> > > > > > > Disclaimer
> >> > > > > > >
> >> > > > > > > This message and any attachments are confidential
and/or
> >> > > privileged.
> >> > > > If
> >> > > > > > > this has been sent to you in error, please do
not use,
> retain
> >> or
> >> > > > > disclose
> >> > > > > > > them, and contact the sender as soon as possible.
> >> > > > > > >
> >> > > > > > > Oxford Analytica Ltd
> >> > > > > > > Registered in England: No. 1196703
> >> > > > > > > 5 Alfred Street, Oxford
> >> > > > > > > United Kingdom, OX1 4EH
> >> > > > > > > ------------------------------**---------------------------
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > >
> >> > >
> >> > >
> >> >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message