lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Herndon <mhern...@wickedsoftware.net>
Subject Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?
Date Fri, 01 Jul 2011 14:52:38 GMT
@Rory, @All,

The only tickets I currently have for those is LUCENE-419, LUCENE-418

418, I should be able to push into the 2.9.4g branch tonight.    419 is a
long term goal and not as important as getting the tests fixed, of have the
tests broken down into what is actually a unit test, functional test, perf
or long running test. I can get into more why it needs to be done.

I'll also need to make document the what build script currently does on the
wiki & and make a few notes about testing, like using the RAMDirectory,
etc.

Things that need to get done or even be discussed.
 * There needs to be a running list of things to do/not to do with testing.
I don't know if this goes in a jira or do we keep a running list on the wiki
or site for people to pick up and  help with.
 * Tests need to run on mono and not Fail (there is a good deal of failing
tests on mono, mostly due to the temp directory have the C:\ in the path).
 * Assert.Throw<ExceptionType>() needs to be used instead of Try/Catch
Assert.Fail.  **
 * File & Path combines to the temp directory need helper methods,
     * e,g, having this in a hundred places is bad   new
System.IO.FileInfo(System.IO.Path.Combine(Support.AppSettings.Get("tempDir",
""), "testIndex"));
 * We should still be testing deprecated methods, but we need to use #pragma
warning disable/enable 0618  for testing those. otherwise compiler warnings
are too numerous to be anywhere near helpful.
 * We should only be using deprecated methods in places where they are being
explicitly tested, other tests that need that functionality in order to
validate those tests should be re factored to use methods that are not
deprecated.
 * Identify code that could be abstracted into test utility classes.
 * Infrastructure Validation tests need to be made, anything that seems like
infrastructure.  e.g. does the temp directory exist, does the folders that
the tests use inside the temp directory exist, can we read/write to those
folders. (if a ton of tests fail due to the file system, we should be able
to point out that it was due to permissions or missing folders, files,
etc).
 * Identify what classes need an interface, abstract class or inherited in
order to create testing mocks. (once those classes are created, they should
be documented in the wiki).



** Asset.Throws needs to replace stuff like the following. We should also be
checking the messages for exceptions and make sure they make sense and can
help users fix isses if the exceptions are aimed at the library users.
try
{
d = DateTools.StringToDate("97"); // no date
Assert.Fail();
}
catch (System.FormatException e)
{
/* expected exception */
}

On Thu, Jun 30, 2011 at 11:48 PM, Rory Plaire <codekaizen@gmail.com> wrote:

> So, veering towards action - are there concrete tasks written up anywhere
> for the unit tests? If a poor schlep like me wanted to dig in and start to
> improve them, where would I get the understanding of what is good and what
> needs help?
>
> -r
>
> On Thu, Jun 30, 2011 at 3:29 PM, Digy <digydigy@gmail.com> wrote:
>
> > I can not say I like this approach, but till we find an automated
> way(with
> > good results), it seems to be the only way we can use.
> >
> > DIGY
> >
> > -----Original Message-----
> > From: Troy Howard [mailto:thoward37@gmail.com]
> > Sent: Friday, July 01, 2011 12:43 AM
> > To: lucene-net-dev@lucene.apache.org
> > Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?
> >
> > Scott -
> >
> > The idea of the automated port is still worth doing. Perhaps it makes
> sense
> > for someone more passionate about the line-by-line idea to do that work?
> >
> > I would say, focus on what makes sense to you. Being productive,
> regardless
> > of the specific direction, is what will be most valuable. Once you start,
> > others will join and momentum will build. That is how these things work.
> >
> > I like DIGY's approach too, but the problem with it is that it is a
> > never-ending manual task. The theory behind the automated port is that it
> > may reduce the manual work. It is complicated, but once it's built and
> > works, it will save a lot of future development hours. If it's built in a
> > sufficiently general manner, it could be useful for other project like
> > Lucene.Net that want to automate a port from Java to C#.
> >
> > It might make sense for that to be a separate project from Lucene.Net
> > though.
> >
> > -T
> >
> >
> > On Thu, Jun 30, 2011 at 2:13 PM, Scott Lombard <lombardenator@gmail.com
> > >wrote:
> >
> > > Ok I think I asked the wrong question.  I am trying to figure out where
> > to
> > > put my time.  I was thinking about working on the automated porting
> > system,
> > > but when I saw the response to the .NET 4.0 discussions I started to
> > > question if that is the right direction.  The community seemed to be
> more
> > > interested in the .NET features.
> > >
> > > The complexity of the automated tool is going to become very high and
> > will
> > > probably end up with a line-for-line style port.  So I keep asking my
> > self
> > > is the automated tool worth it.  I don't think it is.
> > >
> > > I like the method has been Digy is using for porting the code.  So I
> > guess
> > > for me the real question is Digy where did you see 2.9.4g going next
> and
> > > what do you need help on?
> > >
> > > Scott
> > >
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Digy [mailto:digydigy@gmail.com]
> > > > Sent: Thursday, June 30, 2011 4:20 PM
> > > > To: lucene-net-dev@lucene.apache.org
> > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port
> > needed?
> > > >
> > > > Michael,
> > > > You interpret the report as "whoever commits code wins"? But when I
> > look
> > > > at it, I see "a lof of talk, no work". .Net community is not
> interested
> > > in
> > > > contributing.
> > > > I really don't understand what hinders people to work on Lucene.Net.
> As
> > I
> > > > did for 2.9.4g, grab the code, do whatever you want on it and submit
> > > back.
> > > > If it doesn't fit to the project's direction it can still find a
> place
> > in
> > > > contrib or in branch. All of the approaches can live side by side
> > happily
> > > > in the Lucene.Net repository.
> > > >
> > > > Troy,
> > > > I also don't understand why do you wait for 2.9.4g? It is a *branch*
> > and
> > > > has nothing to do with the trunk. It need not be an offical release
> and
> > > > can live in branch as a PoC.
> > > >
> > > >
> > > > As a result, I got bored to listen to "this should be done that way".
> > > What
> > > > I want to see is "I did it that way, should we continue with this".
> > > >
> > > > DIGY
> > > >
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Troy Howard [mailto:thoward37@gmail.com]
> > > > Sent: Thursday, June 30, 2011 10:47 PM
> > > > To: lucene-net-dev@lucene.apache.org
> > > > Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port
> > needed?
> > > >
> > > > Michael,
> > > >
> > > > I agree with everything you said. My point in saying "whoever commits
> > > code
> > > > wins" was to illustrate the reality of how and why the project has
> the
> > > > current form.
> > > >
> > > > Building consensus is difficult. It is an essential first step before
> > we
> > > > can
> > > > do something like make a list of bit-sized pieces of work that others
> > can
> > > > work on.
> > > >
> > > > This is why my real message of "Let's find a way to accommodate both"
> > is
> > > > so
> > > > important. It allows us to build consensus, so that we can settle on
> a
> > > > direction and structure our work.
> > > >
> > > > Until we accomplish that, it really is "whoever commits code wins",
> and
> > > > that
> > > > is an unhealthy and unmaintainable way to operate.
> > > >
> > > > From a technical perspective, your statements about the unit tests
> are
> > > > completely accurate. They really need a LOT of reworking. That's the
> > very
> > > > first step before making any significant changes. Part of the problem
> > is
> > > > that the tests themselves are not well written. The other part is
> that
> > > the
> > > > Lucene object model was not designed for testability, and it makes
> > > writing
> > > > good tests more difficult, and certain tests might not be possible.
> It
> > > > will
> > > > be difficult to write good unit tests without re-structuring. The
> > biggest
> > > > issue is the use of abstract classes with base behaviour vs
> interfaces
> > or
> > > > fully abstracted classes. Makes mocking tough. This is the direction
> I
> > > was
> > > > going when I started the Lucere project. I'd like to start in on that
> > > work
> > > > after the 2.9.4g release.
> > > >
> > > > Thanks,
> > > > Troy
> > > >
> > > >
> > > > On Thu, Jun 30, 2011 at 12:04 PM, Michael Herndon <
> > > > mherndon@wickedsoftware.net> wrote:
> > > >
> > > > > I'd say that is all the more reasons that we need to work smarter
> and
> > > > not
> > > > > harder. I'd also say thats a good reason to make sure we build
> > > consensus
> > > > > rather than just saying whoever commits code wins.
> > > > >
> > > > > And its a damn good reason to focus on the effort of growing the
> > number
> > > > of
> > > > > contributors and lowing the barrier to submitting patches, breaking
> > > > things
> > > > > down into pieces that people would feel confident to work on
> without
> > > > > being overwhelmed by the complexity of Lucene.Net.
> > > > >
> > > > > There is a pretty big gap between Lucene 2.9.x to Lucene 4.0 and
> the
> > > > > internals and index formats are significantly different including
> > > nixing
> > > > > the
> > > > > current vint file format and using byte[] array slices for Terms
> > > instead
> > > > of
> > > > > char[].
> > > > >
> > > > > So while porting 1 to 1 while require less knowledge or thought,
> its
> > > > most
> > > > > likely going to require more hours of work. And Its definitely not
> > > going
> > > > to
> > > > > guarantee the stability of the code or that its great code.
> > > > >
> > > > > I'd have to say that its not as stable as most would believe at the
> > > > moment.
> > > > >
> > > > > Most of the tests avoid anything that remotely looks like it knows
> > > about
> > > > > the
> > > > > DRY principle and there is a static constructor in the core test
> case
> > > > that
> > > > > throws an exception if it doesn't find an environment variable
> "TEMP"
> > > > which
> > > > > will fail 90% of the tests and nunit will be unable to give you a
> > clear
> > > > > reason why.  Just to name a few issues I came across working
> towards
> > > > > getting
> > > > > Lucene.Net into CI.  I haven't even started really digging in under
> > the
> > > > > covers of the code yet.
> > > > >
> > > > > So my suggestion is to chew on this a bit more and build consensus,
> > > > avoid
> > > > > fracturing people into sides.  Be open to reservations and concerns
> > > that
> > > > > others have and continue to address them.
> > > > >
> > > > > - Michael
> > > > >
> > > > >
> > > > > On Thu, Jun 30, 2011 at 2:10 PM, Digy <digydigy@gmail.com>
wrote:
> > > > >
> > > > > > Although there are a lot of people using Lucene.Net, this is
our
> > > > > > contribution report for the past 5 years.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ConfigureReport.jspa?atl_token=A5KQ-
> > > > 2Q
> > > > > >
> > > > > >
> > > > >
> AV-T4JA-FDED|3204f7e696067a386144705075c074e991db2a2b|lin&versionId=-
> > > > 1&issue
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> Status=all&selectedProjectId=12310290&reportKey=com.sourcelabs.jira.plugin
> > > > .r
> > > > > > eport.contributions%3Acontributionreport&Next=Next
> > > > > >
> > > > > >
> > > > > > DIGY
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Ayende Rahien [mailto:ayende@ayende.com]
> > > > > > Sent: Thursday, June 30, 2011 8:16 PM
> > > > > > To: Rory Plaire; lucene-net-dev@lucene.apache.org
> > > > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave
port
> > > > needed?
> > > > > >
> > > > > > As someone from the nhibernate project
> > > > > > We stopped following hibernate a while ago, and haven't regretted
> > it
> > > > > > We have mire features, less bugs and better code base
> > > > > >
> > > > > > Sent from my Windows Phone From: Rory Plaire
> > > > > > Sent: Thursday, June 30, 2011 19:58
> > > > > > To: lucene-net-dev@lucene.apache.org
> > > > > > Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave
port
> > > > needed?
> > > > > > I don't want to drag this out much longer, but I am curious
with
> > > > people
> > > > > who
> > > > > > hold the "line-by-line" sentiment - are you NHibernate users?
> > > > > >
> > > > > > -r
> > > > > >
> > > > > > On Thu, Jun 30, 2011 at 2:39 AM, Noel Lysaght <
> > lysaghtn@hotmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Can I just plug in my bit and say I agree 100% with what
Moray
> > has
> > > > > > outlined
> > > > > > > below.
> > > > > > >
> > > > > > > If we move away from the line by line port then over time
we'll
> > > > loose
> > > > > out
> > > > > > > on the momentum that is Lucene and the improvements that
they
> > make.
> > > > > > > It is only if the Lucene.NET community has expertise in
search,
> >  a
> > > > >  deep
> > > > > > > knowledge of the project and the community can guarantee
that
> the
> > > > > > knowledge
> > > > > > > will survive members coming and going should such a
> consideration
> > > be
> > > > > > give.
> > > > > > >
> > > > > > > When Lucene.NET has stood on it's feet for a number of
years
> > after
> > > > it
> > > > > has
> > > > > > > moved out of Apache incubation should consideration be
given to
> > > > > > abandoning
> > > > > > a
> > > > > > > line by line port.
> > > > > > > By all means extend and wrap the libraries in .NET equivalents
> > and
> > > > .NET
> > > > > > > goodness like LINQ (we do this internally in our company
at the
> > > > > moment);
> > > > > > but
> > > > > > > leave the core of the project on a line by line port.
> > > > > > >
> > > > > > > Just my tu-pence worth.
> > > > > > >
> > > > > > > Kind Regards
> > > > > > > Noel
> > > > > > >
> > > > > > >
> > > > > > > -----Original Message----- From: Moray McConnachie
> > > > > > > Sent: Thursday, June 30, 2011 10:25 AM
> > > > > > >
> > > > > > > To: lucene-net-user@lucene.apache.**org<
> > > > > > lucene-net-user@lucene.apache.org>
> > > > > > > Cc:
> > > > > > lucene-net-dev@incubator.**apache.org<
> > > > > lucene-net-dev@incubator.apache.org>
> > > > > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line
Jave
> port
> > > > > needed?
> > > > > > >
> > > > > > > I don't think I'm as hard core on this as Neal, but remember:
> the
> > > > > > > history of the Lucene.NET project is that all the intellectual
> > > work,
> > > > > all
> > > > > > > the understanding of search, all the new features come
from the
> > > > Lucene
> > > > > > > Java folks. Theirs is an immensely respected project, and
I
> trust
> > > > them
> > > > > > > to add new features that will be well-tested and
> well-researched,
> > > > and
> > > > > to
> > > > > > > have a decent roadmap which I can trust they will execute
on.
> > > > > > >
> > > > > > > Now I know there's been an influx of capable developers
to
> > > > Lucene.NET
> > > > > > > who are ready, willing and (I'm going to assume) able to
add a
> > lot
> > > > more
> > > > > > > value in a generic .NET implementation as they change it.
But
> > it'll
> > > > > take
> > > > > > > a while before I trust a .NET dedicated framework which
is
> > > > > significantly
> > > > > > > diverged from Java in the way I do the line-by-line version.
> And
> > at
> > > > > what
> > > > > > > stage is it not just not a line-by-line port, but not a
port at
> > > all?
> > > > > > >
> > > > > > > At the same time, I recognise that if this project is going
to
> > > > > continue,
> > > > > > > and attract good developers, it has to change in this
> direction.
> > > > > > >
> > > > > > > So that said, I can see why a line-by-line port might not
be
> > > > > > > sustainable. And most people don't need it. But most of
us
> using
> > > > Lucene
> > > > > > > in production systems do need a system that we can trust
and
> rely
> > > > on.
> > > > > So
> > > > > > > let me chime in with someone else's plea, to keep the general
> > > > structure
> > > > > > > close to Lucene, to keep the same general objects and
> inheritance
> > > > > > > set-up, and to keep the same method names, even if you
add
> other
> > > > > methods
> > > > > > > and classes to provide additional functionality. ABSOLUTELY
the
> > > same
> > > > > > > file formats. End users benefit a lot from a high degree
of
> > > > similarity,
> > > > > > > with good documentation and help being available from the
Java
> > > > > > > community.
> > > > > > >
> > > > > > > Yours,
> > > > > > > Moray
> > > > > > > ------------------------------**-------
> > > > > > > Moray McConnachie
> > > > > > > Director of IT    +44 1865 261 600
> > > > > > > Oxford Analytica  http://www.oxan.com
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Granroth, Neal V.
> > > > > >
> > > > [mailto:neal.granroth@**thermofisher.com<
> > neal.granroth@thermofisher.com>
> > > > > > > ]
> > > > > > > Sent: 29 June 2011 20:47
> > > > > > > To: lucene-net-user@lucene.apache.**org<
> > > > > > lucene-net-user@lucene.apache.org>
> > > > > > > Cc:
> > > > > > lucene-net-dev@incubator.**apache.org<
> > > > > lucene-net-dev@incubator.apache.org>
> > > > > > > Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line
Jave
> port
> > > > > needed?
> > > > > > >
> > > > > > > This is has been discussed many times.
> > > > > > > Lucene.NET is not valid, the code cannot be trusted, if
it is
> not
> > a
> > > > > > > line-by-line port.  It ceases to be Lucene.
> > > > > > >
> > > > > > > - Neal
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Scott Lombard
> > > > > > [mailto:lombardenator@gmail.**com<lombardenator@gmail.com>
> > > > > > > ]
> > > > > > > Sent: Wednesday, June 29, 2011 1:58 PM
> > > > > > > To: lucene-net-dev@lucene.apache.**org <
> > > > > lucene-net-dev@lucene.apache.org
> > > > > > >;
> > > > > > > lucene-net-user@lucene.apache.**org <lucene-net-
> > > > user@lucene.apache.org
> > > > > >
> > > > > > > Subject: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave
port
> > > needed?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > After the large community response about moving the code
base
> > from
> > > > .Net
> > > > > > > 2.0 to Net 4.0 I am trying to figure out what is the need
for a
> > > > > > > line-by-line port.  Starting with Digy's excellent work
on the
> > > > > > > conversion to generics a priority of the 2.9.4g release
is the
> 2
> > > > > > > packages would not be interchangeable.  So faster turnaround
> from
> > a
> > > > > java
> > > > > > > release won't matter to non line-by-line users they will
have
> to
> > > > wait
> > > > > > > until the updates are made to the non line-by-line code
base.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > My question is there really a user base for the line-by-line
> > port?
> > > > > > > Anyone have a comment?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Scott
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > ------------------------------**---------------------------
> > > > > > > Disclaimer
> > > > > > >
> > > > > > > This message and any attachments are confidential and/or
> > > privileged.
> > > > If
> > > > > > > this has been sent to you in error, please do not use,
retain
> or
> > > > > disclose
> > > > > > > them, and contact the sender as soon as possible.
> > > > > > >
> > > > > > > Oxford Analytica Ltd
> > > > > > > Registered in England: No. 1196703
> > > > > > > 5 Alfred Street, Oxford
> > > > > > > United Kingdom, OX1 4EH
> > > > > > > ------------------------------**---------------------------
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> > >
> > >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message