lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Currens <currens.ch...@gmail.com>
Subject Re: Lucene v3.6
Date Wed, 02 Jan 2013 00:14:21 GMT
I would like to add async/await, but I think adding a MMapDirectory, first,
is more important, and also something we could do now for the 3.x versions
of lucene.  I know we've looked at it before (LUCENE-425, I think?), and it
was turned down because it had poor performance.  I've tested a quick port
that I wrote and it performs the same under light load (8 threads) as it
does under heavy load (250 threads), able to complete 6000 queries in 74
seconds.  It performs the same as SimpleFSDirectory when it comes to light
load, but the SimpleFSDirectory performs horribly under heavy load, where
the 250 threads querying 6000 times total took about 160 seconds.

Another problem with supporting both .NET 3.5 and .NET 4.0, is that it
makes it more difficult in some respects to add things like methods that
return Task<T>, since it's not present in 3.5.  I think we need to maintain
two separate ports, a 3.x branch and a 4.x branch, although we don't have a
committer base that is active enough to do that.  The 3.x branch could
support .NET 3.5 and 4.0, where the 4.0 branch would only support .NET 4.0.
 If you look at the differences between 3.0.3 and 3.1, there are huge
performance optimizations that have been done, and to not give those to
.NET 3.5 users seems to be a wasted opportunity, in my opinion.  A lot of
the difficulties that come with porting Lucene 4.0 into .NET 3.5 come in
the later versions of the 3.x branch, probably around 3.5 or 3.6.


On Tue, Jan 1, 2013 at 3:43 PM, Nicholas Paldino [.NET/C# MVP] <
casperOne@caspershouse.com> wrote:

> As a somewhat related issue with .NET 4.0, are there plans to move to
> asynchronous calls at any point for what are now blocking I/O operations?
>
> Task<T> in 4.0 has made removing blocking I/O operations in C#/.NET 4.0
> fairly simple (although not as simple as C# 4.5 with async/await and core
> framework changes to support Task<T>).
>
> I ask because I'm thinking the throughput and/or performance of of
> Lucene.NET would be increased dramatically (by reducing waits on file-based
> I/O operations) but it would be a large architectural change.
>
> Perhaps it's something to keep in mind for the future.
>
>  - Nicholas Paldino
>
> On Jan 1, 2013, at 5:38 PM, "Christopher Currens" <currens.chris@gmail.com>
> wrote:
>
> > There are issues that need to be discussed about the 4.0 port and being
> > able to continue support of .NET 3.5.  I can only think of one example
> > right now, but I've looked through most of the code for lucene 4.0, and
> > there's a *very* heavy use of variance that would be difficult to
> maintain
> > outside of .NET 4.x.  If you want a good example of this, check out the
> > lucene.util.ast package and its usages.  This isn't the only area that
> uses
> > both contravariance and covariance, but it's the only one I could think
> of
> > off of the top of my head.
> >
> > I started porting parts of it, just to see what could be done (nothing
> > significant).  It's been a month or so since I worked on it, so my memory
> > of what I found is foggy.  The variance was the biggest thing I saw that
> > was an issue, but I think there were a few other things.  I'd have to
> look
> > at it again to see.
> >
> >
> > Thanks,
> > Christopher
> >
> >
> > On Tue, Jan 1, 2013 at 12:50 AM, Itamar Syn-Hershko <itamar@code972.com
> >wrote:
> >
> >> The general direction should be to put most of the efforts on a v4 port
> >> (4.1 probably...) and to start finding pieces in the codebase we can
> easily
> >> isolate and .NET-ify. Mostly readers, writers, structures and
> conversions.
> >>
> >> Re git on apache - I believe you should ask infra.
> >>
> >>
> >> On Tue, Dec 18, 2012 at 6:54 PM, Christopher Currens <
> >> currens.chris@gmail.com> wrote:
> >>
> >>> Don't think I've forgotten about this.  While I have no idea how git
> >> under
> >>> Apache looks like, I have a lot of comments on Lucene 3.6 (and 4.0)
> that
> >> I
> >>> need to discuss on this mailing list directly relating to the porting
> >> work
> >>> and the future direction of lucene.net.  I've had an email in my
> drafts
> >>> folder for about 3 weeks now but this time of year has been so busy I
> >>> haven't had a chance to finish it.  Soon, though, I hope.
> >>>
> >>>
> >>> Thanks,
> >>> Christopher
> >>>
> >>>
> >>> On Sat, Dec 15, 2012 at 1:09 PM, Prescott Nasser <
> geobmx540@hotmail.com
> >>>> wrote:
> >>>
> >>>> Hey Guys -  I've been quietly working in the background on
> >> administrative
> >>>> stuff for a while. I really only have two things on my to-do list -
 -
> >>> What
> >>>> does git under apache look like? I can't remember who asked this, but
> I
> >>>> know I owe digging up the answer- Lucene 3.6 - planning, moving
> >> forward (
> >>>> https://github.com/apache/lucene-solr/compare/5261b571...e4402c22c).
> >> Do
> >>>> we just want to start picking things off and committing them? Are
> there
> >>>> other refactoring issues we want to tackle with 3.6? We should make
> >> jira
> >>>> tickets and start tracking. Happy holidays everyone! ~Prescott
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message