lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Itamar Syn-Hershko <ita...@code972.com>
Subject Re: NIOFSDirectory is dangerous and should be re-written
Date Sun, 23 Aug 2015 16:17:09 GMT
Again, PR for this would be welcome.

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Lucene.NET committer and PMC member

On Sun, Aug 23, 2015 at 6:58 PM, Oren Eini (Ayende Rahien) <
ayende@ayende.com> wrote:

> It should be removed, then, or the old impl brought back.
> Note that a much better alternative would be to have an mmap directory that
> actually works.
> At least for input.
>
> A major headache in properly building efficient code here is that a lot of
> the code isn't actually closing the cloned instances, that means that we
> can't really know when the usage of an input is truly at an end.
>
> *Hibernating Rhinos Ltd  *
>
> Oren Eini* l CEO l *Mobile: + 972-52-548-6969
>
> Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
>
>
>
> On Sun, Aug 23, 2015 at 6:28 PM, Itamar Syn-Hershko <itamar@code972.com>
> wrote:
>
> > NIOFSDir is a concept that is easy to write in Java (because of java.nio)
> > and impossible to port as-is to .NET. A good, stable, version of an
> > equivalent implementation would use proper CLR APIs and possible
> > async/await.
> >
> > PR would be welcome.
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Lucene.NET committer and PMC member
> >
> > On Sun, Aug 23, 2015 at 6:00 PM, Oren Eini (Ayende Rahien) <
> > ayende@ayende.com> wrote:
> >
> > > I checked the latest implementation of NIOFSDirectory in the
> lucene.net
> > > repository, and it is doing horrible things there.
> > > In particular, it is supposed to be non blocking, but it is calling
> this
> > > method:
> > >
> > > [image: Inline image 1]
> > >
> > > This is now a SINGLE GLOBAL LOCK for all reads.
> > > So if you are trying to switch from the SimpleFSDirectory lock per file
> > to
> > > a global lock across all indexes.that would kill any hope of good
> > > performance.
> > >
> > > A proper way to handle this, at least on Win32, is shown here:
> > >
> > >
> > >
> >
> https://github.com/ayende/ravendb/blob/7c0cce5b1ab08f824d6c51908d88d00a417f5c1f/Raven.Database/Indexing/LuceneCodecDirectory.cs#L129
> > >
> > >
> > >
> > > *Hibernating Rhinos Ltd  *
> > >
> > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > >
> > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > >
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message