lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Aroush" <geo...@aroush.net>
Subject RE: [VOTE] Release of Apache Lucene.Net 2.0.0
Date Thu, 01 Mar 2007 14:59:46 GMT
Hi Erich,

Now that you mentioned the version you are using, I believe those are fixed
in 2.0 -- not necessarily in the same way as your fix.  Can you:

1) Try Lucene.Net 1.9.1 and see if you are still seeing the leaks.  I rather
have you try 2.0 but some API's have changed in 2.0.

2) Share with us the patch you used for 1.4.3 -- others may find it useful.

Regards,

-- George Aroush 

-----Original Message-----
From: Erich Eichinger [mailto:E.Eichinger@diamonddogs.cc] 
Sent: Thursday, March 01, 2007 4:12 AM
To: lucene-net-user@incubator.apache.org;
lucene-net-dev@incubator.apache.org
Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0


Hi George,

> The memory and resource leaks that you mentioned, have you observed 
> them or do you think they exist?

Yes, I experienced them in a couple of high loaded web-applications and the
fixes I mentioned took me 2 weeks to find them. TBH I experienced them using
Lucene.NET 1.4.3 but checking the codebase showed, that the respective code
didn't change to 2.0.

> Do you have sample code that demonstrates them?  
> I ask this because you said "possible". 

This is misleading. I said "possible" because the underlying IndexInput
might be a RAMDirectory. In case of FileDirectory (or the DBDirectory I
mentioned in my private email) there is almost certainly a leak because of
unclosed file-handles or db-connections.
I can write some sample code to show up the leaks if you like. Btw: I found
and fixed the leaks in v1.4.3 using SciTech's ".NET Memory Profiler" (don't
want to ad here, but it has been the only useful tool I found for this
purpose)

> As for your WeakHashtable suggestion and the use of NHibernate, this 
> isn't possible.

Another misunderstanding ;-). I didn't want you to link Lucene.NET to
NHibernate. I just wanted to point you to an implementation I am aware of.


> In my opinion, none of those issues are critical to stop this release.

Since I have to deal almost 100% with web-applications, for me these leaks
are definitely a showstopper. Depending on the application's traffic I
experienced OutOfMemoryExceptions every few hours. After applying the fixes
I described, everything is running smooth.

cheers,
Erich

> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net]
> Sent: Thursday, March 01, 2007 2:19 AM
> To: lucene-net-dev@incubator.apache.org;
> lucene-net-user@incubator.apache.org
> Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> 
> Hi Erich,
> 
> In my opinion, none of those issues are critical to stop this release.
> 
> The memory and resource leaks that you mentioned, have you observed 
> them or do you think they exist?  Do you have sample code that 
> demonstrates them?  I ask this because you said "possible".  Yes, I am 
> aware of a leak issue during sorting when Lucene.Net is compiled using 
> .NET 1.1, but with .NET 2.0, it disappears.
> 
> As for your WeakHashtable suggestion and the use of NHibernate, this 
> isn't possible.  This release is supporting .NET 1.1 so we are 
> somewhat limited with what we have.  No, I can't use NHibernate as 
> it's a 3rd party artifact and none ASF.
> 
> Regards,
> 
> -- George
> 
> 
> -----Original Message-----
> From: Erich Eichinger [mailto:E.Eichinger@diamonddogs.cc]
> Sent: Tuesday, February 27, 2007 9:04 AM
> To: lucene-net-user@incubator.apache.org;
> lucene-net-dev@incubator.apache.org
> Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> 
> Hi,
> 
> there are some resource leaks that lead to really nasty problems in 
> highloaded webapplications. I summed up the most required changes:
> 
> *)
> Index/CompoundFileReader.cs:260:
> Lucene.Net.Index.Compound.FileReader.CSIndexInput doesn't close 
> underlying IndexInput (possible resource leak)
> 
> 
> *)
> Index/TermInfosReader.cs
> Index/SegmentReader.cs
> 
> Usage of System.Threading.Thread.GetData()/.SetData() may result in 
> memory leaks in web-applications. Using 
> System.Runtime.Remoting.Messaging.CallContext is a much better choice.
> 
> 
> *)
> Search/FieldCacheImpl.cs
> 
> cache should be a "WeakHashtable" instead of Hashtable. 
> Otherwise cached readers can't ever be collected and cause 
> memory/resource leaks.
> 
> *)
> FieldSortedHitQueue.cs
> 
> the same problem as in FieldCacheImpl but with "Comparators" table.
> 
> Asfaik a possible implementation of a "WeakHashtable" can be found in 
> the NHibernate project.
> 
> 
> cheers,
> Erich
> 
> 
> > -----Original Message-----
> > From: George Aroush [mailto:george@aroush.net]
> > Sent: Tuesday, February 27, 2007 4:06 AM
> > To: lucene-net-user@incubator.apache.org;
> > lucene-net-dev@incubator.apache.org
> > Subject: [VOTE] Release of Apache Lucene.Net 2.0.0
> > 
> > Hi folks,
> > 
> > To follow the proper release process of ASF, please take a
> moment to
> > cast your vote to release Lucene.Net 2.0.0.  I have placed both the 
> > release candidate of source code and binary
> > here: http://people.apache.org/~aroush/
> > 
> > The change history for this release can be found here:
> > https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/C%
> > 23/src/HISTORY
> > .txt
> > 
> > In addition to Lucene.Net release, this release also
> contains ported
> > code in "contrib".
> > 
> > Please cast your vote on releasing Apache Lucene.Net 2.0.0 no later 
> > then this coming Saturday.
> > 
> > [ ] +1 Approve release
> > [x] -1 Veto release (please give reason)
> > 
> > Regards,
> > 
> > -- George Aroush
> > 
> > 
> 
> 


Mime
View raw message