lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erich Eichinger" <E.Eichin...@diamonddogs.cc>
Subject RE: [VOTE] Release of Apache Lucene.Net 2.0.0
Date Wed, 07 Mar 2007 20:53:04 GMT

Hi all,

I attached the patch for 1.4.3-final-004. It fixes a couple of memory-/resource leaks and
some bugs in GermanStemmer. I have this version in use for over 1 year now in the "wilderness"
and never had any problems (at least with Lucene) since then.

Unfortunately I didn't have time to look into 2.0 more detailed yet.

hope this helps,
Erich

> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net] 
> Sent: Thursday, March 01, 2007 4:00 PM
> To: lucene-net-user@incubator.apache.org; 
> lucene-net-dev@incubator.apache.org
> Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> 
> Hi Erich,
> 
> Now that you mentioned the version you are using, I believe 
> those are fixed in 2.0 -- not necessarily in the same way as 
> your fix.  Can you:
> 
> 1) Try Lucene.Net 1.9.1 and see if you are still seeing the 
> leaks.  I rather have you try 2.0 but some API's have changed in 2.0.
> 
> 2) Share with us the patch you used for 1.4.3 -- others may 
> find it useful.
> 
> Regards,
> 
> -- George Aroush 
> 
> -----Original Message-----
> From: Erich Eichinger [mailto:E.Eichinger@diamonddogs.cc]
> Sent: Thursday, March 01, 2007 4:12 AM
> To: lucene-net-user@incubator.apache.org;
> lucene-net-dev@incubator.apache.org
> Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> 
> 
> Hi George,
> 
> > The memory and resource leaks that you mentioned, have you observed 
> > them or do you think they exist?
> 
> Yes, I experienced them in a couple of high loaded 
> web-applications and the
> fixes I mentioned took me 2 weeks to find them. TBH I 
> experienced them using
> Lucene.NET 1.4.3 but checking the codebase showed, that the 
> respective code
> didn't change to 2.0.
> 
> > Do you have sample code that demonstrates them?  
> > I ask this because you said "possible". 
> 
> This is misleading. I said "possible" because the underlying 
> IndexInput
> might be a RAMDirectory. In case of FileDirectory (or the 
> DBDirectory I
> mentioned in my private email) there is almost certainly a 
> leak because of
> unclosed file-handles or db-connections.
> I can write some sample code to show up the leaks if you 
> like. Btw: I found
> and fixed the leaks in v1.4.3 using SciTech's ".NET Memory 
> Profiler" (don't
> want to ad here, but it has been the only useful tool I found for this
> purpose)
> 
> > As for your WeakHashtable suggestion and the use of 
> NHibernate, this 
> > isn't possible.
> 
> Another misunderstanding ;-). I didn't want you to link Lucene.NET to
> NHibernate. I just wanted to point you to an implementation I 
> am aware of.
> 
> 
> > In my opinion, none of those issues are critical to stop 
> this release.
> 
> Since I have to deal almost 100% with web-applications, for 
> me these leaks
> are definitely a showstopper. Depending on the application's traffic I
> experienced OutOfMemoryExceptions every few hours. After 
> applying the fixes
> I described, everything is running smooth.
> 
> cheers,
> Erich
> 
> > -----Original Message-----
> > From: George Aroush [mailto:george@aroush.net]
> > Sent: Thursday, March 01, 2007 2:19 AM
> > To: lucene-net-dev@incubator.apache.org;
> > lucene-net-user@incubator.apache.org
> > Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> > 
> > Hi Erich,
> > 
> > In my opinion, none of those issues are critical to stop 
> this release.
> > 
> > The memory and resource leaks that you mentioned, have you observed 
> > them or do you think they exist?  Do you have sample code that 
> > demonstrates them?  I ask this because you said "possible". 
>  Yes, I am 
> > aware of a leak issue during sorting when Lucene.Net is 
> compiled using 
> > .NET 1.1, but with .NET 2.0, it disappears.
> > 
> > As for your WeakHashtable suggestion and the use of 
> NHibernate, this 
> > isn't possible.  This release is supporting .NET 1.1 so we are 
> > somewhat limited with what we have.  No, I can't use NHibernate as 
> > it's a 3rd party artifact and none ASF.
> > 
> > Regards,
> > 
> > -- George
> > 
> > 
> > -----Original Message-----
> > From: Erich Eichinger [mailto:E.Eichinger@diamonddogs.cc]
> > Sent: Tuesday, February 27, 2007 9:04 AM
> > To: lucene-net-user@incubator.apache.org;
> > lucene-net-dev@incubator.apache.org
> > Subject: RE: [VOTE] Release of Apache Lucene.Net 2.0.0
> > 
> > Hi,
> > 
> > there are some resource leaks that lead to really nasty problems in 
> > highloaded webapplications. I summed up the most required changes:
> > 
> > *)
> > Index/CompoundFileReader.cs:260:
> > Lucene.Net.Index.Compound.FileReader.CSIndexInput doesn't close 
> > underlying IndexInput (possible resource leak)
> > 
> > 
> > *)
> > Index/TermInfosReader.cs
> > Index/SegmentReader.cs
> > 
> > Usage of System.Threading.Thread.GetData()/.SetData() may result in 
> > memory leaks in web-applications. Using 
> > System.Runtime.Remoting.Messaging.CallContext is a much 
> better choice.
> > 
> > 
> > *)
> > Search/FieldCacheImpl.cs
> > 
> > cache should be a "WeakHashtable" instead of Hashtable. 
> > Otherwise cached readers can't ever be collected and cause 
> > memory/resource leaks.
> > 
> > *)
> > FieldSortedHitQueue.cs
> > 
> > the same problem as in FieldCacheImpl but with "Comparators" table.
> > 
> > Asfaik a possible implementation of a "WeakHashtable" can 
> be found in 
> > the NHibernate project.
> > 
> > 
> > cheers,
> > Erich
> > 
> > 
> > > -----Original Message-----
> > > From: George Aroush [mailto:george@aroush.net]
> > > Sent: Tuesday, February 27, 2007 4:06 AM
> > > To: lucene-net-user@incubator.apache.org;
> > > lucene-net-dev@incubator.apache.org
> > > Subject: [VOTE] Release of Apache Lucene.Net 2.0.0
> > > 
> > > Hi folks,
> > > 
> > > To follow the proper release process of ASF, please take a
> > moment to
> > > cast your vote to release Lucene.Net 2.0.0.  I have 
> placed both the 
> > > release candidate of source code and binary
> > > here: http://people.apache.org/~aroush/
> > > 
> > > The change history for this release can be found here:
> > > https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/C%
> > > 23/src/HISTORY
> > > .txt
> > > 
> > > In addition to Lucene.Net release, this release also
> > contains ported
> > > code in "contrib".
> > > 
> > > Please cast your vote on releasing Apache Lucene.Net 
> 2.0.0 no later 
> > > then this coming Saturday.
> > > 
> > > [ ] +1 Approve release
> > > [x] -1 Veto release (please give reason)
> > > 
> > > Regards,
> > > 
> > > -- George Aroush
> > > 
> > > 
> > 
> > 
> 
> 

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message