lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "TJ Kolev" <tjko...@gmail.com>
Subject Re: [jira] Commented: (LUCENENET-106) Lucene.NET (Revision: 603121) is leaking memory
Date Thu, 15 Jan 2009 00:24:03 GMT
Hi guys,

I just came across this thread looking for something else. The version
I added in the Jira today works on exactly the idea being discussed
here.

tjk :)

On Mon, Jan 12, 2009 at 7:21 AM, digy digy <digydigy@gmail.com> wrote:
> OK. I see. It can be used in WeakHashTable v3.
>
> DIGY
>
>
> On Mon, Jan 12, 2009 at 11:28 AM, Eyal Post <eyalpost@epocalipse.com> wrote:
>
>> void Add(object keyX,object valueX)
>> {
>>         InternalTable.Add( new WeakEntry(keyX), value);
>> }
>>
>> Object Get(object keyX)
>> {
>>     return InternalTable[new WeakEntry(keyX)];
>> }
>>
>>
>> WeakEntry.GetHashCode = key.getHashCode;
>> WeakEntry.Equals = keyA == keyB
>>
>> Not sure this is clear enough - I can post full implementation if you'd
>> like
>> but that'll take me some more time.
>>
>> Eyal
>>
>>
>> > -----Original Message-----
>> > From: Digy [mailto:digydigy@gmail.com]
>> > Sent: Sunday, January 11, 2009 23:16 PM
>> > To: lucene-net-dev@incubator.apache.org
>> > Subject: RE: [jira] Commented: (LUCENENET-106) Lucene.NET
>> > (Revision: 603121) is leaking memory
>> > Importance: High
>> >
>> > Hi Eyal,
>> >
>> > I couldn't find a simple way to use the WeakEntry as a key
>> > and then access its value without enumerating all the keys in
>> > InternalTable.
>> > I think you suggest
>> >
>> > void Add(object keyX,object valueX)
>> > {
>> >       InternalTable.Add( new WeakEntry(keyX,valueX), value);
>> > } // or something like that.
>> >
>> > Then how do you think that we get the value of "Get(keyX)" ?
>> > Enumerating all the keys of InternalTable and comparing the
>> > "weakEntry.Key" with "keyX"?
>> > When thinking of successive "Add"s, this has the complexity of O(n*n).
>> >
>> > Can you share your solution?
>> >
>> > DIGY
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Eyal Post [mailto:eyalpost@epocalipse.com]
>> > Sent: Sunday, January 11, 2009 1:08 PM
>> > To: lucene-net-dev@incubator.apache.org
>> > Subject: RE: [jira] Commented: (LUCENENET-106) Lucene.NET
>> > (Revision: 603121) is leaking memory
>> >
>> > As Luc noted in the first comment to this patch, I think that
>> > if this class is to be called WeakHashTable (and therefor
>> > imply it is a general purpose weak hash table) then it should
>> > be implemented as a real hash table.
>> > Therefor, you should not use an array list and make linear
>> > comparisons.
>> > Instead WeakHashTable should internally use a hashtable.  The
>> > first implementation was in the right direction, the only fix
>> > should have been to use WeakEntry as a key to the table and
>> > not the hash code.
>> >
>> > Eyal
>> >
>> >
>> > > -----Original Message-----
>> > > From: Digy [mailto:digydigy@gmail.com]
>> > > Sent: Friday, January 09, 2009 20:03 PM
>> > > To: lucene-net-dev@incubator.apache.org
>> > > Subject: RE: [jira] Commented: (LUCENENET-106) Lucene.NET
>> > > (Revision: 603121) is leaking memory
>> > >
>> > > Hi Tim,
>> > >
>> > >
>> > >
>> > > Thanks for the comments. I will correct the 3,4,5.
>> > >
>> > >
>> > >
>> > > 1) Since ArrayList contains WeakEntries & sync.ed for multiple
>> > > threads, they can not be null and can not be any other
>> > type. I can not
>> > > find a bug here. Maybe you can send a more detailed description.
>> > >
>> > > 2), WeakHashTable is compatible with "HashTable" in .Net world.
>> > > "Ht.Add(key,value)" throws an exception for duplicate keys but
>> > >
>> > > "Ht[key]=value" just adds or replaces.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > DIGY
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > -----Original Message-----
>> > > From: Tim Januario (JIRA) [mailto:jira@apache.org]
>> > > Sent: Friday, January 09, 2009 7:16 PM
>> > > To: lucene-net-dev@incubator.apache.org
>> > > Subject: [jira] Commented: (LUCENENET-106) Lucene.NET
>> > > (Revision: 603121) is leaking memory
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >     [
>> > > https://issues.apache.org/jira/browse/LUCENENET-106?page=com.a
>> > tlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&fo
>> > cusedCommentId
>> > =12662433#action_12662433 ]
>> >
>> >
>> >
>> > Tim Januario commented on LUCENENET-106:
>> >
>> > ----------------------------------------
>> >
>> >
>> >
>> > Digy,
>> >
>> > 1) The WeakHashTable.Exists method creates a possible
>> > NullReferenceException because you used WeakEntries[i] As
>> > WeakEntry instead of (WeakEntry)WeakEntries[i].  This is only
>> > a warning but...
>> >
>> > 2) The Java WeakHashMap does not throw an execption when
>> > put() is called with a key that already exists in the table.
>> > It simply replaces the old value with the new value.  .NET
>> > Hashtable does the same.  The Add method of WeakHashTable
>> > should remove the WeakEntry if the key is found and replace
>> > it with the new value in order to stay in line instead of
>> > throwing the Exception.
>> >
>> > 3) WeakHashTableEnumarator should be WeakHashTableEnumerator
>> >
>> > 4) WeakEntry.HashCode is not used and should be removed.
>> >
>> > 5) WeakHashTable.InitialMem is not used and should be removed.
>> >
>> >
>> >
>> > Other than that, I think this looks good.
>> >
>> >
>> >
>> > -tim
>> >
>> >
>> >
>> > > Lucene.NET (Revision: 603121) is leaking memory
>> >
>> > > -----------------------------------------------
>> >
>> > >
>> >
>> > >                 Key: LUCENENET-106
>> >
>> > >                 URL:
>> > > https://issues.apache.org/jira/browse/LUCENENET-106
>> >
>> > >             Project: Lucene.Net
>> >
>> > >          Issue Type: Bug
>> >
>> > >         Environment: .NET 2.0
>> >
>> > >            Reporter: Anton K.
>> >
>> > >            Assignee: Digy
>> >
>> > >            Priority: Critical
>> >
>> > >         Attachments: DIGY-FieldCacheImpl.patch, Digy.rar,
>> > > luceneSrc_memUsage.patch, Paches for v2.3.1.rar, WeakHashTable
>> > > v2.patch, WeakHashTable+FieldCacheImpl.rar, WeakReferences.rar
>> >
>> > >
>> >
>> > >
>> >
>> > > readerCache Hashtable field (see FieldCacheImpl.cs) never releases
>> > > some
>> > hash items that have closed IndexReader object as a key. So a
>> > lot of Term instances are never released.
>> >
>> > > Java version of Lucene uses WeakHashMap and therefore doesn't have
>> > > this
>> > problem.
>> >
>> > > This bug can be reproduced only when Sort functionality used during
>> > search.
>> >
>> > > See following link for additional information.
>> >
>> > > http://www.gossamer-threads.com/lists/lucene/java-user/55681
>> >
>> > > Steps to reproduce:
>> >
>> > > 1)Create index
>> >
>> > > 2) Modify index by IndexWiter; Close IndexWriter
>> >
>> > > 3) Use IndexSearcher for searching with Sort; Close InexSearcher
>> >
>> > > 4) Go to step 2
>> >
>> > > You'll get OutOfMemoryException after some time of running
>> > this algorithm.
>> >
>> >
>> >
>> > --
>> >
>> > This message is automatically generated by JIRA.
>> >
>> > -
>> >
>> > You can reply to this email to add a comment to the issue online.
>> >
>> >
>>
>>
>

Mime
View raw message