lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Digy" <digyd...@gmail.com>
Subject RE: [jira] Commented: (LUCENENET-106) Lucene.NET (Revision: 603121) is leaking memory
Date Sun, 11 Jan 2009 12:58:16 GMT
It does, in fact, use a HashTable table internally. ArrayLists are used to
store WeakEntries whose keys' HashCodes are the same. Since the probability
of a collision is very low, ArrayLists contain most of the time only one
WeakEntry.

 

 

 

     * HashTable:                                 +------> WeakEntry

     * +----------|--------------------------+    |

     * | HashCode | WeakEntries (ArrayList)  |----+------> WeakEntry

     * +----------+--------------------------+    |

     * | HashCode | WeakEntries (ArrayList)  |    +------> WeakEntry

     * +----------+--------------------------+

 

DIGY

 

 

 

 

-----Original Message-----
From: Eyal Post [mailto:eyalpost@epocalipse.com] 
Sent: Sunday, January 11, 2009 1:08 PM
To: lucene-net-dev@incubator.apache.org
Subject: RE: [jira] Commented: (LUCENENET-106) Lucene.NET (Revision: 603121)
is leaking memory

 

As Luc noted in the first comment to this patch, I think that if this class

is to be called WeakHashTable (and therefor imply it is a general purpose

weak hash table) then it should be implemented as a real hash table.

Therefor, you should not use an array list and make linear comparisons.

Instead WeakHashTable should internally use a hashtable.  The first

implementation was in the right direction, the only fix should have been to

use WeakEntry as a key to the table and not the hash code.

 

Eyal

 

 

> -----Original Message-----

> From: Digy [mailto:digydigy@gmail.com] 

> Sent: Friday, January 09, 2009 20:03 PM

> To: lucene-net-dev@incubator.apache.org

> Subject: RE: [jira] Commented: (LUCENENET-106) Lucene.NET 

> (Revision: 603121) is leaking memory

> 

> Hi Tim,

> 

>  

> 

> Thanks for the comments. I will correct the 3,4,5.

> 

>  

> 

> 1) Since ArrayList contains WeakEntries & sync.ed for 

> multiple threads, they can not be null and can not be any 

> other type. I can not find a bug here. Maybe you can send a 

> more detailed description.

> 

> 2), WeakHashTable is compatible with "HashTable" in .Net 

> world. "Ht.Add(key,value)" throws an exception for duplicate keys but

> 

> "Ht[key]=value" just adds or replaces.

> 

>  

> 

>  

> 

> DIGY

> 

>  

> 

>  

> 

>  

> 

> -----Original Message-----

> From: Tim Januario (JIRA) [mailto:jira@apache.org]

> Sent: Friday, January 09, 2009 7:16 PM

> To: lucene-net-dev@incubator.apache.org

> Subject: [jira] Commented: (LUCENENET-106) Lucene.NET 

> (Revision: 603121) is leaking memory

> 

>  

> 

>  

> 

>     [ 

> https://issues.apache.org/jira/browse/LUCENENET-106?page=com.a

tlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId

=12662433#action_12662433 ] 

 

 

 

Tim Januario commented on LUCENENET-106:

 

----------------------------------------

 

 

 

Digy,

 

1) The WeakHashTable.Exists method creates a possible NullReferenceException

because you used WeakEntries[i] As WeakEntry instead of

(WeakEntry)WeakEntries[i].  This is only a warning but...

 

2) The Java WeakHashMap does not throw an execption when put() is called

with a key that already exists in the table.  It simply replaces the old

value with the new value.  .NET Hashtable does the same.  The Add method of

WeakHashTable should remove the WeakEntry if the key is found and replace it

with the new value in order to stay in line instead of throwing the

Exception.

 

3) WeakHashTableEnumarator should be WeakHashTableEnumerator

 

4) WeakEntry.HashCode is not used and should be removed.

 

5) WeakHashTable.InitialMem is not used and should be removed.

 

 

 

Other than that, I think this looks good.

 

 

 

-tim

 

 

 

> Lucene.NET (Revision: 603121) is leaking memory

 

> -----------------------------------------------

 

> 

 

>                 Key: LUCENENET-106

 

>                 URL: 

> https://issues.apache.org/jira/browse/LUCENENET-106

 

>             Project: Lucene.Net

 

>          Issue Type: Bug

 

>         Environment: .NET 2.0

 

>            Reporter: Anton K.

 

>            Assignee: Digy

 

>            Priority: Critical

 

>         Attachments: DIGY-FieldCacheImpl.patch, Digy.rar, 

> luceneSrc_memUsage.patch, Paches for v2.3.1.rar, WeakHashTable 

> v2.patch, WeakHashTable+FieldCacheImpl.rar, WeakReferences.rar

 

> 

 

> 

 

> readerCache Hashtable field (see FieldCacheImpl.cs) never releases some

hash items that have closed IndexReader object as a key. So a lot of Term

instances are never released.

 

> Java version of Lucene uses WeakHashMap and therefore doesn't have this

problem.

 

> This bug can be reproduced only when Sort functionality used during

search. 

 

> See following link for additional information.

 

> http://www.gossamer-threads.com/lists/lucene/java-user/55681

 

> Steps to reproduce:

 

> 1)Create index

 

> 2) Modify index by IndexWiter; Close IndexWriter

 

> 3) Use IndexSearcher for searching with Sort; Close InexSearcher

 

> 4) Go to step 2

 

> You'll get OutOfMemoryException after some time of running this algorithm.

 

 

 

-- 

 

This message is automatically generated by JIRA.

 

-

 

You can reply to this email to add a comment to the issue online.

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message