lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Matos <bruno.ma...@paradigmaxis.pt>
Subject Re: Search with Sort
Date Thu, 16 Oct 2008 10:23:40 GMT
Thanks for the reply I will trie to make a merge of both suggestions :)

--
Bruno Matos
 
Michael Garski wrote:
> Digy's patch will address the memory leakage in the field cache, but even though the
garbage collector will now free the memory associated with the sort values, you may still
end up with memory issues due to how the .NET runtime manages memory.
>
> For example, if you have an index with 250,000 items in it and perform queries on it
using a sort based integer value on one of the fields.  The field cache will create an array
of 250,000 integers to provide for fast access to the sort values.  As the array is a large
object (> 85,000 bytes) it will be allocated on the large object heap (LOH).  While during
a garbage collection memory is freed from the LOH, the LOH is not compacted like generations
0, 1, & 2.  If your application swaps over indexes without restarting, such as in a Windows
Service, the LOH could become fragmented over time.  This fragmentation can lead to an OutOfMemoryException
as the runtime is not able to allocate a contiguous chunk of memory for the array.
>
> There is a good MSDN article on the LOH and how to use windbg to peek at what's in the
managed heap: http://msdn.microsoft.com/en-us/magazine/cc534993.aspx
>
> Michael
>
>
>
> -----Original Message-----
> From: Digy [mailto:digydigy@gmail.com] 
> Sent: Wednesday, October 15, 2008 2:41 PM
> To: lucene-net-dev@incubator.apache.org
> Subject: RE: Search with Sort
>
> Yes, there is a MemoryLeakage-bug while searching+sorting in versions 2.1 &
> 2.3.1 
> and as far as I can reproduce the bug, I prepared two patches
> ("WeakHashTable+FieldCacheImpl.rar" for v2.1 and "Paches for v2.3.1.rar" for
> v2.3.1 (https://issues.apache.org/jira/browse/LUCENENET-106) ).
>
> Although there are still many complains about this issue, there isn't(or I
> don't have) any test case to track it down.
>
> DIGY.
>
>
>
> -----Original Message-----
> From: Bruno Matos [mailto:bruno.matos@paradigmaxis.pt] 
> Sent: Wednesday, October 15, 2008 8:34 PM
> To: lucene-net-dev@incubator.apache.org
> Subject: Search with Sort
>
> Hello,
>
> I'm with a problem with memory when I use a Sort. Can anyone help me 
> with this?
> I sow a lot of people posting about this, but I didn't sow any answers.
>
> Thank you.
>
>   


Mime
View raw message