lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rezgar Cadro (JIRA)" <>
Subject [jira] Created: (LUCENENET-358) CloseableThreadLocal memory leak in LocalDataStoreSlot (with workaround)
Date Tue, 27 Apr 2010 12:17:32 GMT
CloseableThreadLocal memory leak  in LocalDataStoreSlot (with workaround)

                 Key: LUCENENET-358
             Project: Lucene.Net
          Issue Type: Bug
         Environment: Microsoft WIndows Server 2008 Enterprise x64. SP2.
.NET Framework 4.0
            Reporter: Rezgar Cadro
            Priority: Critical
         Attachments: CloseableThreadLocal MemoryLeak.patch

Recently we have been suffering from a severe memory leak when executing intense open/close
operations on IndexSearcher and IndexModifier. 
Memory profiling showed that memory is being held by LocalDataStore[] objects.

After some digging, the root of the problem has been found in CloseableThreadLocal class:

private System.LocalDataStoreSlot t = System.Threading.Thread.AllocateDataSlot();

What we see is that every instantiated object of CloseableThreadLocal causes new data slot
allocation performed for every thread. 
Thread.AllocateDataSlot() does not simply allocate a new slot, replacing an old one, but enlarging
an existing buffer in-thread, appending data to the end of internal LocalDataStore[] collection,
which  causes a severe memory leak .

As long as "t" variable is instantiated on every object creation, and (in current class implementation)
every object is used by a single thread, replacing "private System.LocalDataStoreSlot t =
System.Threading.Thread.AllocateDataSlot();" with simple "private object dataSlot;" and removing
"hardRefs" Dictionary solves the problem and prevents memory leak. 

We have tried to implement the expected behavior by using [ThreadStatic] attribute instead
of LocalDataStoreSlot, but the attempt failed because of unexpected exceptions being thrown.

Patch can be found at Lucene.Net repository under 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message