lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "TJ Kolev" <tjko...@gmail.com>
Subject TestHugeRamFile
Date Mon, 12 Jan 2009 20:56:39 GMT
Greetings,

I thought I'd run the NUnit tests, and TestHugeFile() (in
/Test/Store/TestHugeRamFile.cs) fails:

System.OutOfMemoryException: Exception of type
'System.OutOfMemoryException' was thrown.
at Lucene.Net.Store.RAMFile.AddBuffer(Int32 size) in
D:\svn_lucene\src\Lucene.Net\Store\RAMFile.cs: line 85
at Lucene.Net.Store.RAMOutputStream.SwitchCurrentBuffer() in
D:\svn_lucene\src\Lucene.Net\Store\RAMOutputStream.cs: line 152
at Lucene.Net.Store.RAMOutputStream.WriteBytes(Byte[] b, Int32 offset,
Int32 len) in D:\svn_lucene\src\Lucene.Net\Store\RAMOutputStream.cs:
line 136
at Lucene.Net.Store.TestHugeRamFile.TestHugeFile() in
TestHugeRamFile.cs: line 83

The test tries to write out 2 * int.MaxValue bytes to RAMOutputStream.
I can't make it work for int.MaxValue bytes. It does not even work for
0.9. It works for 0.8 of int.MaxValue:

   private static readonly long MAX_VALUE = (long) (0.8 *
System.Int32.MaxValue);

I am running on .Net 3.5, Windows Server 2003, 4 GB RAM.

So, what are the expectations for this test? In my case, I was
expecting it to complete, since I had more memory than the needed 2
GB. And yet it failed. Then I found this article:
http://technet.microsoft.com/en-us/library/bb124810.aspx and I set the
/3GB switch. Same results. I don't know what is limiting the CLR from
allocating more memory.

Also, the test harness has its own class DenseRAMFile : RAMFile, with
the only purpose of doing its own NewBuffer(int size). However, the
base method virtual byte[] NewBuffer(int size) is not used, so
DenseRAMFile doesn't get its shot either. I am suspecting that the
intention was to have line 85 in /Lucene.Net/Store/RAMFile.cs in
AddBuffer(int size) look like this:

    byte[] buffer = NewBuffer(size); // instead of = new byte[size];

If this change is made, then TestHugeFile() passes, but as the
comments say, DenseRAMFile reuses a byte[] buffer, and does not
allocate huge amounts of memory, so it is not a huge ram test at
all...

tjk :)

Mime
View raw message