lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENENET-488) Can't open IndexReader, get OutOFMemory Exception
Date Tue, 01 May 2012 12:53:49 GMT

    [ https://issues.apache.org/jira/browse/LUCENENET-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265773#comment-13265773
] 

Steven commented on LUCENENET-488:
----------------------------------

Hi Simon, thanks very much, set the option to 10 (have no idea what that means but it works)
the reader open in about 4 seconds but the search is still hugely impressive (300ms to search
through 1bn records and return the first 10).
I will try to build a native 64bit version on the server itself (my development box is only
32 which might be the problem) and let you know how I get on.
Thanks again, can't believe you guys do this for free, I pay millions for products that aren't
any where near as good!
                
> Can't open IndexReader, get OutOFMemory Exception
> -------------------------------------------------
>
>                 Key: LUCENENET-488
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-488
>             Project: Lucene.Net
>          Issue Type: Bug
>          Components: Lucene.Net Core
>    Affects Versions: Lucene.Net 2.9.4g
>         Environment: Windows server 2008R2
>            Reporter: Steven
>
> Have build a large database with ~1Bn records (2 items per document) it has size 200GB
on disk. I managed to write the indexe by chunking into 100,000 blocks as I ended up with
some threading issues (another bug submission). Anyway the index is built but I can't open
it and get a memory exception (process explorer gets to 1.5GB allocated before it dies but
not sure how reliable that is, but do know there is plenty more RAM left on the box).
> Stack trace below:
> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was
>  thrown.
>    at Lucene.Net.Index.TermInfosReader..ctor(Directory dir, String seg, FieldInf
> os fis, Int32 readBufferSize, Int32 indexDivisor)
>    at Lucene.Net.Index.SegmentReader.CoreReaders..ctor(SegmentReader origInstanc
> e, Directory dir, SegmentInfo si, Int32 readBufferSize, Int32 termsIndexDivisor)
>    at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, Directory dir, Segmen
> tInfo si, Int32 readBufferSize, Boolean doOpenStores, Int32 termInfosIndexDiviso
> r)
>    at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, SegmentInfo si, Int32
>  termInfosIndexDivisor)
>    at Lucene.Net.Index.DirectoryReader..ctor(Directory directory, SegmentInfos s
> is, IndexDeletionPolicy deletionPolicy, Boolean readOnly, Int32 termInfosIndexDi
> visor)
>    at Lucene.Net.Index.DirectoryReader.<>c__DisplayClass1.<Open>b__0(String
segm
> entFileName)
>    at Lucene.Net.Index.SegmentInfos.FindSegmentsFile.Run(IndexCommit commit)
>    at Lucene.Net.Index.DirectoryReader.Open(Directory directory, IndexDeletionPo
> licy deletionPolicy, IndexCommit commit, Boolean readOnly, Int32 termInfosIndexD
> ivisor)
>    at Lucene.Net.Index.IndexReader.Open(String path, Boolean readOnly)
>    at Lucene.Net.Demo.SearchFiles.Main(String[] args)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message