lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Currens (JIRA)" <j...@apache.org>
Subject [Lucene.Net] [jira] [Issue Comment Edited] (LUCENENET-425) MMapDirectory implementation
Date Tue, 14 Jun 2011 19:38:47 GMT

    [ https://issues.apache.org/jira/browse/LUCENENET-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049363#comment-13049363
] 

Christopher Currens edited comment on LUCENENET-425 at 6/14/11 7:37 PM:
------------------------------------------------------------------------

On a 1.18GB index of only one text field:
{panel}
MMap Dir: 74
FS Dir: 34
-----------
MMap Dir: 79
FS Dir: 29
Press any key to continue . . .
{panel}
Same index, order of search changed:
{panel}
FS Dir: 25
MMap Dir: 110
-----------
FS Dir: 112
MMap Dir: 78
Press any key to continue . . .
{panel}

On a 241MB index of text and binary data (used a field selector to only get the text field):
{panel}
FS Dir: 151
MMap Dir: 679
-----------
FS Dir: 130
MMap Dir: 627
Press any key to continue . . .
{panel}
Same index, order of search changed:
{panel}
MMap Dir: 867
FS Dir: 134
-----------
MMap Dir: 600
FS Dir: 135
Press any key to continue . . .
{panel}

The second index, while smaller, requires a lot more seeking, due to the amount of fields
per doc (anywhere from 15-30 fields per doc).  Seems it would be a more realistic index to
search.

EDIT: I ran each one twice per execution of the program, I switched the execution of the indexes,
in case of any buffering done by the runtime or os.

This was on 32-bit Windows-XP Pro SP3

      was (Author: ccurrens):
    On a 1.18GB index of only one text field:
{panel}
MMap Dir: 74
FS Dir: 34
-----------
MMap Dir: 79
FS Dir: 29
Press any key to continue . . .
{panel}
Same index, order of search changed:
{panel}
FS Dir: 25
MMap Dir: 110
-----------
FS Dir: 112
MMap Dir: 78
Press any key to continue . . .
{panel}

On a 241MB index of text and binary data (used a field selector to only get the text field):
{panel}
FS Dir: 151
MMap Dir: 679
-----------
FS Dir: 130
MMap Dir: 627
Press any key to continue . . .
{panel}
Same index, order of search changed:
{panel}
MMap Dir: 867
FS Dir: 134
-----------
MMap Dir: 600
FS Dir: 135
Press any key to continue . . .
{panel}

The second index, while smaller, requires a lot more seeking, due to the amount of fields
per doc (anywhere from 15-30 fields per doc).  Seems it would be a more realistic index to
search.
  
> MMapDirectory implementation
> ----------------------------
>
>                 Key: LUCENENET-425
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-425
>             Project: Lucene.Net
>          Issue Type: New Feature
>    Affects Versions: Lucene.Net 2.9.4g
>            Reporter: Digy
>            Priority: Trivial
>             Fix For: Lucene.Net 2.9.4g
>
>         Attachments: MMapDirectory.patch
>
>
> Since this is not a direct port of MMapDirectory.java, I'll put it under "Support" and
implement MMapDirectory as 
> {code}
> public class MMapDirectory:Lucene.Net.Support.MemoryMappedDirectory
> {
> }
> {code}
> If a Mem-Map can not be created(for ex, if the file is too big to fit in 32 bit address
range), it will default to FSDirectory.FSIndexInput
> In my tests, I didn't see any performance gain in 32bit environment and I consider it
as better then nothing. 
> I would be happy if someone could send test results on 64bit platform.
> DIGY

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message