lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Digy (JIRA)" <j...@apache.org>
Subject [Lucene.Net] [jira] [Commented] (LUCENENET-425) MMapDirectory implementation
Date Tue, 14 Jun 2011 18:32:47 GMT

    [ https://issues.apache.org/jira/browse/LUCENENET-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049322#comment-13049322
] 

Digy commented on LUCENENET-425:
--------------------------------

I would like to compare search speeds of "FSDirectory" and "MMapDirectory" (if possible, with
a big index)

1- IndexReader.Open( FSDirectory.Open(new System.IO.FileInfo(INDEX))  ,true );
2- IndexReader.Open( new MMapDirectory(new System.IO.FileInfo(INDEX)) ,true );

Something like
{code}
    public class TestSearchSpeed
    {
        string INDEX = @"Path to Index";  //
        string[] _words = new string[]{"","","","","","","","",""}; //Some words to search
        Directory _dir;

        public long TestFSDir()
        {
            _dir = FSDirectory.Open(new System.IO.FileInfo(INDEX));
            return Test();

        }

        public long TestMMapDir()
        {
            _dir = new MMapDirectory(new System.IO.FileInfo(INDEX));
            return Test();
        }

        long Test()
        {
            IndexReader reader = IndexReader.Open(_dir, true);
            Search(reader, "sometext");

            Stopwatch sw = new Stopwatch();
            sw.Start();

            for (int i = 0; i < 5; i++)
            {
                Parallel.For(0, 50, j =>
                    {
                        Search(reader, _words[j % _words.Length]);
                    }
                );
            }

            long dur = sw.ElapsedMilliseconds;
            sw.Stop();

            reader.Close();

            return dur;
        }

        void Search(IndexReader reader,string criteria)
        {
            IndexSearcher src = new IndexSearcher(reader);
            Query q = new QueryParser("field", new WhitespaceAnalyzer()).Parse(criteria);
            TopDocs hits =  src.Search(q, 100);
            for (int i = 0; i < hits.ScoreDocs.Length; i++)
            {
                Document doc = reader.Document(hits.ScoreDocs[i].doc);
                string s = doc.GetField("field").StringValue();
            }
        }
    }
{code}

DIGY


> MMapDirectory implementation
> ----------------------------
>
>                 Key: LUCENENET-425
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-425
>             Project: Lucene.Net
>          Issue Type: New Feature
>    Affects Versions: Lucene.Net 2.9.4g
>            Reporter: Digy
>            Priority: Trivial
>             Fix For: Lucene.Net 2.9.4g
>
>         Attachments: MMapDirectory.patch
>
>
> Since this is not a direct port of MMapDirectory.java, I'll put it under "Support" and
implement MMapDirectory as 
> {code}
> public class MMapDirectory:Lucene.Net.Support.MemoryMappedDirectory
> {
> }
> {code}
> If a Mem-Map can not be created(for ex, if the file is too big to fit in 32 bit address
range), it will default to FSDirectory.FSIndexInput
> In my tests, I didn't see any performance gain in 32bit environment and I consider it
as better then nothing. 
> I would be happy if someone could send test results on 64bit platform.
> DIGY

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message