lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir (JIRA)" <j...@apache.org>
Subject [Lucene.Net] [jira] [Updated] (LUCENENET-428) How to do that the results are displayed in the first original tokens and them with synonyms?
Date Tue, 28 Jun 2011 11:24:17 GMT

     [ https://issues.apache.org/jira/browse/LUCENENET-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vladimir updated LUCENENET-428:
-------------------------------

    Issue Type: Wish  (was: Task)

> How to do that the results are displayed in the first original tokens and them with synonyms?
> ---------------------------------------------------------------------------------------------
>
>                 Key: LUCENENET-428
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-428
>             Project: Lucene.Net
>          Issue Type: Wish
>          Components: Lucene.Net Core
>    Affects Versions: Lucene.Net 2.9.4
>         Environment: .net 4.0
>            Reporter: Vladimir
>
> How to do that the results are displayed in the first original tokens and them with synonyms?
> My Analyzer(part) :
> public override TokenStream TokenStream(string fieldName, TextReader reader)
>         {
>             TokenStream result = new StandardTokenizer(reader);
>             result = new LowerCaseFilter(result);
> 			result = new StopFilter(result, stoptable);
>             result = new SynonymFilter(result, synonymEngine); 
>             result = new ExtendedRussianStemFilter(result, charset);
>             return result;
>         }
> My SynonymFilter :
> internal class SynonymFilter : TokenFilter
>     {
>         private readonly ISynonymEngine engine;
>         private readonly Queue<Token> synonymTokenQueue
>             = new Queue<Token>();
>         public SynonymFilter(TokenStream tokenStream, ISynonymEngine engine) : base(tokenStream)
>         {
>             this.engine = engine;
>         }
>         public override Token Next()
>         {
>             if (synonymTokenQueue.Count > 0)
>             {
>                 return synonymTokenQueue.Dequeue();
>             }
>             
>             Token t = input.Next();
>             
>             if (t == null)
>                 return null;
>             if (t.Type() == "<SYNONYM>")
>                 return t;
>             
>             IEnumerable<string> synonyms = engine.GetSynonyms(t.TermText());
>             
>             if (synonyms == null)
>             {
>                 return t;
>             }
>             
>             foreach (string syn in synonyms)
>             {
>                 if (!t.TermText().Equals(syn))
>                 {
>                     var synToken = new Token(syn, t.StartOffset(),
>                                              t.EndOffset(), "<SYNONYM>");
>                     
>                     synToken.SetPositionIncrement(0);
>                     synonymTokenQueue.Enqueue(synToken);
>                 }
>             }
>             return t;
>         }
>     }
> Thanks!

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message