lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shad Storhaug (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LUCENENET-603) ConcurrentMergeScheduler crashes the application if a transient error occurs
Date Wed, 10 Jul 2019 15:14:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENENET-603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16882171#comment-16882171
] 

Shad Storhaug edited comment on LUCENENET-603 at 7/10/19 3:13 PM:
------------------------------------------------------------------

Lucene.Net 4.8.0 has 4 different merge schedulers in the box:

1. ConcurrentMergeScheduler (default on .NET Standard 2.0 and .NET Framework)
 2. [TaskMergeScheduler|https://lucenenetdocs.azurewebsites.net/api/Lucene.Net/Lucene.Net.Index.TaskMergeScheduler.html]
(default on .NET Standard 1.6)
 3. [SerialMergeScheduler|https://lucenenetdocs.azurewebsites.net/api/Lucene.Net/Lucene.Net.Index.SerialMergeScheduler.html]
 4. [NoMergeScheduler|https://lucenenetdocs.azurewebsites.net/api/Lucene.Net/Lucene.Net.Index.NoMergeScheduler.html]

I suggest you try TaskMergeScheduler, which uses the System.Threading.Tasks.TaskScheduler
internally. Or try SerialMergeScheduler, which doesn't do merges in parallel (so it is not
possible to have a merge exception).
{code}
    IndexWriterConfig config = new IndexWriterConfig(LuceneVersion.LUCENE_48, analyzer);
    config.MergeScheduler = new Lucene.Net.Index.SerialMergeScheduler();
    IndexWriter w = new IndexWriter(directory, config);
{code}
I think (have yet to confirm) the main issue here is that in Java the exception is thrown
on the same thread that calls Merge(), but in .NET we are throwing it on the background thread.
I believe the exception must exist in order to make the IndexWriter function correctly, but
it needs to occur on the correct thread, and that is where we are falling down.


was (Author: nightowl888):
Lucene.Net 4.8.0 has 4 different merge schedulers in the box:

1. ConcurrentMergeScheduler (default)
2. [TaskMergeScheduler|https://lucenenetdocs.azurewebsites.net/api/Lucene.Net/Lucene.Net.Index.TaskMergeScheduler.html]
3. [SerialMergeScheduler|https://lucenenetdocs.azurewebsites.net/api/Lucene.Net/Lucene.Net.Index.SerialMergeScheduler.html]
4. [NoMergeScheduler|https://lucenenetdocs.azurewebsites.net/api/Lucene.Net/Lucene.Net.Index.NoMergeScheduler.html]

I suggest you try TaskMergeScheduler, which uses the System.Threading.Tasks.TaskScheduler
internally. Or try SerialMergeScheduler, which doesn't do merges in parallel (so it is not
possible to have a merge exception).


{code:c#}
    IndexWriterConfig config = new IndexWriterConfig(LuceneVersion.LUCENE_48, analyzer);
    config.MergeScheduler = new Lucene.Net.Index.SerialMergeScheduler();
    IndexWriter w = new IndexWriter(Directory, config);
{code}


I think (have yet to confirm) the main issue here is that in Java the exception is thrown
on the same thread that calls Merge(), but in .NET we are throwing it on the background thread.
I believe the exception must exist in order to make the IndexWriter function correctly, but
it needs to occur on the correct thread, and that is where we are falling down.

> ConcurrentMergeScheduler crashes the application if a transient error occurs
> ----------------------------------------------------------------------------
>
>                 Key: LUCENENET-603
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-603
>             Project: Lucene.Net
>          Issue Type: Improvement
>          Components: Lucene.Net Core
>    Affects Versions: Lucene.Net 3.0.3, Lucene.Net 4.8.0
>            Reporter: Laura Nash
>            Priority: Major
>              Labels: easyfix, newbie
>
> We are using Lucene.NET 3.0.3 within a larger application hosted in a Windows Service. 
The Lucene.NET use occurs in a background processing thread and is non-critical, it shouldn't
ever cause the Windows Service to crash.
> Currently if an error occurs (even a very transient error) within our implementation
of Lucene.Net.Store.BufferedIndexOutput FlushBuffer() then our Windows Service crashes.  
> This is because ConcurrentMergeScheduler.HandleMergeException throws an exception, even
though it is being called inside a background thread generated within the Lucene code and
so the exception thrown can never be caught and will always crash the application.  The code
and comments around this seem to suggest this is not expected to crash out (maybe due to the
port from java and java behaves differently from .NET for this?).
> Handling all errors inside our FlushBuffer implementation causes the file to become corrupted
as the flush is considered a success.
> I think this throw should be removed.  We have commented it out and this appears to
have had no detrimental affect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message