From lucene-net-dev-return-957-apmail-incubator-lucene-net-dev-archive=incubator.apache.org@incubator.apache.org Tue Aug 28 19:57:17 2007 Return-Path: Delivered-To: apmail-incubator-lucene-net-dev-archive@locus.apache.org Received: (qmail 68823 invoked from network); 28 Aug 2007 19:57:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 28 Aug 2007 19:57:17 -0000 Received: (qmail 30965 invoked by uid 500); 28 Aug 2007 19:57:12 -0000 Delivered-To: apmail-incubator-lucene-net-dev-archive@incubator.apache.org Received: (qmail 30934 invoked by uid 500); 28 Aug 2007 19:57:12 -0000 Mailing-List: contact lucene-net-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: lucene-net-dev@incubator.apache.org Delivered-To: mailing list lucene-net-dev@incubator.apache.org Received: (qmail 30925 invoked by uid 99); 28 Aug 2007 19:57:12 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Aug 2007 12:57:12 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [63.115.58.195] (HELO mail.inmagic.com) (63.115.58.195) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Aug 2007 19:58:02 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C7E9AD.905EF1FC" Subject: Unexpected behavior in Lucene locking code Date: Tue, 28 Aug 2007 15:56:45 -0400 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Unexpected behavior in Lucene locking code Thread-Index: AcfprZARAEB4yaIIQDOQugHzj+l7uQ== From: "Ross Faneuf" To: X-Virus-Checked: Checked by ClamAV on apache.org ------_=_NextPart_001_01C7E9AD.905EF1FC Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable We are using Lucene.Net-2.0 final-004, and have encountered intermittent errors in the locking code when running under a heavy load in a multi-process, multi-thread environment. This is a partial call stack from one of the errors: =20 Type : System.IO.IOException, mscorlib, Version=3D2.0.0.0, Culture=3Dneutral, PublicKeyToken=3Db77a5c561934e089 Message : The process cannot access the file 'C:\...\system\index\lock\lucene-82b3c5ad936757ece94a8582252daee7-commit .lock' because it is being used by another process. Source : mscorlib Help link :=20 Data : System.Collections.ListDictionaryInternal TargetSite : Void WinIOError(Int32, System.String) Stack Trace : at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.File.Delete(String path) at Lucene.Net.Store.FSDirectory.AnonymousClassLock.Release() at Lucene.Net.Store.Lock.With.Run() at Lucene.Net.Index.IndexWriter.MergeSegments(Int32 minSegment, Int32 end) at Lucene.Net.Index.IndexWriter.FlushRamSegments() at Lucene.Net.Index.IndexWriter.Close() =20 This error is from a process which is updating a Lucene index. =20 Our code which calls IndexWriter is: =20 IndexWriter indexWriter =3D null; ReaderWriterLock _indexReaderLock.AcquireWriterLock(Timeout.Infinite); try { indexWriter =3D GetIndexWriter(); Document doc =3D = LuceneIndexHelper.GetDocument(ct, ctData); indexWriter.AddDocument(doc); } catch (Exception e) { ... } finally { if (indexWriter !=3D null) { indexWriter.Close(); indexWriter =3D null; } _indexReaderLock.ReleaseWriterLock(); } =20 We note, though, the IndexReader code also acquires the same lock - for instance in: =20 private static IndexReader Open(Directory directory, bool closeDirectory) { lock (directory) { // in- & inter-process sync return (IndexReader) new AnonymousClassWith(directory, closeDirectory, directory.MakeLock(IndexWriter.COMMIT_LOCK_NAME), IndexWriter.COMMIT_LOCK_TIMEOUT).Run(); } } =20 Is the purpose of this lock to prevent an index update while a Read is in progress? Is it possible that under the right load and complexity conditions, this lock attempt interferes with lock handling in IndexWriter? I note that IndexWriter operation actually take this lock multiple times, using the various AnonymousClassWith objects, and it appears possible for the IndexReader lock to be acquired in between the IndexWriter locked operations. Any insight into the origins of this problem would be helpful. =20 Ross Faneuf; rfaneuf@inmagic.com =20 =20 ------_=_NextPart_001_01C7E9AD.905EF1FC--