lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shad Storhaug (Jira)" <>
Subject [jira] [Commented] (LUCENENET-610) Reduce locking in FieldCacheImpl::Cache::Get
Date Sat, 26 Oct 2019 20:08:00 GMT


Shad Storhaug commented on LUCENENET-610:

While having a thread safe collection would be ideal, that is not how it was implemented in
Lucene. However, I took a look at the implementation of {{WeakDictionary}} and it is dissimilar
to that of the {{java.util.WeakHashMap}} that was used in the OpenJDK. In particular the {{CleanIfNeeded()}}
method moves the entries from one dictionary instance to another every time it is called,
which is less than ideal.

According to [this StackOverflow answer|], it may be possible
to use a [ConditionalWeakTable<TKey,TValue>|]
as a replacement.

However, before it is attempted, it would be nice to have a reproduction case so it is possible
to see whether we are indeed solving the issue or making it worse. Could you please provide

> Reduce locking in FieldCacheImpl::Cache::Get
> --------------------------------------------
>                 Key: LUCENENET-610
>                 URL:
>             Project: Lucene.Net
>          Issue Type: Improvement
>          Components: Lucene.Net Core
>    Affects Versions: Lucene.Net 4.8.0
>            Reporter: Thomas Mathew
>            Priority: Minor
>              Labels: up-for-grabs
> We noticed a lot of contention in FieldCacheImpl::Cache::Get (our queries use a lot
of query time joins + sorting, so we hit the field cache a lot).
> We use a SearcherManager with warm-up queries to populate the field cache so we would
expect it to be initialized in most cases before we hit it for actual requests.
> The implementation seems to lock even for the happy path (when everything's already initialized).
This seems like a by-product of the choice of data structures (the underlying WeakDictionary,
WeakHashMap etc are not threadsafe) and so the locking is required in case the dictionary
gets resized.
> Ideally we could be using thread-safe data structures and only lock when initializing
the data.

This message was sent by Atlassian Jira

View raw message