lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENENET-565) Port Lucene.Net.Replicator
Date Wed, 09 Aug 2017 10:51:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENENET-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119708#comment-16119708
] 

ASF GitHub Bot commented on LUCENENET-565:
------------------------------------------

Github user NightOwl888 commented on the issue:

    https://github.com/apache/lucenenet/pull/209
  
    Jens, thanks for the feedback. 
    
    While I generally agree with your points, we are unfortunately obligated to work within
the confines of the Apache organization, which comes with its own set of conditions. For one,
there is a [formal release process](http://www.apache.org/legal/release-policy.html) we need
to abide by (including a vote process that takes a minimum of 3 days per release), and I am
not sure what is involved in setting up another Apache repo (nor am I sure I really want to
go down that rabbit hole).
    
    That said, we could follow the [Lucene project's](https://github.com/apache/lucene-solr)
example with their separate release cycles of Lucene and SOLR: put everything into one repo
and use separate tag names for each product. At least they started out that way - now they
are releasing both products at the same time.
    
    It probably makes sense, though. If we have a release vote on both product A and product
B and product B depends on product A, and product A doesn't pass the release vote, then product
B will be stalled anyway.
    
    Of course, there is an alternative - build these integration packages outside of Apache's
umbrella. But I am not sure what Apache's policy is in this regard (anyone?). Given the current
situation with Spatial4n and LuceneNetDemo being on Itamar's personal account and no ability
to push to them without being given special permission (by Itamar), and the fact that most
of the projects on NuGet that depend on Lucene.Net are either dead or dying, it feels like
this idea could go wrong as well. Sure, it is feasible for a huge team like Microsoft to do
this, but trying to pull this off with a small team probably isn't very realistic.
    
    Not to mention, platform support. With the release of .NET Standard 2.0 looming, we are
looking at a sweeping update that will hit every project. And what of .NET Standard 2.5 when
that is eventually released? As much as Microsoft has promised by creating .NET Standard,
we still have some branching between platforms we need to maintain, although it isn't as bad
as if we were trying to use a portable library. Now, multiply that update by the number of
repos that we would have to update (each with their own ever diverging build script)...and
we have a huge pile of work.
    
    But I will take this into consideration. Perhaps after a few releases that are synced
up, we can put together a new template in TeamCity so each of these integration packages can
have their own release cycle. But since the team of active PMC members is so small, finding
the 3 votes necessary for every one of those separate releases might not be very realistic.
Our last beta vote had exactly the minimum 3 votes that were required (including my own).
    
    > Also, your already on the edge of making some decisions that would be bad for larger
scale projects... E.g. making a Singleton LuceneWriter (registered in a IoC Container or not)
wouldn't that limit you to only having a single index?... that would seem like an odd direction
to consider within this context as the Replicator directly supports shards... (aka multiple
indexes?)
    
    Actually, I was planning to make an API to be able to register multiple IndexWriters or
IndexReaders at application startup as singleton. Something along the lines of...
    
    ```
    services.AddSearch().WithIndex(...).WithIndex(...)
    ```
    
    But the fact remains that the recommendation is to use a singleton IndexWriter (or Reader)
per index, meaning if we didn't provide the tools do it with a simple straightforward API,
everyone would end up having to do the research how to do it and write the same boilerplate
code. And a lot of them would assume they could create an IndexReader instance on each request
(or register it with the wrong lifetime) and find out just how poorly that performs.
    
    I need to explore exactly how this can work in a way that doesn't block people from doing
it their own way, providing all of the overloads that allow pre-built implementations to be
passed in as an alternative to using fluent builders. But regardless of how you slice it,
setting up an index reader or writer belongs in the application startup phase and nowhere
else, although you need to access those instances at runtime (again best solved with DI).
Microsoft is providing a standard way for doing this in their latest UI frameworks and it
would be a real shame not to take advantage of this fact.
    
    > As a final note, I won't be able to find time to take the Lucene.Net.Replicator.AspNetCore
project much further, my goal was to focus on Lucene.Net.Replicator it self and provide a
meaningful port of that that could be integrated into AspNetCore (which we haven't even adopted
yet), AspNet4x, Nancy or similar frameworks with relative ease...
    
    > Which path you take with Lucene.Net.Replicator.AspNetCore is up to you... But I don't
feel right about building anything two involved for a framework/platform I don't even use
and therefore don't know all the ins and outs of...
    
    No problem. Your contribution is much appreciated.
    
    > To me it was an example integration that could help me find points in the Lucene.Net.Replicator
which might cause difficulty when integrating or straight out felt wrong and there are a few
things about the ReplicatorService that annoys me in that regard, but there is a broader picture
as the HttpReplicator aligns to that same theme and then there is your desire to stay compatible
with Java versions, which is questionably because they choose to actually serialize Exception
objects, if java clients expect any meaning of that then we are not compatible at all as all
Exceptions would end up as "Unknown Exceptions" in a Java client... and that goes for the
other way around as well...
    
    I see. You are right that it might not entirely interoperate with the Java version (at
least not if there are exceptions), but at least we have explored that option and came to
this conclusion. 
    
    > I do have a small super simplistic and extremely crude DEMO AspNetCore app that indexes
and publishes data and then a Console client that pulls the replicas which works. I am trying
to use this to weed out annoyances.
    
    Cool. Just let me know when you are ready to hand over the reigns. I am still exploring
whether I can make API docs part of the upcoming beta release.
    
    I would also like to tinker with getting replicator working in a demo setup. Perhaps you
can put the demo project in a separate repo? When you are ready, that is.



> Port Lucene.Net.Replicator
> --------------------------
>
>                 Key: LUCENENET-565
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-565
>             Project: Lucene.Net
>          Issue Type: Task
>          Components: Lucene.Net.Replicator
>    Affects Versions: Lucene.Net 4.8.0
>            Reporter: Shad Storhaug
>            Priority: Minor
>              Labels: features
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message