lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <e...@ehatchersolutions.com>
Subject Re: Remote searches with Lucene
Date Sun, 27 Aug 2006 23:36:00 GMT
I'm sure I have commit privileges also, and would be happy to apply  
an "svn add" for the initial dump and clean patches for a while, as  
long as the code is ASF licensed and George doesn't mind.  It'd be  
better if he did so to vet it, as I'm not a .NET programmer.

	Erik


On Aug 27, 2006, at 5:22 PM, Jeff Rodenburg wrote:

> That's likely our only option for now.  I believe George would need  
> to do
> the posting; I'm not aware of anyone else with commit access.
> As long as the turnaround is rapid and it doesn't present an admin  
> burden,
> I'm ok with it.
>
> -- j
>
>
> On 8/27/06, Erik Hatcher <erik@ehatchersolutions.com> wrote:
>>
>> Jeff,
>>
>> I haven't heard anything back about your account request yet.  (but
>> the officialness of the vote is in question - so it may be a while)
>>
>> What about posting a .zip file to JIRA and having George or someone
>> commit it on your behalf and submit patches from then on?
>>
>>         Erik
>>
>>
>> On Aug 26, 2006, at 10:23 PM, Jeff Rodenburg wrote:
>>
>> > As promised, an update to the list.
>> >
>> > I have code ready for delivery, if I can get svn access to the  
>> contrib
>> > section.  A request has been made for this but it's going nowhere,
>> > so I'm
>> > going to find another place to host the files.
>> >
>> > There's quite a bit of documentation behind this so I'm working
>> > diligently
>> > to explain how this works.  If anyone has a place to hold the code
>> > until the
>> > uber-powers at apache decide to grant me access, we would greatly
>> > appreciate
>> > the assistance.
>> >
>> > cheers,
>> > jeff r.
>> >
>> >
>> > On 8/23/06, Jeff Rodenburg <jeff.rodenburg@gmail.com> wrote:
>> >>
>> >> Just a follow-up to everyone on this topic.  I received a lot of
>> >> offlist
>> >> mail about this, so this message has a rather wide distribution.
>> >>
>> >> I'm in process of modifying the code for our distributed search
>> >> components
>> >> so that they're generic enough for general usage and public
>> >> consumption.
>> >> This is taking a little of my time, but nonetheless I expect to
>> >> complete it
>> >> soon.
>> >>
>> >> As for distributing the code, it will be located in the contrib
>> >> portion of
>> >> the Lucene.Net repository at apache.org.  There is some  
>> logistic work
>> >> involved, but ideally this is moving forward.
>> >>
>> >> As soon as I have more information to relay, I'll pass it along to
>> >> the
>> >> list.
>> >>
>> >> cheers,
>> >> jeff r.
>> >>
>> >>
>> >>
>> >>
>> >> On 8/21/06, Jeff Rodenburg < jeff.rodenburg@gmail.com> wrote:
>> >> >
>> >> > Hello all -
>> >> >
>> >> > I've been watching this thread to follow the direction and
>> >> thought I
>> >> > might be able to offer some assistance.  I run a search system
>> >> that involves
>> >> > 4 separate search servers -- 3 serving search objects via
>> >> RemoteSearchable,
>> >> > and a 4th that serves in an index updating role.
>> >> >
>> >> > The codebase for Lucene.Net provides all the library routines
>> >> one needs
>> >> > to provide distributed search capabilities, but does not provide
>> >> facilities
>> >> > for distributed search operation -- nor should it.  The ideas
>> >> presented here
>> >> > are certainly possible; I've implemented a working operation
>> >> without
>> >> > requiring the changes described here.  I'm confident in our
>> >> implementation;
>> >> > for the calendar year, our uptime/availability of search
>> >> services is
>> >> > 99.99%.  Our only outage was related to network hardware,  
>> otherwise
>> >> > we're sitting solid at 100%.
>> >> >
>> >> > I've been authorized to provide our operational code for
>> >> distributed
>> >> > search under Lucene.Net to the community at large.  Some of the
>> >> code is
>> >> > customized to our operation, but for the most part it's rather
>> >> generic.  We
>> >> > started the project under Lucene v1.4.3, but the operational  
>> aspect
>> >> > still applies under v1.9.
>> >> >
>> >> > The system consists of a LuceneServer, which provides  
>> searchability
>> >> > against indexes as defined in XML configuration files.  In
>> >> addition, an
>> >> > IndexUpdateServer provides master index updating, master/slave
>> >> index
>> >> > replication and automated index maintenance.  Integration with
>> >> our web site
>> >> > ensures the index stays available, updated and current.  There's
>> >> a great
>> >> > deal of applied knowledge and learned behavior of many of the
>> >> underlying
>> >> > sub-system components that distributed search under Lucene.Net
>> >> makes use
>> >> > of -- .Net remoting, garbage collection, etc.
>> >> >
>> >> > If anyone has interest, please reply.  Contributing this code
>> >> requires a
>> >> > little cleanup of our customization work, so my response may  
>> not be
>> >> > immediate but I would make efforts to release the code in short
>> >> order.
>> >> >
>> >> > thanks,
>> >> > jeff r.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On 8/19/06, Robert Boulanger < robert@boulanger.at> wrote:
>> >> > >
>> >> > > Hi Elena, hi Rest,
>> >> > >
>> >> > > > Dear All,
>> >> > > >
>> >> > > > The application I am working on is intended to make use 

>> of the
>> >> > > > distributed search capabilities of the Lucene library. While
>> >> trying
>> >> > > to
>> >> > > > work with the Lucene's RemoteSearchable class, I faced some
>> >> problems
>> >> > >
>> >> > > > cased by the current Lucene implementation. In following
>> >> I'll try to
>> >> > > > describe them, as well as the possible ways of their
>> >> solution, I
>> >> > > > identified. The most important question for me is, if these
>> >> changes
>> >> > > > have a chance to be integrated in the coming Lucene
>> >> versions, such
>> >> > > > that remote searches would really become feasible. I would
>> >> > > appreciate
>> >> > > > any feedback.
>> >> > >
>> >> > > Same problem for me and I found some more issues which I  
>> explain
>> >> > > below:
>> >> > >
>> >> > > >
>> >> > > > The first problem concerns the construction of the
>> >> RemoteSearchable
>> >> > > > object. .Net framework allows for both, server and client
>> >> activation
>> >> > > > models of the remote objects. Currently, RemoteSearchable
 
>> class
>> >> > > > possesses only one constructor that requires knowledge of
a
>> >> local
>> >> > > > Searchable object:
>> >> > > >
>> >> > > > public RemoteSearchable(Lucene.Net.Search.Searchable local)
>> >> > > >
>> >> > > I just added a new constructor to RemoteSearchable
>> >> > > public RemoteSearchable(): base()
>> >> > > {
>> >> > > this.local = this.local;
>> >> > > }
>> >> > >
>> >> > > not the fine method but for me it works so far.
>> >> > >
>> >> > > > Since this "local" object is located on the server,
>> >> knowledge of the
>> >> > >
>> >> > > > server's index paths is needed for its creation. However,
>> >> there are
>> >> > > at
>> >> > > > least some scenarios where only the server, but not the 

>> client,
>> >> > > knows
>> >> > > > where the indexes are stored on the server side. I think
 
>> this
>> >> > > problem
>> >> > > > could be solved by extending RemoteSearchable class with
a
>> >> standard
>> >> > > > constructor that reads the names of the indexes to be
>> >> published out
>> >> > > of
>> >> > > > a configuration file on the server side.
>> >> > > >
>> >> > > My "Server" now implements a Class which inherits directly
>> >> from Remote
>> >> > >
>> >> > > Searchable.
>> >> > > in the parameterless constructor there I read the server sided
>> >> > > configfile which contains the index location , create a new
>> >> > > IndexReader
>> >> > > and pass it as Argument to MyBase.New()
>> >> > > See sample below.
>> >> > >
>> >> > > > 2. Bug in Term construction
>> >> > > [snip]
>> >> > >
>> >> > > This whole chapter was very useful and I can commit everything
>> >> works
>> >> > > fine from there on.
>> >> > >
>> >> > > But there is still a bug in FieldDocSortedHitQueue line 130
>> >> and below:
>> >> > > I figured out that the castings are not working when the
>> >> system is
>> >> > > running in a non english globalization context.
>> >> > > The String in docAFields[i] which might be for example
>> >> 1.345678 is
>> >> > > casted to 1345678.0 since the decimal sign is misinterpreted
>> >> in German
>> >> > >
>> >> > > systems as it seems.
>> >> > > So the casting results in an overflow.
>> >> > >
>> >> > > So I changed it as follows:
>> >> > >
>> >> > > case SortField.SCORE:
>> >> > > float r1 = (float)Convert.ToSingle(docA.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo );
>> >> > > float r2 = (float)Convert.ToSingle(docA.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo);
>> >> > > if (r1 > r2)
>> >> > > c = - 1;
>> >> > > if (r1 < r2)
>> >> > > c = 1;
>> >> > > break;
>> >> > >
>> >> > > Same in line 172 and 174:
>> >> > >
>> >> > > float f1 = (float)Convert.ToSingle(docA.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo);
>> >> > > //UPGRADE_TODO: The equivalent in .NET for method
>> >> > > 'java.lang.Float.floatValue' may return a different value.
>> >> > >
>> >> > > "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!
>> >> DefaultContextWindowIndex'&keyword='jlca1043'"
>> >> > > float f2 = (float)Convert.ToSingle(docB.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo );
>> >> > >
>> >> > >
>> >> > >
>> >> > > A tiny Client Server Solution now looks like this (Here in
>> >> VB.NET)
>> >> > > SERVER:
>> >> > > Public Class RemoteQuery
>> >> > > Inherits RemoteSearchable
>> >> > > Public Sub New()
>> >> > > MyBase.New(New IndexSearcher("C:\lucene\index"))
>> >> > > End Sub
>> >> > > Public Sub New(ByVal local As Searchable)
>> >> > > MyBase.New(local)
>> >> > > End Sub
>> >> > >
>> >> > > End Class
>> >> > >
>> >> > > Module Module1
>> >> > > Public Sub Main(ByVal args As System.String())
>> >> > > Dim chnl As New HttpChannel(8888)
>> >> > > ChannelServices.RegisterChannel (chnl, False)
>> >> > > Dim indexName As System.String = Nothing
>> >> > > RemotingConfiguration.RegisterWellKnownServiceType
>> >> > > (GetType(RemoteQuery),
>> >> > > "Searchable", WellKnownObjectMode.Singleton)
>> >> > > System.Console.ReadLine()
>> >> > > End Sub
>> >> > > End Module
>> >> > > CLIENT
>> >> > > Sub Main()
>> >> > > Dim searchables As Lucene.Net.Search.Searchable() = New
>> >> > > Lucene.Net.Search.Searchable() {LookupRemote()}
>> >> > > Dim searcher As Searcher = New MultiSearcher(searchables)
>> >> > > Dim sort As New Lucene.Net.Search.Sort
>> >> > > sort.SetSort(Lucene.Net.Search.SortField.FIELD_SCORE)
>> >> > > Dim query As Query = QueryParser.Parse("Harry", "body", New
>> >> > > StandardAnalyzer())
>> >> > > Dim result As Hits = searcher.Search (query, sort)
>> >> > > End Sub
>> >> > > Private Function LookupRemote() As  
>> Lucene.Net.Search.Searchable
>> >> > > Return CType(Activator.GetObject(GetType
>> >> (Lucene.Net.Search.Searchable
>> >> > > ),
>> >> > > " http://192.168.8.7:8888/Searchable"),
>> >> Lucene.Net.Search.Searchable)
>> >> > > End Function
>> >> > >
>> >> > > Hope this helps you and anybody else how has problems with
>> >> > > remotesearch
>> >> > > so far.
>> >> > >
>> >> > > BTW: this all refers Version 1.9rc1
>> >> > >
>> >> > > --Robert Boulanger
>> >> > >
>> >> >
>> >> >
>> >>
>>
>>


Mime
View raw message