lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Aroush" <geo...@aroush.net>
Subject RE: Remote searches with Lucene
Date Mon, 28 Aug 2006 13:43:26 GMT
I have no problem adding the code to the SVN under contribute.

Jeff: Just ZIP up the code, and submit it to the mailing list and I can do
the rest.  Make sure, like Erik said, to have ASF licensed in every CS file
you submit.  Also, it would be nice to have an NUnit test written for it.
This will serve as a validation for the code as well as an example on how to
use the code.

Regards,

-- George

-----Original Message-----
From: Erik Hatcher [mailto:erik@ehatchersolutions.com] 
Sent: Sunday, August 27, 2006 7:36 PM
To: lucene-net-dev@incubator.apache.org
Subject: Re: Remote searches with Lucene

I'm sure I have commit privileges also, and would be happy to apply an "svn
add" for the initial dump and clean patches for a while, as long as the code
is ASF licensed and George doesn't mind.  It'd be better if he did so to vet
it, as I'm not a .NET programmer.

	Erik


On Aug 27, 2006, at 5:22 PM, Jeff Rodenburg wrote:

> That's likely our only option for now.  I believe George would need  
> to do
> the posting; I'm not aware of anyone else with commit access.
> As long as the turnaround is rapid and it doesn't present an admin  
> burden,
> I'm ok with it.
>
> -- j
>
>
> On 8/27/06, Erik Hatcher <erik@ehatchersolutions.com> wrote:
>>
>> Jeff,
>>
>> I haven't heard anything back about your account request yet.  (but
>> the officialness of the vote is in question - so it may be a while)
>>
>> What about posting a .zip file to JIRA and having George or someone
>> commit it on your behalf and submit patches from then on?
>>
>>         Erik
>>
>>
>> On Aug 26, 2006, at 10:23 PM, Jeff Rodenburg wrote:
>>
>> > As promised, an update to the list.
>> >
>> > I have code ready for delivery, if I can get svn access to the  
>> contrib
>> > section.  A request has been made for this but it's going nowhere,
>> > so I'm
>> > going to find another place to host the files.
>> >
>> > There's quite a bit of documentation behind this so I'm working
>> > diligently
>> > to explain how this works.  If anyone has a place to hold the code
>> > until the
>> > uber-powers at apache decide to grant me access, we would greatly
>> > appreciate
>> > the assistance.
>> >
>> > cheers,
>> > jeff r.
>> >
>> >
>> > On 8/23/06, Jeff Rodenburg <jeff.rodenburg@gmail.com> wrote:
>> >>
>> >> Just a follow-up to everyone on this topic.  I received a lot of
>> >> offlist
>> >> mail about this, so this message has a rather wide distribution.
>> >>
>> >> I'm in process of modifying the code for our distributed search
>> >> components
>> >> so that they're generic enough for general usage and public
>> >> consumption.
>> >> This is taking a little of my time, but nonetheless I expect to
>> >> complete it
>> >> soon.
>> >>
>> >> As for distributing the code, it will be located in the contrib
>> >> portion of
>> >> the Lucene.Net repository at apache.org.  There is some  
>> logistic work
>> >> involved, but ideally this is moving forward.
>> >>
>> >> As soon as I have more information to relay, I'll pass it along to
>> >> the
>> >> list.
>> >>
>> >> cheers,
>> >> jeff r.
>> >>
>> >>
>> >>
>> >>
>> >> On 8/21/06, Jeff Rodenburg < jeff.rodenburg@gmail.com> wrote:
>> >> >
>> >> > Hello all -
>> >> >
>> >> > I've been watching this thread to follow the direction and
>> >> thought I
>> >> > might be able to offer some assistance.  I run a search system
>> >> that involves
>> >> > 4 separate search servers -- 3 serving search objects via
>> >> RemoteSearchable,
>> >> > and a 4th that serves in an index updating role.
>> >> >
>> >> > The codebase for Lucene.Net provides all the library routines
>> >> one needs
>> >> > to provide distributed search capabilities, but does not provide
>> >> facilities
>> >> > for distributed search operation -- nor should it.  The ideas
>> >> presented here
>> >> > are certainly possible; I've implemented a working operation
>> >> without
>> >> > requiring the changes described here.  I'm confident in our
>> >> implementation;
>> >> > for the calendar year, our uptime/availability of search
>> >> services is
>> >> > 99.99%.  Our only outage was related to network hardware,  
>> otherwise
>> >> > we're sitting solid at 100%.
>> >> >
>> >> > I've been authorized to provide our operational code for
>> >> distributed
>> >> > search under Lucene.Net to the community at large.  Some of the
>> >> code is
>> >> > customized to our operation, but for the most part it's rather
>> >> generic.  We
>> >> > started the project under Lucene v1.4.3, but the operational  
>> aspect
>> >> > still applies under v1.9.
>> >> >
>> >> > The system consists of a LuceneServer, which provides  
>> searchability
>> >> > against indexes as defined in XML configuration files.  In
>> >> addition, an
>> >> > IndexUpdateServer provides master index updating, master/slave
>> >> index
>> >> > replication and automated index maintenance.  Integration with
>> >> our web site
>> >> > ensures the index stays available, updated and current.  There's
>> >> a great
>> >> > deal of applied knowledge and learned behavior of many of the
>> >> underlying
>> >> > sub-system components that distributed search under Lucene.Net
>> >> makes use
>> >> > of -- .Net remoting, garbage collection, etc.
>> >> >
>> >> > If anyone has interest, please reply.  Contributing this code
>> >> requires a
>> >> > little cleanup of our customization work, so my response may  
>> not be
>> >> > immediate but I would make efforts to release the code in short
>> >> order.
>> >> >
>> >> > thanks,
>> >> > jeff r.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On 8/19/06, Robert Boulanger < robert@boulanger.at> wrote:
>> >> > >
>> >> > > Hi Elena, hi Rest,
>> >> > >
>> >> > > > Dear All,
>> >> > > >
>> >> > > > The application I am working on is intended to make use 

>> of the
>> >> > > > distributed search capabilities of the Lucene library. While
>> >> trying
>> >> > > to
>> >> > > > work with the Lucene's RemoteSearchable class, I faced some
>> >> problems
>> >> > >
>> >> > > > cased by the current Lucene implementation. In following
>> >> I'll try to
>> >> > > > describe them, as well as the possible ways of their
>> >> solution, I
>> >> > > > identified. The most important question for me is, if these
>> >> changes
>> >> > > > have a chance to be integrated in the coming Lucene
>> >> versions, such
>> >> > > > that remote searches would really become feasible. I would
>> >> > > appreciate
>> >> > > > any feedback.
>> >> > >
>> >> > > Same problem for me and I found some more issues which I  
>> explain
>> >> > > below:
>> >> > >
>> >> > > >
>> >> > > > The first problem concerns the construction of the
>> >> RemoteSearchable
>> >> > > > object. .Net framework allows for both, server and client
>> >> activation
>> >> > > > models of the remote objects. Currently, RemoteSearchable
 
>> class
>> >> > > > possesses only one constructor that requires knowledge of
a
>> >> local
>> >> > > > Searchable object:
>> >> > > >
>> >> > > > public RemoteSearchable(Lucene.Net.Search.Searchable local)
>> >> > > >
>> >> > > I just added a new constructor to RemoteSearchable
>> >> > > public RemoteSearchable(): base()
>> >> > > {
>> >> > > this.local = this.local;
>> >> > > }
>> >> > >
>> >> > > not the fine method but for me it works so far.
>> >> > >
>> >> > > > Since this "local" object is located on the server,
>> >> knowledge of the
>> >> > >
>> >> > > > server's index paths is needed for its creation. However,
>> >> there are
>> >> > > at
>> >> > > > least some scenarios where only the server, but not the 

>> client,
>> >> > > knows
>> >> > > > where the indexes are stored on the server side. I think
 
>> this
>> >> > > problem
>> >> > > > could be solved by extending RemoteSearchable class with
a
>> >> standard
>> >> > > > constructor that reads the names of the indexes to be
>> >> published out
>> >> > > of
>> >> > > > a configuration file on the server side.
>> >> > > >
>> >> > > My "Server" now implements a Class which inherits directly
>> >> from Remote
>> >> > >
>> >> > > Searchable.
>> >> > > in the parameterless constructor there I read the server sided
>> >> > > configfile which contains the index location , create a new
>> >> > > IndexReader
>> >> > > and pass it as Argument to MyBase.New()
>> >> > > See sample below.
>> >> > >
>> >> > > > 2. Bug in Term construction
>> >> > > [snip]
>> >> > >
>> >> > > This whole chapter was very useful and I can commit everything
>> >> works
>> >> > > fine from there on.
>> >> > >
>> >> > > But there is still a bug in FieldDocSortedHitQueue line 130
>> >> and below:
>> >> > > I figured out that the castings are not working when the
>> >> system is
>> >> > > running in a non english globalization context.
>> >> > > The String in docAFields[i] which might be for example
>> >> 1.345678 is
>> >> > > casted to 1345678.0 since the decimal sign is misinterpreted
>> >> in German
>> >> > >
>> >> > > systems as it seems.
>> >> > > So the casting results in an overflow.
>> >> > >
>> >> > > So I changed it as follows:
>> >> > >
>> >> > > case SortField.SCORE:
>> >> > > float r1 = (float)Convert.ToSingle(docA.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo );
>> >> > > float r2 = (float)Convert.ToSingle(docA.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo);
>> >> > > if (r1 > r2)
>> >> > > c = - 1;
>> >> > > if (r1 < r2)
>> >> > > c = 1;
>> >> > > break;
>> >> > >
>> >> > > Same in line 172 and 174:
>> >> > >
>> >> > > float f1 = (float)Convert.ToSingle(docA.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo);
>> >> > > //UPGRADE_TODO: The equivalent in .NET for method
>> >> > > 'java.lang.Float.floatValue' may return a different value.
>> >> > >
>> >> > > "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!
>> >> DefaultContextWindowIndex'&keyword='jlca1043'"
>> >> > > float f2 = (float)Convert.ToSingle(docB.fields[i],
>> >> > > System.Globalization.NumberFormatInfo.InvariantInfo );
>> >> > >
>> >> > >
>> >> > >
>> >> > > A tiny Client Server Solution now looks like this (Here in
>> >> VB.NET)
>> >> > > SERVER:
>> >> > > Public Class RemoteQuery
>> >> > > Inherits RemoteSearchable
>> >> > > Public Sub New()
>> >> > > MyBase.New(New IndexSearcher("C:\lucene\index"))
>> >> > > End Sub
>> >> > > Public Sub New(ByVal local As Searchable)
>> >> > > MyBase.New(local)
>> >> > > End Sub
>> >> > >
>> >> > > End Class
>> >> > >
>> >> > > Module Module1
>> >> > > Public Sub Main(ByVal args As System.String())
>> >> > > Dim chnl As New HttpChannel(8888)
>> >> > > ChannelServices.RegisterChannel (chnl, False)
>> >> > > Dim indexName As System.String = Nothing
>> >> > > RemotingConfiguration.RegisterWellKnownServiceType
>> >> > > (GetType(RemoteQuery),
>> >> > > "Searchable", WellKnownObjectMode.Singleton)
>> >> > > System.Console.ReadLine()
>> >> > > End Sub
>> >> > > End Module
>> >> > > CLIENT
>> >> > > Sub Main()
>> >> > > Dim searchables As Lucene.Net.Search.Searchable() = New
>> >> > > Lucene.Net.Search.Searchable() {LookupRemote()}
>> >> > > Dim searcher As Searcher = New MultiSearcher(searchables)
>> >> > > Dim sort As New Lucene.Net.Search.Sort
>> >> > > sort.SetSort(Lucene.Net.Search.SortField.FIELD_SCORE)
>> >> > > Dim query As Query = QueryParser.Parse("Harry", "body", New
>> >> > > StandardAnalyzer())
>> >> > > Dim result As Hits = searcher.Search (query, sort)
>> >> > > End Sub
>> >> > > Private Function LookupRemote() As  
>> Lucene.Net.Search.Searchable
>> >> > > Return CType(Activator.GetObject(GetType
>> >> (Lucene.Net.Search.Searchable
>> >> > > ),
>> >> > > " http://192.168.8.7:8888/Searchable"),
>> >> Lucene.Net.Search.Searchable)
>> >> > > End Function
>> >> > >
>> >> > > Hope this helps you and anybody else how has problems with
>> >> > > remotesearch
>> >> > > so far.
>> >> > >
>> >> > > BTW: this all refers Version 1.9rc1
>> >> > >
>> >> > > --Robert Boulanger
>> >> > >
>> >> >
>> >> >
>> >>
>>
>>


Mime
View raw message