lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wyatt Barnett <wyatt.barn...@gmail.com>
Subject Re: Build & CI Considerations
Date Wed, 26 Jan 2011 23:47:07 GMT
On build systems -- I think MSBuild can definitely get you where you'd
want to be -- the triple-targeted version I posted had a very rough
MSBuild deployment script. The "alternative" build systems really
start to shine in places we don't need the help -- we don't need to do
complex packaging, build installers, manipulate database servers or
message queues. Said alternative build systems also generally call
MSBuild to do the actual building of the project so we need a bit of a
MSBuild build script anyhow.

NuGet is actually avaliable as the standalone command line tool -- see
the downloads page on codeplex
[http://nuget.codeplex.com/releases/view/57303]. It is meant for
publication side but it is a complete client near as I can tell. In
addition, you can still use NuGet to manage packages and other
dependencies and not require it be installed on the clients. When you
install a nuget package, the bits are pulled locally into the
solution, alls we need to do from there is to check said bits into the
solution and they can ride with everything else in SVN.

Speaking of dependencies, there are actually two -- sharpziplib is
required by the tests. This is actually a bit of an issue as it is
verboten to ship it with the project so it screws the CI pooch. Or at
least the alt.net/agilist one I subscribe to.

Is there a requirement to use Apache for CI? Or can we go out on our
own? I suspect they both can get us there, but if we are going to be
pulling off stunts like incorporating the sharpen (or whatever) port
into things we'll probably need a bit more control of the build agent
machine than apache can provide. I'm also fairly confident I can get a
complex, multi-step, multi-platform and perhaps even multi-build-agent
build process working in TeamCity. Has anyone here done such a thing
in Hudson?

I was having some success with .csproj files working all the way up
and down from VS2005 thru VS2010 without any conflicts with this
project. It seems that VS2008/2010 adds some stuff to the .csproj
file, but VS2005 walks right past that -- my suspicion is that pain
had to do with the more complex project types, all we've got is class
libraries which haven't changed since the days when men were men and
cars were cars. I tried everything I could to break the All that said,
is there anyone in this room who needs to *develop* on the solution in
VS2005? Or is the requirement "can build in a pure 2.0 environment and
be used in a pure 2.0 environment?" Good point on getting 4.0 into the
process as well and this is also something CI can cover easily.

My 2c on silverlight and compact framework clients would be we are
getting way ahead of ourselves -- we don't have a working conversion
process for the full framework yet, not sure if we want to open that
can of worms. Moreover, in terms of usage profiles, chances are major
search operations you'd want to do from a silverlight or CF app would
get pushed onto the cloud server somewhere. WP7 would be a more
interesting angle, but until that gets local storage abilities it is
kind of pointless to even consider.

On Wed, Jan 26, 2011 at 4:56 PM, Michael Herndon <mherndon@o19s.com> wrote:
> Robert,
>
>> .
>>
>
> I don't believe this is necessary. At least there were no requests
> for alternative build systems in the past.
>
> There may never be a need for the alternative building scripts, its was more
> of a curious question. I've seen a few projects on github use both albacore
> and psake. Maybe it was done in vain by the authors or possibly to attract
> people from the alt.net crowd.
>
>
> The Mono ecosystem is also preferring MSBuild/Visual Studio projects to
> other build systems.
>
> Are they using the xbuild or literally using msbuild and are they
> completely compatible ?  I haven't kept up with how far xbuild has come
> lately.  I know that mono develop and mono is still a little flakey with c#
> 4.0 and things like the expando object but that probably will not affect the
> current lucene builds.
>
> I believe we should rather provide our own NuGet package
> than using it for external dependencies. We currently
> have only one dependency: NUnit.  - Robert
>
> So I think is in more regards to dependencies in general. This has
> a possibility to increase, especially if we incorporate other libraries such
> as Moq for testing, or expand lucene functionality to include linq.
>
> Nuget isn't just for compiled assemblies but can also be used for importing
> classes such as the notorious internal Guard class.
>
> http://www.clariusconsulting.net/blogs/kzu/archive/2010/12/08/HowtocreatelightweightreusablesourcecodewithNuGet.aspx
>
>
> Regarding NuGet for dependencies.. I'd say we should not do that at
> this time. Primarily because, AFAIK, NuGet is an add-on for Visual
> Studio which can only be used in a paid-for version of 2010. You can't
> use it with the Express versions. You can't use it in MonoDevelop. So,
> we'd be raising the barrier of entry to only those who have a licensed
> version of Visual Studio 2010.  - Troy
>
> Troy,
>
> The core of Nuget itself does not actually have a ui or dependency on Visual
> Studio.  While nuget's default interfaces are currently limited to
> powershell or visual studio 2010, it is possible to create additional user
> interfaces for it.
>
> Its just a matter of time before there is a command line version of nuget.
>  There has already been patches made to the nuget core to support mono and
> there already hooks in nuget aimed at command line usage in the future.
>
> However, another thing to pay attention to is that once the binaries are
> added to the project, they become a part of the source tree.  Thus people
> can still build and code without using Nuget.
>
> While we do not need to run or rush to this right away, its something to be
> considered.
>
>
> CI
>
> As far as CI is concerned, I'm not particular on any technology. Hudson or
> cc.net, or insert another one here, is all fine with me. I would just like
> to see one become a part of the process especially when we are looking at
> using tools like Sharpen.
>
> Build targets: Real question is what do we want to target. I do
> think we can target different versions for development work versus
> runtime compatibility. EG, nothing is stopping us from making the
> dev/build environment VS 2010 while still targeting the 2.0 runtime.
> Personally, I'd vote aiming for 3.5/VS 2010, at least for the 3.0
> branch.  - Wyatt
>
>
> Wyatt,
>
> I know that its possible to load the same csproj files for both VS 2008 and
> 2010 (using different solutions).  These will generally load inside of Mono
> Develop as well.
>
> VS 2005 is where it gets tricky.
>
> Another thing we should look at is making sure can compile on both the 2x
> and 4x runtime.  I remember getting log4net to compile in 4x was a little
> bit of work.  I am sure other would like for this to build on other versions
> of the framework. I remember CF being one of those mentioned.
>
> I don't know if the list includes Silverlight or Micro. If it does we'll
> most likely need to look hard at the BCL as well as the Supported
> Interfaces, core types and methods.
>
> Possibly the Portable Tools Library could help with that in the future.
> http://blogs.msdn.com/b/bclteam/archive/2011/01/19/announcing-portable-library-tools-ctp-justin-van-patten.aspx
>
>
> - Michael.
>
>
>
>
>
> Below are the responses from another thread: Proposal Status, Initial
> Committors List, Contributors List
> ----------------------------------------
>
>
>
> Robert Jordan,
>
>
>
>
> Michael,
>
>
> On 26.01.2011 19:18, Michael Herndon wrote:
>
>> Should the project include build scripts for powershell, nant, albacore
>> scripts? (would this further entice developers to download the code and
>> build it)?  These also could be available as a separate download.
>>
>
> I don't believe this is necessary. At least there were no requests
> for alternative build systems in the past.
>
>
>
>> What would be the preferred way of working with the code base in mono ?
>>  Should we ensure the project loads and compiles in mono develop, etc.
>>
>
> The Mono ecosystem is also preferring MSBuild/Visual Studio projects
> to other build systems.
>
>
> would it be beneficial to add a .gitignore file for those avid git-svn
>> users?
>>
>
> Yes, but let's do it on demand.
>
>
>
>> should we start looking at nuget for external dependencies?
>>
>
> I believe we should rather provide our own NuGet package
> than using it for external dependencies. We currently
> have only one dependency: NUnit.
>
>
> ---------------------------------------------------------
> Apache has a number of CI options available... For details, see:
> http://ci.apache.org/
>
> I'd suggest Hudson.
>
> Regarding Mono support, I think providing a MonoDevelop compatible
> solution file, and testing builds in at least one standard Linux
> environment will be necessary. I'd suggest Ubuntu 10.4 or 10.10.
>
> Regarding NuGet for dependencies.. I'd say we should not do that at
> this time. Primarily because, AFAIK, NuGet is an add-on for Visual
> Studio which can only be used in a paid-for version of 2010. You can't
> use it with the Express versions. You can't use it in MonoDevelop. So,
> we'd be raising the barrier of entry to only those who have a licensed
> version of Visual Studio 2010.
>
> I think including .hgignore and/or .gitignore files are a smart move.
>
> Thanks,
> Troy
> ---------------------------------------
>
>
> On Wed, Jan 26, 2011 at 1:55 PM, Wyatt Barnett <wyatt.barnett@gmail.com>wrote:
>
>> Per Michael's suggestion I'm branching this off into a new thread.
>> I'll start by saying this is somewhere I think I could help alot --
>> I'm still a recovering liberal arts major so I can't claim to grok too
>> much of the underlying computer sciency bits. I've also spent as much
>> if not more time in IT managing infrastructure and processes than
>> building software. Finally, I've got some facilities access that could
>> be helpful in a pinch. Anyhow, here goes:
>>
>> 1) Build targets: Real question is what do we want to target. I do
>> think we can target different versions for development work versus
>> runtime compatibility. EG, nothing is stopping us from making the
>> dev/build environment VS 2010 while still targeting the 2.0 runtime.
>> Personally, I'd vote aiming for 3.5/VS 2010, at least for the 3.0
>> branch.
>>
>> 2) CI : oh hells yeah. My vision would be to setup something where the
>> automated conversion would be triggered by commits to the "stable"
>> branch of the java project. I think if we can construct this bit right
>> we can even really get down the road of automatically running all the
>> conversion options until we get it "right".
>>
>> Another angle to this is that, if it is the paid tool we need, we only
>> need the build agent to have a license -- it could easily just convert
>> the source and check it in where people could check it out and work on
>> it. Or something like that.
>>
>> But back to the mundane -- I've got alot of seat-time with TeamCity
>> and I'd be happy to setup a build server. I've got the facilities to
>> handle this personally. I think to really get it right we'll have to
>> make some significant changes in the project structure which brings me
>> to  .. .
>>
>> 3) What should be in SVN: I'm a big fan of everything and the kitchen
>> sink. Its 2011, server space and bandwidth are cheap. Philosophically,
>> checkout -> build should be a seamless, painless process on this sort
>> of library-style project presuming basic prerequisites. I like the
>> idea of including the .gitignore (and .hgignore) files. Once setup,
>> they really don't change unless you are really re-working stuff. And
>> given the setup with a restricted SVN "master" we are going to leave
>> lots of patches in the street if we can't be DCVS friendly.
>>
>> 4) Mono: my mac-filesystem-foo failed me, but I think the bits I
>> uploaded wrt issue https://issues.apache.org/jira/browse/LUCENENET-377
>> should compile on Mono using the build script included pointed at the
>> appropriate tool. In any case, this can easily be part of a teamcity
>> build process. We can even do it on linux if we'd like.
>>
>> Hope this helps.
>>
>

Mime
View raw message