This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Agile and Other Processes and the fly likes CI with ClearCase Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Engineering » Agile and Other Processes
Bookmark "CI with ClearCase" Watch "CI with ClearCase" New topic
Author

CI with ClearCase

Joe Van Dyk
Greenhorn

Joined: Aug 02, 2005
Posts: 5
We have a fairly large distributed simulation application that's written in a gazillion different langauges (from Ada to Fortran to C to Ruby, I counted 10 different languages some time ago). We're using ClearCase for version control.

To run the simulation, you need to first enter a ClearCase view and build it on a certain platform (either Redhat 8,9 or EL 3). Then, you copy a "runtime directory structure" from that ClearCase view to a few "runtime machines" (where the simulation will be run on). The runtime directory structure contains Makefiles that will download simulation binaries from the view into the runtime structure. If a new build is needed, you enter the view, build, then repopulate the runtime directory files.

Then the user can start a graphical application that 1) lets the user modify certain environment options that the simulation applications use and 2) starts the simulation on the runtime machines.

The configuration of the runtime machines is currently a manual process. A custom kernel needs to be installed, certain Gnome libraries need to upgraded, Ruby is upgraded, kernel shared memory limits are raised, nvidia drivers installed, all types of awful things. This configuration stuff is not kept in ClearCase.

I'm trying to come up with a way to integrate Continuous Integration and automation into this environment.

One obvious thing is to put the configuration of the runtime machines into version control, and then somehow update the runtime machines when a new build is released. I suppose we could try to have one directory on the runtime machine (/Simulation) that contained all the files that are necessary to run the sim (X, Ruby, Gnome, the scenario executables/datafiles, etc) instead of installing the external vendor libraries into /usr.

Anyone done anything similar? Tricks and tips are greatly appreciated.

Thanks,
Joe
Jared Richardson
author
Ranch Hand

Joined: Jun 22, 2005
Posts: 113
I can't say that I've done anything ~quite~ like that... but that won't stop from crazy speculation!

First build, then test.

First automate, second automatic.

So first, automate. Get the builds scripted on each machine. It sounds like you've got components of the build running on more than one box. Get each one scripted and building independently. I'd use "native" build scripts for each language and then wrap them in Ant, but that's just because I know Ant.

Second, automatic. Set up a tiered Continuous Integration system. Build one runs on machine one. On machine two, your CI software watches machine one's CI logs. When he sees a good build of the first step, he starts his part of the build. We do something similar at SAS to do multimachine testing. CruiseControl can do this sort of log watching out of the box.

At this point, you have a CI build system, so then you think about testing. Follow the same pattern. First, automate some of your testing. You mention a GUI. Java? HTML? C++? Find a good
testing framework that can drive your application.

Once you have a few tests automated, then use another instance of the CI system, watching the logs of the CI system running on the machine you want to test on, waiting for a good build. When it sees one, it runs your tests.

Don't underestimate the value of a very basic test in this scenario. Getting the system in place and ensuring that the basic system deploys and runs is extremely valuable.


Check out <b>Ship It! A Practical Guide to Shipping Software</b><br /> <br /><a href="http://www.pragmaticprogrammer.com/titles/prj/" target="_blank" rel="nofollow">http://www.pragmaticprogrammer.com/titles/prj/</a>
Joe Van Dyk
Greenhorn

Joined: Aug 02, 2005
Posts: 5
Thanks for the response! Great book, btw.

A build can be done on one machine (per target platform). The entire thing can be built from that one machine. We use Clearmake to build everything (and it can be amazingly done with one command).

Then, the executables are copied from that machine (actually the ClearCase view) to runtime machines using a Makefile that exists on the runtime machine. When the view gets updated with a new build, a user can go to a runtime machine, run 'make' and get the updated binaries. (A "runtime machine" is just a machine where the simulation applications get run. Tends to have special hardware/graphics/network capabilities.)

The GUI that is used for starting all the applications on the different machines in written in Ruby. There's a daemon that runs on each runtime machine that takes start/kill application commands from that GUI. (Each application takes a bunch of environment variables and the GUI helps the user change those options easily).

The thing I'm wrestling with now is how to get the configuration of the runtime machine stored in ClearCase, and how to update the runtime machine with each deployment/test cycle. From a stock Redhat machine, there's a bunch of kernel modifications that need to be made, kernel modules that need installing, extra applications (Ruby, gnome, nvidia drivers) that need installation... and the details of that depend on the version of the simulation that you are running.
Joe Van Dyk
Greenhorn

Joined: Aug 02, 2005
Posts: 5
Oh, and it's probably not clear from the original post. But the simulation is comprised of many different applications that are on each runtime machine that work in unision. (if that clears anything up)

Generally, each runtime machine has a copy of the entire simulation runtime directory (that contains the executables and data files) and the GUI tells each runtime machine what program to start and what environment options each program should start with.
Jared Richardson
author
Ranch Hand

Joined: Jun 22, 2005
Posts: 113
Thanks for the response! Great book, btw.

Thanks!

The thing I'm wrestling with now is how to get the configuration of the runtime machine stored in ClearCase, and how to update the runtime machine with each deployment/test cycle. From a stock Redhat machine, there's a bunch of kernel modifications that need to be made, kernel modules that need installing, extra applications (Ruby, gnome, nvidia drivers) that need installation... and the details of that depend on the version of the simulation that you are running.


So you have nodes that require changes as extensive as kernel mods and vidio driver changes.

How about installing everything by hand and then backing up the entire box. Save an image of the box to the network. Have your automation test system either (1) initiate a re-stage of the box the appropriate image from a network image server or (2) load the image into one of the PC simulators (Virtual PC, etc) (or would that invalidate your testing?).

I've never actually done an automated restage of a box, but it sounds like fun. And probably the only way to do what you want.

Even if you can't automate it, you could reboot the box over the network to the image you want, run your tests, then restage to something else.

I once had a need for different testing environments at a startup that didn't have much hardware. In that situation we bought removable hard drives and swapped them out as needed.

Not very automatic though unless you build a robot to swap out the hard drives... you'd get a good Slashdot write up and an article in Make too!
Joe Van Dyk
Greenhorn

Joined: Aug 02, 2005
Posts: 5
Automated restaging sounds good. I'd be really surprised if the simulation would work from a virtual machine, but it's something to look in to. It's supposed to be a real-time simulation, so I'm not sure if the virtual machine would handle that correctly. But I'm not that familiar with the internals.

Now... how to do the automated restaging... Hm. I have no idea off the top of my head. I'll have to do some research on it.
Joe Van Dyk
Greenhorn

Joined: Aug 02, 2005
Posts: 5
Also, has anyone used CruiseControl with ClearCase? Do they play well with each other?
Jared Richardson
author
Ranch Hand

Joined: Jun 22, 2005
Posts: 113
Originally posted by Joe Van Dyk:
Also, has anyone used CruiseControl with ClearCase? Do they play well with each other?


I haven't used it personally, but yes, the ClearCase support is built into CC.

ClearCase in CC
 
wood burning stoves
 
subject: CI with ClearCase
 
Similar Threads
Command to find out Linux desktop version?
Eclipse and package statement
Webspere 345 exam questions
000-341 sample test questions
IBM Exam 157 - Questions and Answers (Sample Exam)