Continuous Integration

[Pradeep Kadambar]

I have been going through Continuous Integration for assessing the suitability for small to medium scale projects [25 - 75]klocs. On the whole this looks as a very robust process for ensuring bug free solutions.

I somehow feel that this technique is an overkill for shorter/smaller span projects.

Is this th right assessment? Or there is something more to it? Please give me some perspective on this dilemma.

:roll:

[Ilja Preuss]

Originally posted by Pradeep KG:

I somehow feel that this technique is an overkill for shorter/smaller span projects.

I don't see why you would feel that. Could you please elaborate?

[Pradeep Kadambar]

Continuous Integration suggests a developer to build code on the development system, then run the test cases on it and check-in into the repository only if all the cases pass.

Once a check-in happens to the system a full system build will be run and test suite will run on the build subsequently.

Isn't it sufficient for smaller/medium sized projects have unit testing and integration testing done after coding?

If the time to market is also an important criteria how can we adopt CI?

[Ilja Preuss]

Originally posted by Pradeep Kadambar:
Continuous Integration suggests a developer to build code on the development system, then run the test cases on it and check-in into the repository only if all the cases pass.

And to repeat that cycle several times a day. Only then it's "continuous".

Isn't it sufficient for smaller/medium sized projects have unit testing and integration testing done after coding?

Depends on your definition of "sufficient". You certainly can do it - tons of teams are working that way. In my experience, using continuous integration works significantly better, even on small projects.

If the time to market is also an important criteria how can we adopt CI?

CI will *improve* your time to market, because problems are noticed much earlier, when they are still easy to fix.

[Jeanne Boyarsky]

Originally posted by Pradeep Kadambar:
Isn't it sufficient for smaller/medium sized projects have unit testing and integration testing done after coding?

What's this "coding" thing? Do you type without regard to whether it works?

I somehow feel that this technique is an overkill for shorter/smaller span projects ... If the time to market is also an important criteria how can we adopt CI?

I think it is still important for projects with short time frames to catch bugs as early as possible. On large projects, an early bug can affect other things causing delays. On small projects, the bug can directly cause delays. Do you want to tell the user/customer it is late because it doesn't work?

From personal experience, I've done continuous integration on a project where I was the only developer. I can't tell you how many things my teammate - the Ant build - found. Reasons:
1) Every so often I would forget to check in something. Better to notice that day than weeks later when the machine crashed and my work is gone.
2) I was more aware of logical break points where the code had to work enough to be distributed to "others." This stopped me from slipping into the mindset of being almost "done" with everything.
3) I couldn't forget to run the unit tests.
4) I knew everything was in CVS if I had to recreate my workspace or if we got another developer on the project.
5) I was always ready to deploy a working system.

[Pradeep Kadambar]

Hi Jeanne,

I am more or less in the same situation as you stated about your project. I googled for

continuous integration getting started

but not much of help. Time permitting can you share me the basic blueprints of your CI system.

[Ilja Preuss]

To get started, you don't need a system, especially if working alone.

All you do is checking in your known-to-be-working code every hour. Once you get good at that, or hit some problems, we can talk about the more advanced parts of the practice.

[Pradeep Kadambar]

For the sake of evaluating this process I am using it in my development.

But then if I were to scale this to a team of 15-20 then I would need some assistance from tools/systems.

I found Cruise Control is one such system.

Am I doing the right thing in directly setting up Cruise Control, VSS pair to do my web development/test autmation?

[Ilja Preuss]

Originally posted by Pradeep Kadambar:
But then if I were to scale this to a team of 15-20 then I would need some assistance from tools/systems.

Not really. What you need is an integration token and an integration machine. In fact there are serious drawbacks to using a tool like Cruise Control - see http://www.jamesshore.com/Blog/Continuous-Integration-is-an-Attitude.html

Am I doing the right thing in directly setting up Cruise Control, VSS pair to do my web development/test autmation?

I would advice against it. Try the manual approach first.

[Jeanne Boyarsky]

Pradeep,
I agree with Ilja on not needing Cruise Control yet. I find that it is best to ease into CI gradually.

As a single developer, you just have to say you are going to start doing CI. Try to commit daily, every few hours, every hour. They key is to make sure you always commit code where it compiles and the unit tests run. Then you increase commit frequency until you are doing CI. The only tools you need are a source control system and JUnit. If you don't have these, you aren't ready to be doing CI anyway.

As a team, there are some more steps. Our team's journey to CI went like this: (we had about 6 people when we started the journey and now have about 13)
1) Discuss why CI useful and better than current way of doing things. In our case, this meant identifying the "pain points" and showing how CI would help at a team meeting.
2) Start doing CI on a pilot project. Preferably one with a small # of developers that agree with the premise. I started making a point of checking in often tested code with all my commits on a project. Granted the project wasn't doing CI, but I was gaining experience with it to use in convincing people.
3) Repeat step 1 using examples from how step 2 went well. Propose doing a "mild" version of CI. In our case, I suggested "daily integration." The rule was that all code in the repository must compile and have unit testing passing overnight through 10 am. This allowed everyone to pull in clean code every morning. Pulling in code every day was required and releasing code every day (with a few exceptions) was required.
4) When pulling in code every morning, identify anyone who "forgot" to follow practice and remind them why we are doing this. When answering technical questions, verify person had pulled in code that morning.
5) When step 4 stabilized, introduce nightly build. This probably could have been done earlier like after step 3.
6) Repeat a variant of step 1 using examples from how smoothly step 5 is going. Now that we are all used to daily integration, propose integrating more often.
7) Repeat step 6 gradually tightening up team process rules each time. Keep doing this until the team is following true CI. Very exciting to get to this point!
8) [We haven't gotten to do this yet due to time constraints, but I feel we are ready.] Set up Cruise Control.

And of course the ongoing step: train all new team members on the team process and the reasons behind it.

[Jeanne Boyarsky]

So that post was a bit longer than I expected. But it was interesting thinking about our journey. It sounds like you are up to step 2 - pilot CI.

Concentrate more on process than tools. Getting 15-20 people to understand why CI is good is going to be more important coming up. The experiences you have while doing CI will be very valuable in this.

[Pradeep Kadambar]

On the commencement of this thread I was a skeptic of CI mainly due to the weight of opposition to this in my team.

Now I have a roadmap (sort of that) with a lot of help forn Jeanne. I am now quite convinced that CI may look daunting on the onset ... but the advantages are opaque at the surface.

[Ilja Preuss]

Yeah, great post, Jeanne!

Pradeep, you might find some more interesting information at http://www.jamesshore.com/Agile-Book/continuous_integration.html

Also, in the case you are interested in general advice for change agents, "Fearless Change" is a great book.

[Jeanne Boyarsky]

Thanks Ilja. I hadn't know about the James Shore link. We've been using Fowler's article for people who want to read about "why."

[Frank Carver]

All of this has made me reconsider. I guess by these terms I've been doing more continuous integaration than I thought.

Most of my paying work has been in what might be loosely termed "enterprise integration", where no piece of software or system stands alone, but has to work with other software (and data) out of my control. In this world "integration" is a term used to refer to deploying, configuring and connecting these disparate systems together. I've never used a CI server like cruise control or anthill, because I could never quite work out how it would manage this process. Checking out and compiling software is easy, deploying to a remote server, reconfiguring and restarting are much slower and more tricky, and actually connecting a new delivery to other systems (even in a "test" environment) usually requires manual intervention - emails, phone calls, instant messages and sometimes even plane travel and physical meetings. All of this seems way beyond what a CI server can accomplish.

An approach I used successfully at a recent contract might be thought of as "pair integration" (or more generally, perhaps, "peer integration"):

Each change would be developed by a loose pair each with their own development PC - sometimes sitting together at one machine, sometimes sitting separately for research, experiments, documentation, meetings etc. One development machine, however, is considered the "master". When the change is implemented and tested on the master machine, the other machine of the pair does an update/build/test. If/when that passes, the change is considered released. Both players are then free to join up with other team members and pursue the same approach.

As I believe I have mentioned before, I always run the full unit test suite automatically as part of the build process, so the deployable artefacts will never even be built if any of the unit tests fail. This goes a long way to making sure that the tests pass before check-in.

In the "peer integration" technique there is no central build machine, instead "spare capacity" in the pool of development machines is used. I hadn't thought of it as such, but it seems to meet the criteria for continuous integration in the links above.

Thoughts?

[Jeanne Boyarsky]

Frarnk,
I think that is CI. As far as I'm concerned if the code is in the repository and people release/pull in often, it is CI. The build is there to reinforce things. If other PCs are doing that, it's fine.

On our team, the build runs every night. During the day, we know something is broken, if someone pulls in code and starts shouting about it. This is pretty rare now anyway.

Goes back to Ilja's link about CI is an attitude, not a set of tools.

[Ilja Preuss]

Frank, thanks for your description of the interesting work you are doing. I'm going to suggest that you certainly were doing some form of CI, but that like always, it isn't that black and white and that I'm sure you could still improve those processes.

Originally posted by Frank Carver:
I guess by these terms I've been doing more continuous integaration than I thought.

Depends on what you thought how much CI you were doing...

To me, it sounds like you were doing CI for the software part. Sounds obvious to me that that's better than no CI at all, but not as good as including more of the other parts in the CI process.

Checking out and compiling software is easy

Ha! In my early days, I knew a manager who said that CI couldn't even work for software, because integrating just the classes of different developers so that they would compile and work as expected took two integration engineers two days! They simply couldn't afford to do that more often than every four weeks!

Sometimes, we are so used to the situation we are in, we don't recognize that what we are feeling is pain.

deploying to a remote server, reconfiguring and restarting are much slower and more tricky

Might be. But if the job were easy, where were the fun?

Ron Jeffries is known for saying that ideally, the output of every CI build is an installation CD - or the equivalent for your product. Some teams actually seem to be doing that.

actually connecting a new delivery to other systems (even in a "test" environment) usually requires manual intervention - emails, phone calls, instant messages and sometimes even plane travel and physical meetings. All of this seems way beyond what a CI server can accomplish.

CI is about optimizing the integration process, so that as much of it as possible can be done as often and reliably as possible, and about pushing those limits over what you initially thought *was* possible. Automation certainly is often part of the solution, but not really a hard and fast requirement.

A *CI server* is just a crutch for a very specific problem: long software build times. If you have other problems to solve - and every team has - you need to get creative. As I'm sure you will...

[Frank Carver]

Ron Jeffries is known for saying that ideally, the output of every CI build is an installation CD - or the equivalent for your product. Some teams actually seem to be doing that.

Oh we were doing that, the end result of the build script, run several times a day while we were doing active development, was a fully unit-tested deployable EAR file. This EAR file had already (as part of the automatic build process) been deployed to a local server, started and had some basic remote consistency checks run. The generated (and automatically version-numbered and time-stamped) EAR file could then be deployed and activated on the client system with a single script.

I love this kind of automation and take it as far as I can manage.

However, that's not really the point. Just to deploy anything to a live system with many dependencies and influences on other systems is a major political issue. Often mere developers are not even allowed to do it, requiring a documented change request to an operations team.

To test the interaction between our system and other collaborating systems required both the deployment of the new version, and the agreement and particpation of the people involved in managing the other systems and the product as a whole. Agreement on timings, test sequence and data, as well as broader issues of what is considered acceptable and unacceptable system behaviour, how to communicate results to all interested parties, and what to do if/when tests fail. And many of these other "legacy" systems have no kind of automated tests, and often no test "hooks" at all - so they need to be tested manually on a fully-working live system with real data. Not something which can be done lightly.

All of this is very different from developing a stand-alone application which can run independently. For such an application, producing a deployable image (be it an EAR, WAR, web start image, CD or whatever) can be seen as the end of the line. For the kind of stuff I mostly work with, that's just the starting point for the real testing.

[Ilja Preuss]

I see. I'm not sure whether what you call "integration" I would just call "release". Anway, doesn't really sound like fun, but you are certainly in a much better position to decide what can and cannot be done about it.

[Jeanne Boyarsky]

Originally posted by Frank Carver:
Just to deploy anything to a live system with many dependencies and influences on other systems is a major political issue. Often mere developers are not even allowed to do it, requiring a documented change request to an operations team.

Us too. We don't have a real development server that we control completely.

So it winds up being that every build generates a deployable artifact. That said, when we pull in the code every day, we are "deploying" it and running the intergration tests. So it feels like a reasonable compromise.

[Pradeep Kadambar]

Hi Guys,

With inputs from this thread I have setup basic infrastructure for CI.
An ANT based build to support:
* Compilation of code
* Run unit tests and generate test report.
* Generate code coverage with Cobertura (This I think is not very much needed).

Only when my basic test cases pass 100% I check-in to repository. This has helped me to detect some functional bugs without CI I would have detected only during module testing.

I have given this to my team mates. Hope this will make our development process more robust over time.

[Ilja Preuss]

[Jeanne Boyarsky]

Great Pradeep. That's good to hear!