I've worked in an environment with continuous integration, and I like the basic principle of making sure the build is always working. The problem is that as our project grows large, build times are hovering around 20 minutes. Is that abnormal for a CI process? Obviously we should aim to keep the build as fast as possible, but what do you see as an acceptable maximum for the CI build time?
The problem is that as our project grows large, build times are hovering around 20 minutes. Is that abnormal for a CI process?...what do you see as an acceptable maximum for the CI build time?
In my experience 20 minutes is not that abnormal, but you will also begin to lose the benefits of CI the longer the build takes. The longer it takes, the less people want to run it, the less people want to run it, the less they commit their code, the less they commit their code, the less feedback you get...and on and on. It's all about the technique. There are many techniques we discuss in improving the performance of long-running builds, particularly in chapter 4 (Building Software at Every Change), such as: - Using a dedicated integration build machine - Increasing the integration build machine's hardware capacity - Building system components separately
Many of the teams I've worked on or observed that use CI tend not to move onto their next task until the integration build is successful (a good practice). If your build starts taking a long time then there will be more waiting. It really depends on your team environment, but I think a good rule of thumb is what Kent Beck suggests in "XP2", which is keeping your integration build to less than 10 minutes or so. Another suggestion I have is to establish a build pipeline ("staged" builds) in which you run a "commit" build - which is a real lightweight build - followed by a secondary build based on the success of this commit build that may take much longer and run things like component and functional tests against a database, etc. The lightweight commit build may include only compilation and unit tests (that don't touch a database) so that it is extra fast and you still learn a majority of the most common errors such as not committing a source file to the repository, failed unit tests and configuration errors.
Co-author of <a href="http://www.amazon.com/gp/product/0321336380/" target="_blank" rel="nofollow">Continuous Integration: Improving Software Quality and Reducing Risk </a> <br />(Addison-Wesley Martin Fowler Signature Series, 2007). Companion website for the book is <a href="http://www.integratebutton.com/" target="_blank" rel="nofollow">IntegrateButton.com</a>
Joined: Apr 07, 2007
Thanks for those thoughts. We actually do a lot of those things and are striving to get the build time down to less than 10 minutes, so I think we're on the right track. The only thing I'm not familiar with there is what you mean by building system components separately. Could you elaborate on that?
On another note, is multi-threading a viable option to help increase build times? I think JUnit has some deficiencies when it comes to multi-threading and I'm not overly familiar with any of the extensions or 3rd party tools that allow for multi-threaded unit tests (and I could see how there could be problems when you have tests that access a Singleton that wasn't designed for multi-threaded access). I know this gets a little off-topic, but I'd be interested to hear your comments on this as well.
Thanks for taking the time to come to this forum and answer our questions.
Jeff- check out parallel-junit-- it is a slick library that facilitates running JUnit tests in a threaded manner. It follows the TestSuite paradigm and is quite simple. We've seen dramatic decreases in testing times associated with builds by using this library. Threading tests does introduce issues, especially if your tests are database-centric; plus, you'll need a rather large suite of tests to overcome the cost of creating and managing threads.
Certified City Slicker
Joined: Jul 17, 2007
The only thing I'm not familiar with there is what you mean by building system components separately. Could you elaborate on that?
If you are able to segment parts of your project into discrete components with well-defined interfaces that can be built separately you can significantly reduce your build times. Let me know if you need further elaboration on this.
I totally agree that it's vital to keep the build time under ten minutes. Two years ago, our build time was two hours, which was horrible. Using all kinds of tricks, we are now down to an average of five minutes, which is quite nice.
The soul is dyed the color of its thoughts. Think only on those things that are in line with your principles and can bear the light of day. The content of your character is your choice. Day by day, what you do is who you become. Your integrity is your destiny - it is the light that guides your way. - Heraclitus
Joined: Apr 07, 2007
I get what you're saying about having being able to build your components separately. You could parallelize the building and speed things up that way. I appreciate your input, thanks.