- periodically run tasks on the latest version of the code base,
- access the results through a web interface.
- build the project,
- run all non-regression test suites,
- run static analysis tools,
- present tool results,
- plot metric trends (line count, test count, warning count...),
- package the product.
wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo aptitude update
sudo aptitude install jenkins
sudo service jenkins start
A first build
Checking out the code
https://himeparser.svn.codeplex.com/svn/Hime/trunkNext select 'Emulate clean checkout by first deleting unversioned/ignored files, then 'svn update'' as the 'Check-out Strategy'. This strategy is safer than just use svn update as it will start each build with a clean repository, but largely more efficient than always checking out a full-blown fresh copy. Then save and launch a new build. While it is running, you can click on the build and check the Console Output to follow what is going on. You should see something like:
Started by user anonymous
Checking out a fresh workspace because there's no workspace at /var/lib/jenkins/jobs/Hime.Parser/workspace
Cleaning workspace /var/lib/jenkins/jobs/Hime.Parser/workspace
Checking out https://himeparser.svn.codeplex.com/svn/Hime/trunk
At revision 11108 Finished:
Tracking line count and installing plugins
Plugins are managed in the 'Update Center' which can be accessed from Jenkins entrance page by clicking on Manage Jenkins, then on Manage Plugins and at last on the 'Available' tab. For the time being, select Jenkins SLOCCount Plug-in from the list of all available plugins. (If there seems to be no plugin available, then you may need to check for updates manually. This can be done in the 'Advanced' tab). Install and restart Jenkins.
If you now go to the configuration page of the Hime.Parser, you may notice option 'Publish SLOCCount analysis results' which appeared among the 'Post-build Actions'. Tick it. In addition, we must obviously invoke sloccount. In the 'Build' section, add a build step of type 'Execute shell' with the following command:
sloccount --details --wide --duplicates VersionInfo.cs Lib.CentralDogma Exe.HimeDemo Lib.Redist Exe.HimeCC > sloccount.scAs you may well know, sloccount counts .xml files too. So it is a good idea to run it as one of the first build step. At this point, the workspace is still clear from any .xml result files that may be produced by other build steps.
Save the new configuration and launch a build. A new link SLOCCount, should have appeared in the left menu. If you click it you will get to a sortable table which displays all file sizes. If you build a second time (and refresh the page) then Jenkins plots the graph of total program size per build in the main page of the job. For now it should evidently show a flat trend.
Again back to the configuration page, this time to set up a whole project compilation. We add a shell script in the 'Build' section which uses mdtool to compile all projects:
mdtool build -p:Lib.Redist -c:ReleaseUntil now we launched all our builds manually. It would be more convenient to have Jenkins start a build every time the code changes. So we turn to the 'Build Triggers' section. We select 'Poll SCM' and have Jenkins poll the subversion repository every quarter-hour. Following the cron syntax:
mdtool build -p:Lib.CentralDogma -c:Releasemdtool build -p:Exe.HimeDemo -c:Releasemdtool build -p:Exe.HimeCC -c:Releasemdtool build -p:Tests -c:Release
*/15 * * * *If you want more details about the syntax, just click on the question mark next to the field. Jenkins offers a very comprehensive help. Some people prefer to build at regular intervals of time, such as every night. This is done by opting for 'Build periodically'. Also notice the 'Build after other projects are built' which is useful when your software product is composed of several small projects which depend on each other.
Save and launch a build. If you like, you may check the mdtool compiler progress in the Console Output. This time click on trend next to the 'Build History'. This page displays a 'Timeline' with all builds and plots each build duration. As you can check, the last build lasts much longer than the previous ones.
nunit-console -noshadow Tests/bin/Release/Tests.dll
Then we set up each build step from, as usual, the configuration page of the Hime.Parser job.
For compiler warnings, select 'Scan for compiler warnings' among the 'Post-build Actions'. There is no ad-hoc parser for mdtool outputs, so we choose 'MSBuild' to 'Scan console log'.
Then, for tasks, select 'Scan workspace for open tasks' and fill 'Files to scan' with a fileset representing all C# source files:
Also, fill 'Tasks tags' with TODO as a 'Normal priority'.**/*.cs
gendarme --config gendarmeConfig.xml --xml gendarmeResults.xml Exe.HimeCC/bin/Release/himecc.exe Exe.HimeDemo/bin/Release/HimeDemo.exe Lib.Redist/bin/Release/Hime.Redist.dll Tests/bin/Release/Tests.dll Lib.CentralDogma/bin/Release/Hime.CentralDogma.dll || exit 0
We are now ready to save, build (twice again to get trends) and check the result. We get three new trend graphs, all click-able. Clicking on 'Open Tasks Trend' brings a list of all open tasks in the project down to the source file. Similarly for 'Compiler Warnings Trend' and 'gendarme'. Note that the MSBuild parser sometimes fails to extract the correct file name or warning description from the compilation outptut. This is because of line breaks that mdtool inserts in order to keep column-width under a certain limit. I would really like to know how to avoid this bug...gendarmeResults.xml
mkdir himecc_latestThen tell Jenkins it should present the resulting archive on the main page of the job by clicking the post-build action 'Archive the artefacts'. Set himecc_latest.zip as the name of 'Files to archive'. Let us build one last time and check the downloadable artefact.
cp Exe.HimeCC/bin/Release/* himecc_latest
cp Lib.Redist/bin/Release/Hime.Redist.xml himecc_latest
zip -r himecc_latest.zip himecc_latest
By now, you must have gotten a fairly good grip on Jenkins. Going further is pretty easy and can be done by gradual improvements. For instance, having test coverage trends is pretty necessary and so are code duplication detection warnings....
Here are some plug-ins which may prove useful:
- Jenkins disk-usage plugin: lets you monitor the disk usage of jobs. In the configuration page, you can tell Jenkins to discard plug-ins when they are too old or too many.
- Downstream build view: necessary when your whole product in split in several small projects,
- Plot plugin: to display your own metrics (like the number of tasks in your project tracker for the current deadline),
- Green Balls: just better than blue,
- Safe Restart Plugin,
- Hudson Status Monitor,
- ChuckNorris Plugin: I will let you figure out this one by yourself.
I have found that the two most important trends to follow in any project are its line and test count. Test count needs to increase while line count should decrease relentlessly. Keeping these two trends steady, is the best recipe to attain (almost) flawless software. Doing so, you may realise how much trend is, in fact, destiny.
Here are some more metrics, that can not, to the best of my knowledge, be tracked in Jenkins and that would be great to have:
- More information from the bug-tracker, such as the time taken to complete each issue, the number of implemented and remaining features...
- The time elapsed between the discovery of two consecutive bugs,
- The ability to sort tasks (TODO comments) according to their age (i.e. the number of commits they have been in existence).
- Do you use a continuous integration server?
- Which one do you prefer?
- What about sonar?
- If you are using Jenkins, which plug-ins do you find helpful?
- How to order Jenkins with a script using its command-line interface,
- How to write your own plug-in for Jenkins, in particular to follow tests coverage statistics as computed by opencover.