A friend of mine convinced me that I should at least give continuous integration servers a try to see if they might streamline my development process. I thought that a CI server might be a wonderful way to aid in communication with my clients. They would be able to see that status of work as it is being developed. Knowing that the customer is able to see not only the final version of the code, but every intermediate version and it status in testing can only improve the quality of the released product.

I had a few requirements for any system that I would use. First, I wanted it to be easy to administer. I'm not a system administrator, I'm a programmer, and I don't want to spend my time keeping things working. I want to spend my time programming. Any system I used needed to be deployable to Tomcat. I just moved from Geronimo to Tomcat for my testing server and I didn't want to have to change it again. It needed to support my source code management system, GIT. And lastly it needed to easily support use on any development project from the entire Java ecosystem, most importantly today, it needed to be usable with Groovy on Grails.

After looking around on the web for a good open source product that met my requirements, I found Hudson. It looked like it would do what I needed it to do and in addition the Hudson team had provided a web start based demo that allowed me to try it out before ever installing it on my server. After playing with it for a little while and looking into who else was using it successfully, I decided to install it and give it a try.

The rest of this document is going to be about the procedure for installing Hudson on an Ubuntu server. It assumes you are familiar with Linux, GIT, Java, and Tomcat. It will also walk through setting up a Groovy on Grails project as a job under Hudson. For this portion you will probably need to be familiar with Groovy on Grails. There are probably numerous details that I'm missing in this tutorial. Getting this all to work together was a bit of a chore and unless you have several hours to work and are willing to get your hands dirty, you might want to wait until another time to go through this.

Installing a GIT Server

If you are familiar with GIT you would know that it is a very democratic source control system. Every user has their own repository and no repository is really any more important, technically, than any other. Users can push and pull from or to any other repository.

Hudson is going to need a repository that is defined to be used for releases to the continuous integration server. So when you are ready for the CI server to see your changes, you will push them to this GIT repository and Hudson will take a look at them. You will need one of these repositories created for each project you want Hudson to manage. Luckily they are easy to create.

The first step is to get the server ready to handle the repositories. You'll need to install a few packages, if they aren't already on your system:

  sudo apt-get install sshd
  sudo apt-get install git

SSH will be used to communicate with your server through GIT. For this to work your developers will need accounts on the server. To make things easy on us all we're going to create a new group on the server called "developer".

  sudo addgroup developer

We're going to need a user called "git" for the source control system, and a user for each of the developers created. The "git" user will have the developer group as its default group.

  sudo adduser --ingroup developer git 
  sudo adduser ken
  sudo adduser paul
  sudo adduser sue
  sudo adduser bill

And we're going to have to add them all to the developer group.

  sudo adduser ken developer
  sudo adduser paul developer
  sudo adduser sue developer
  sudo adduser bill developer

Since these users are going to all be sharing files we are going to want them to have a umask of 0002. For me this is easy since this is really the only thing I'm using this server for. So I just set the umask in /etc/profile and be done with it. You may have to handle this differently on your end. You could set the umask for each of these users independently in their .bashrc file. There are other ways to handle it. No matter how you do it, the users need to have a umask of 0002 when connecting through SSH.

  sudoedit /etc/profile

Go to the end and change "umask 022" to "umask 0002".

Now we are all set. We just nee to create the actual repository to be used by Hudson.

  sudo -s -H -u git
  cd $HOME
  mkdir MyProject.git
  chmod g+s MyProject.git
  ls -ld MyProject.git

If your umask is set right you should see something like this:

  drwxrwsr-x 2 git developer 4096 2009-10-03 22:26 MyProject.git/

If you've done something wrong and are unfamiliar with Unix file permissions you may not notice anything until you start having problems sharing files. So before you move on make sure that line stars with "drwxrws" and not "drwxrwx" or "drwxr-x".

Now lets create the actual repository.

  cd MyProject.git
  git --bare init
  ls

This is what the output from ls looked like on my server. It should look very close to the same on your server. Note the file permission flags. You need the rws on every directories group, and rw on every files group.

  git@ovz-test:~/MyProject.git$ ls -l
  total 32
  drwxrwsr-x 2 git developer 4096 2009-10-03 22:32 branches
  -rw-rw-r-- 1 git developer   66 2009-10-03 22:32 config
  -rw-rw-r-- 1 git developer   58 2009-10-03 22:32 description
  -rw-rw-r-- 1 git developer   23 2009-10-03 22:32 HEAD
  drwxrwsr-x 2 git developer 4096 2009-10-03 22:32 hooks
  drwxrwsr-x 2 git developer 4096 2009-10-03 22:32 info
  drwxrwsr-x 4 git developer 4096 2009-10-03 22:32 objects
  drwxrwsr-x 4 git developer 4096 2009-10-03 22:32 refs

That's it. GIT is set up! Now all you have to do is push to this repository and you're set. Why don't we do that now. You'll need to go to the machine where you do development and change directories to your MyProject directory (which, of course, already has the .git directory in it created when you started the project). There are two steps. First you need to add the repository you just created and then you need to push your changes to it.

  git remote add origin ken@myserver.com:~git/MyProject.git
  git push origin master

You will need to replace "myserver.com" with your actual server, of course. And you will be asked to login with the username and password of the user, ken, that you created earlier. If you have problems here the error messages should be of help. If not, GIT will give you feedback as it sends the changes up to the new repository.

Installing Tomcat

You may already have Tomcat installed on your system, but you should probably read this section anyway. There are a couple of gotchas here that you should be aware of.

Installing Tomcat under ubutuntu 9.04 is quite easy since it is in the package system. Of course you will also need a modern Java, so we'll get that too.

  sudo apt-get install sun-java6-bin
  sudo apt-get install tomcat6

The problem is that Hudson will not work with the default Tomcat installation. Hudson, like many web apps that you might install under Tomcat requires a greater flexibility than is allowed by the default policy files provided with tomcat6 under Ubunutu. There are two solutions to this problem. We could turn off using a SecurityManager at all and allow web applications to do whatever they want to do, or we could alter the policy file to allow Hudson to have the permissions it needs to run correctly. I'm uncomfortable with the first choice since I would like to have the policy files correct in case I want to install some untrusted software some day in the future. Unfortunately I don't have the time or the information to get the second choice right. So my solution is to keep the policy file, but to grant all permissions to all web applications. In the future I can go into the policy file and grant only the permissions needed to run the applications I want to run. So lets edit the installed policy file first. Go back to the server to do the following.

  sudoedit /etc/tomcat6/policy.d/04webapps.policy

Comment out everything in the file and then add the following three lines to the end of the file.

  grant {
     permission java.security.AllPermission;
  };

I should note that this will allow Hudson (and any other web application you install) do anything it wants to do, subject of course to the permissions granted to the tomcat6 user by the operating system. I have already run into one circumstance where this is a problem. Whenever Hudson asks you to restart the application by pushing a button, you will find that pushing the button crashes Tomcat. My suspicion is that Hudson makes a "System.exit()" call whenever you do this, exiting Tomcat. Normally this would not be allowed by the policy file, but we have circumvented this protection.

Next you'll need to make sure that Tomcat can create the folders it needs to create. I'm pretty sure that we've already done what's needed here, but just to make sure lets go check the file permissions.

  ssh -s -H -u tomcat6   # You will probably need to be root to do this
  cd $HOME
  chmod g+rwXs .

Now Tomcat is all set up for the purposes of this tutorial. You will probably want to spend some time on this configuration, but for now, let's set it aside.

Groovy and Grails

You can safely skip this section if you aren't going to be using Hudson to track a Grails project. If you are, you're going to need a working copy of Grails on the CI server too.

Groovy is easy to install because it is part of the Ubuntu package system. Unfortunately Grails hasn't made it to Ubuntu yet, so we'll have to install that by hand. I actually had quite a bit of trouble getting this all to work with Hudson. I'm going to try to help you avoid the problems I had, but the solution I've come up with is not pretty. You may want to skip it and see if you have a better solution. If you do, please let me know what it is.

To install groovy, just ask the package system for it.

  sudo apt-get install groovy

To install Grails you'll have to download the latest release of Grails from the Internet. I'm using Grails-1.1.1 for the purposes of this tutorial. I'm going to assume that you've gotten the tarball into your home directory on the server and that is our current home directory.

  sudo -s 
  cd /usr/local/lib/
  tar xzf ~/grails-bin-1.1.1.tar.gz
  chmod -R a+rX grails-1.1.1

Now if everything is set up right you should be able to step into the Grails bin directory and run it.

  exit  # Change back to your normal user
  cd /usr/local/lib/grails-1.1.1/bin
  ./grails help    

Now we have, GIT, Tomcat, and Grails installed. Now we just need to install Hudson and configure the first job.

Installing Hudson

Compared to the previous steps, installing Hudson is a breeze. Once we have all the prerequisites together, Hudson just works. First we should probably restart Tomcat to make sure that the changes we've made take hold.

  sudo /etc/init.d/tomcat6 restart

Now you should be able to go to your client machine and see the manager application. Just point your browser at your server and you should see the "It works" page provided by Tomcat. On this page you'll see a link to the manager application. Just got there and make sure you can see the application itself.

Now we know that Tomcat is ready and waiting for the Hudson war file. You'll need to download it from its home on the Internet in another tab in your browser. Once you've downloaded the application to your client, you'll need to send it to the manager application on your server. Just deploy it from the screen you saw earlier. Go to the file chooser toward the bottom the page just upload the war. Tomcat will deploy it and let you know it started. This part went off without a hitch for me. You should be able to click on the

/hudson

link once it deploys and see Hudson ready and waiting for your commands.

Adding a Grails Job to Hudson

This step was more difficult for me than most of the others. It is where I discovered the path problem mentioned earlier. There were other problems too which I hope I have helped you avoid. I'll go through the steps I found necessary to test my grails application under Hudson. I should warn you that my server required security since I am intending to use it for client code. I got the system all set up and the automated builds happening and then I added security to Hudson. The web app became confused when I changed my mind about how I wanted to handle authentication. It started giving me 'not authorized' pages and refused to let me make any more changes. I had to start this step over from the beginning. It didn't take but a few minutes to do it the second time, but you may want to configure security first to avoid this hassle.

First lets configure the Hudson server itself. Click on the "Manage Hudson" link. Then click on the "Manage Plugins" link. You'll want to install the "Hudson Grails Plugin" and "Hudson GIT Plugin". There are others you might like, including the Chuck Norris plugin, but lets keep it simple for now.

You will be asked to restart Hudson now. When you click the link Tomcat will shutdown and you'll have to go back to the server to restart it.

  sudo /etc/init.d/tomcat6 start

Now we're ready to set up our first job. It is our Grails project, MyProject. Go back to the Hudson home page. Click the "New Job" link at the upper left. Select a descriptive name for the job and enter it. Select the "Build a free style software project" option. Hit "OK"

Now we are ready to configure our job. Enter a short description for your project and then move down the page to the "Source Code Management" section. Select "GIT". Enter "/home/git/MyProject.git" to point Hudson to the repository we created earlier. Move down to "Build Triggers" and select "Poll SCM". It will pop up a box for you to enter a schedule. Just add the one line, "*/10 * * * *". This will tell Hudson to check the GIT repository to see if anyone has pushed source code to it ever 10 minutes. If there have been changes then Hudson will perform the build specified by this job.

Go down to the "Build" section. Click on the "Add Build Step" button. Select "Build With Grails". In the "Targets" text field enter "test-app war". This will run your tests and if they are successful, build the war file for you.

You can add other options to the job once we get this part working. The two I would select now are the post build steps, "Archive the Artifacts" and "Publish JUnit test result report". For the first one enter "MyProject-*.war" in the "Files to Archive" text field. For the second enter "test/reports/*.xml", for the "Text report XMLs" text field. For now that should get us started.

Hit the save button. You're done. Well, you're sort of done. You are done configuring Hudson for the job, but you're about to find out that Hudson still won't be able to run the job. Let's give it a try. Go back to the Hudson home page. Go to MyProject. Click the "Build Now" button. What you will find is that GIT is unable to do tagging without user.name and user.email set for the project. You'll need to log in as the user that should be running Hudson and use

git config

to set these variables. So back to the server.

Now we've been starting and stopping Tomcat over and over again during this tutorial. One of the things that might surprise you is that Tomcat may have a rather strange idea of where its home directory is. If you use

ps

at the prompt to look at the environment you may find that HOME is not set the way you would expect it to be. The first step is to make sure that in the future HOME is set to the tomcat6 home directory. So lets edit the file that starts up Tomcat first.

  sudoedit /etc/init.d/tomcat6

We're going to want to make two changes to this file. The first is to add the path to Grails to the PATH statement and the second is to make sure that the HOME environment variable is set. So go down the file about a dozen lines and you will see a statement that looks like this:

  PATH=/bin:/usr/bin:/sbin:/usr/sbin

Change it to this:

  PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/lib/grails-1.1.1/bin

Directly after that line, add this line:

HOME=/var/lib/tomcat6

Now save the file. Tomcat should be able to find everything it needs to now, but there are a couple more steps we need to complete before the build will work. First we need to make sure that GIT can tell who the tomcat6 user is so it will allow a tag operation to execute.

  sudo -s -H -u tomcat6
  git config --global user.name "My Name"
  git config --global user.email "myemail@mydomain.com"
  exit

Next we need to make sure that the repository created by Hudson has all the necessary files for running a test. I've found that these tests will crash if you haven't at least run the Grails app in the directory once. It doesn't have to succeed, you just care about the files being copied into the correct places. So lets move to Hudson's checked out version of MyProject and run grails there once.

  cd /usr/share/tomcat6/.hudson/jobs/Time Tracker/workspace
  /usr/local/lib/grails-1.1.1/bin/grails run-app

It will crash. That's OK. Now that we've completed these steps, the job should run under Hudson. Go give it a try.

---

I'm sure that there are errors and omissions above. If this doesn't work for you, or there are things that should change in this tutorial, please leave a comment below and I'll update the page.