Contributing to apache-hadoop (hadoop-1.0.0)

I decided it was time to contribute to the apache hadoop project. I haven't touched java in a while so thought i'd start with something easy. I found an unassigned JIRA, fixed the issue (using vim as it just worked) and then started to look at how i actually test / submit my patch. There's some information on how to contribute from the following links:

http://wiki.apache.org/hadoop/HowToContribute
http://wiki.apache.org/hadoop/QwertyManiac/BuildingHadoopTrunk

I had a fedora 16 box and installed the necessary software to develop my hadoop patch. I checked out the source using svn :

# svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1 hadoop

Then checked my environment by running ant:

# cd hadoop

# ant

This took a while, but built cleanly. Cool. I thought my environment must be good. Think again....

Following the instructions on the HowToContribute page, I created a patch. I then needed to test the patch against the existing tests. Easy....

# ant test

I left it running overnight as it can take up to 3 hours to complete (as suggested on the mailing lists). I check the next morning and about 25% of the tests failed. I read through some of the logs and it looks like permission errors. I'm quite competent in this area so dug a little and the JUnit tests expect specific file permissions. After some trial and error, the umask of 0022 seemed to work just fine:

# umask 0022

Re-run the tests and wait a few hours.

It complete again with a lot of errors. Less than before but not zero so something else must be wrong. I look through these logs and notice the regular expressions expect the hostname to be localhost. That's clearly not the case. I *correct* this and am currently re-running the tests. We shall see how it goes in the morning......

If and when the tests complete OK, I then need to read up on any pre-requisites to submitting patches. As this is only a small fix (and a similar one didnt require a new test), I'm not planning to (learn how to) write a JUnit test. Other than that I'll just submit the patch and hope the community doesn't bite