How to set up and compile programs on the Beowulf cluster using distcc

Distcc is a program to distribute compilation of C or C++ code across several machines on a network. distcc should always generate the same results as a local compile, is simple to install and use, and is often two or more times faster than a local compile.

1. Login to the head node of the Beowulf cluster.
2. Type in: export DISTCC_HOSTS=”host1 host2 host3” where host1, host2, and host3 are the names of the machines you want to participate in the distributed compile.
   Ex. export DISTCC_HOSTS="amazon biobio danube euphrates ganges harbinge" will set it up so that the 6 machines listed will participate in the distributed compile.
3. Now we need to start distccd, the distcc daemon, on all the machines we specified. The easiest way to do so is to type in: for f in $DISTCC_HOSTS; do ssh $f "distccd --daemon"; done
4. After the above command finishes, you are ready to start compiling. Proceed with the normal steps for compiling, but specify distcc as your C compiler. An example of a distributed compile on 6 machines is: CC='distcc' ./configure make CC='distcc’ –j6 the make –jN option specifies how many make jobs can be ran in parallel. Typically we want N to be equal to the number of machines being used.
5. When you are done using distcc, shutdown distccd on all machines by typing: for f in $DISTCC_HOSTS; do ssh $f “killall distccd”; done

A complete list of Beowulf execution node names can be found in /etc/hosts.