Black&White Project/Compilation Farm/Setup

From Kiwix
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Organization

The Compilation Farm is installed on the download.kiwix.org server. It is organized as:

  • A buildbot master running as user buildbot on the server.
  • A set of VirtualBox VMs running buildbot slaves.
    • ubuntu32 (192.168.5.50): i686 static
    • ubuntu64 (192.168.5.51): x86_64 static
    • macosx (192.168.5.52): universal (i386+x86_64) dmg with app.
    • win32 (192.168.5.53): i386 windows static
    • win64 (192.168.5.54): x64 windows static
    • fedora (192.168.5.55): i686 sugar static (kiwixdev)
    • arm (no direct IP. ssh on localhost through port 5022 10.0.2.2 192.168.5.56): armel kiwix-serve static
    • debian32 (192.168.5.57): i686 wheezy deb
    • debian64 (192.168.5.58): x86_64 wheezy deb

Usage

A utility script, kbox is present to facilitate start and stop of the VMs.

(buildbot)buildbot@www:~/buildbot$ kbox --help
Usage: kbox [options]

Handy wrapper around virtualbox for kiwix

Options:
  -h, --help   show this help message and exit
  --listlocal  list names of VMs. Edit script to change.
  --listall    list all VMs with their status.
  --usb        list all USB devices on the host.
  --startall   start all VMs.
  --stopall    sends shutdown to all VMs.
  --start=VM   starts the VM named [VM].
  --stop=VM    sends shutdown to the VM named [VM].
  --info=VM    displays info for the VM named [VM].
  --ssh=VM     connects via SSH to the VM named [VM].
  • Example connection to ubuntu32 VM:
kbox --ssh ubuntu32

Master Setup

  • Install required packages
# apt-get install build-essential git zip unzip python-pip vim python-dev
pip install virtualenv virtualenvwrapper
  • Create user buildbot
  • Log-in as user buildbot
  • Edit .bashrc and add at the end
export EDITOR="vim -v"

export WORKON_HOME=~/envs
source /usr/local/bin/virtualenvwrapper.sh
  • Log-out then log-in to enable virtualenvwrapper
  • Create virtualenv
mkvirtualenv --no-site-packages buildbot
mkdir -p buildbot && cd buildbot
  • Install builbot and create master
easy_install buildbot
buildbot create-master master
  • Deploy configuration
git archive --prefix=code/ --remote git://git.code.sf.net/p/kiwix/kiwix HEAD:buildbot | tar x
cd master && ln -sf ../code/master.cfg master.cfg && cd -
cp code/secret_pass.py master/secret_pass.py
  • Edit the password file in master/secret_pass.py
  • Start the master
buildbot start master
  • Create an nginx proxy at /etc/nginx/sites-available/buildbot.kiwix.org
server {
  listen localhost:81;
  listen 88.190.13.50:80;
  server_name buildbot.kiwix.org;

  access_log off;

  proxy_buffering off;
  proxy_max_temp_file_size 0;

  set_real_ip_from 127.0.0.1;
  set_real_ip_from 88.190.13.50;
  real_ip_header X-Forwarded-For;

  location / {
    proxy_set_header X-Real-IP  $remote_addr;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_pass http://localhost:8010;
    proxy_set_header Host $host;
  }
}

Automatic Startup

  • crontab -e on builbot user
@reboot /home/buildbot/envs/buildbot/bin/buildbot start /home/buildbot/buildbot/master

Slave Setup

buildbot slave does not require much: a working python/setuptools environnment.

Linux Slave Setup

  • Create a VirtualBox VM (20GB of storage - dynamic)
  • Install Ubuntu 10.04.4 (Newer versions of Ubuntu uses a recent glibc)
  • Select OpenSSH server during install.
  • Create a regular user (dev)
  • Use whole disk partitioning
  • Install packages:
# apt-get install --fix-missing acpi acpid build-essential autoconf automake libtool pkg-config fastjar libssl-dev libexpat1-dev libxml2-dev libgnutls-dev libgpg-error-dev libgcrypt-dev uuid-dev libuuid1 git zip unzip python-pip vim python-dev
  • Install python tools
# pip install virtualenv virtualenvwrapper

From now, you won't need root privileges anymore.

  • Log in as dev user
  • Edit ~.bashrc and add at the end
export EDITOR="vim -v"

export WORKON_HOME=~/envs
source /usr/local/bin/virtualenvwrapper.sh
source ${WORKON_HOME}/buildbot/bin/activate
  • Log out then log-in again (to activate virtualenvwrapper)
  • Create virtualenv
mkvirtualenv --no-site-packages buildbot
  • Install buildbot
easy_install buildbot-slave
  • Create the slave
mkdir -p buildbot && cd buildbot
buildslave create-slave slave 192.168.5.1:9989 ubuntu32 BUILDER_PASS
  • Edit files in slave/info/ and put host description and maintainer
  • Start the slave
cd slave/
buildslave start

Automatic Startup

  • crontab -e on dev user
@reboot /home/dev/envs/buildbot/bin/buildslave start /home/dev/buildbot/slave

VirtualBox Setup

  • Install the non-free version of Virtual Box:
wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
echo "deb http://download.virtualbox.org/virtualbox/debian lucid contrib non-free" >> /etc/apt/sources.list
apt-get update
apt-get install virtualbox-4.1
apt-get install linux-headers
/etc/init.d/vboxdrv setup
wget -c http://download.virtualbox.org/virtualbox/4.1.14/Oracle_VM_VirtualBox_Extension_Pack-4.1.14-77440.vbox-extpack
VBoxManage extpack install /home/buildbot/vms/Oracle_VM_VirtualBox_Extension_Pack-4.1.14-77440.vbox-extpack
  • VMs will be stored in /home/buildbot/vms
  • Configure Network
VBoxManage hostonlyif create
VBoxManage hostonlyif ipconfig vboxnet0 --ip 192.168.5.1 --netmask 255.255.255.0
VBoxManage dhcpserver add --ifname vboxnet0 --ip 192.168.5.1 --netmask 255.255.255.0 --lowerip 192.168.5.10 --upperip 192.168.5.50 --enable
  • Create a HDD Image
VBoxManage createhd --filename ~/vms/ubuntu32/ubuntu32.vdi --size 20480 --variant Standard
  • Create VM (VBoxManage list ostypes for types list)
VBoxManage createvm --register --name ubuntu32 --ostype Ubuntu --basefolder ~/vms/
VBoxManage modifyvm ubuntu32 --memory 512 --audio none --clipboard disabled --usb off --usbehci off --nic1 nat --nictype1 82543GC --nic2 hostonly --nictype2 82543GC --hostonlyadapter2 vboxnet0 --vrde on --boot1 dvd --boot2 disk
VBoxManage storagectl ubuntu32 --name hdd --add sata --controller IntelAhci --bootable on
VBoxManage storagectl ubuntu32 --name cd --add ide --controller PIIX4 --bootable on
VBoxManage storageattach ubuntu32 --storagectl hdd --type hdd --port 1 --medium ~/vms/ubuntu32/ubuntu32.vdi
VBoxManage storageattach ubuntu32 --storagectl cd --type dvddrive --port 1 --device 1 --medium ~/vms/ubuntu-10.04.4-server-i386.iso
  • Start VM & Install System using rdesktop
VBoxManage startvm ubuntu32 --type headless

On your computer, connect with:

rdesktop download.kiwix.org:3389
  • Configure Network (eth1) with a static IP according to mapping above in /etc/network/interfaces
iface eth1 inet static
    address    192.168.5.50
    netmask    255.255.255.0
    gateway    192.168.5.1
auto eth1
  • Once installed, tweak the VM
VBoxManage storageattach ubuntu32 --storagectl cd --type dvddrive --port 1 --device 1 --medium none
VBoxManage modifyvm ubuntu32  --vrde off

From now, connection to the VM will be SSH only.

Post B&W todo

  • Start VM automaticaly/transparently if a compilation is required - otherwise VM is offline.
  • Start chained compilations : only the first one is planned, and all other ones occur one by one afterwards.
  • Bump code with bumpVersion.sh to have a version like svn-[date]
  • Rename bin packages

Tips

  • win32 upload issue: VBoxManage modifyvm win32 --natsettings1 1500,64,64,64,1024