wiki:Internal/BuildingStandaloneOrbit

Context Navigation

Version 14 (modified by Surya Satyavolu, 18 years ago) ( diff )
—

Orbit > Building

Building an ORBIT sandbox setup

This page documents the steps necessary to build an ORBIT sandbox setup.

The test bed consists of two server side machine: one that supports all the ORBIT services and the other machine is the console to run the experiments. The following figure gives an overview of the test bed setup

The following naming convention and IP addressing have been configured.

Services: The machine “services” station is used to host various services such as DHCP, DNS, NTP, NFS, mysql, OML, repository etc. We have different aliases for the management host to segregate the services that it hosts

Station (services.orbit-lab.org)	IP Address (on eth1)	IP address (on eth0)
dhcp.orbit-lab.org	10.10.0.9 (fixed)	10.1.200.2 (fixed)
ntp.orbit-lab.org
idb1.orbit-lab.org
edb1.orbit-lab.org
repository1.orbit-lab.org
repository2.orbit-lab.org

Console: The machine console is used to run the experiments as well as host the CMC service. Hence, console machine has nodeHandler and CMC in order to support the above functionality. The aliases for this machine are

Station (console.orbit-lab.org)	IP Address (on eth1)	IP address (on eth0)
cmc.orbit-lab.org	10.10.0.21 (fixed)	10.1.200.1 (fixed)
console.orbit-lab.org
consolec.orbit-lab.org

Installation Common to both machines

The following procedure applies to both services and console. On a clean machine,

install debian using CD
- Download the net-install ISO image (about 104 MB) from here
- Burn this onto a CD, make sure that you burn it as an ISO image and not a file; otherwise the CD will not be bootable. You can use the shareware utility UltraISO (http://www.ezbsystems.com/ultraiso/) to burn this image onto a CD
- Use this CD to boot the new machine up (make sure that you have the correct boot sequence in the BIOS settings)
- Ensure that you have a network connection to outside
- At the boot prompt: type linux26 to instruct the installer to use kernel 2.6 for the installation. This will ensure that most of the hardware (including serial ATA harddisks) will be recognized without any problems.
- Follow instructions on the installer to proceed with the installation

Run apt-get install kernel-image-2.6.10-1-i686
- In order to do this make sure you have the debian sources pointing to unstable in the /etc/apt/sources.list file
Run update-grub
Reboot into 2.6.10

Setting up the networking for these machines can be done as follows

Edit /etc/network/interfaces

For services.orbit-lab.org

auto lo eth0 eth1

iface lo inet loopback

iface eth0 inet static
	address 10.1.200.2
	netmask 255.255.0.0

iface eth1 inet static
address 10.10.0.9
     netmask 255.255.0.0

For Console.orbit-lab.org

auto lo eth0 eth1

iface lo inet loopback

iface eth0 inet static
	address 10.1.200.1
	netmask 255.255.0.0

iface eth1 inet static
address 10.10.0.21
     netmask 255.255.0.0

Software and Services

The services and software running on each of the two servers is listed below. Configuration of individual services is explained in detail as well

(services.orbit-lab.org)	(console.orbit-lab.org)
DHCP	CMC
DNS	NodeHandler
NTP
TFTP
PXE
Frisbee
NFS
Apt
Mysql
OML collection server
Apache

DHCP

Purpose: This software runs a DHCP server that assigns IP addresses to clients on demand

Installation and Configuration: :

apt-get install dhcp3-server : select eth1 as interface
Edit /etc/dhcp3/dhcpd.conf
Make sure that /etc/default/dhcp3-server has eth1 as interface
First line is use-host-decl-names on which means DNS has actual IP mappings, DHCP just gives out hostnames. This means only IP to name updates need to be done only at DNS.

To Run: : /etc/init.d/dhcp3-server start — errors go to /var/log/daemon.log

Configuration :

#############dhcpd.conf################
ddns-update-style interim;
authoritative;
use-host-decl-names on;

allow booting;
allow bootp;

default-lease-time 259200;
max-lease-time 259200;

filename "/pxelinux.bin";
option vendor-encapsulated-options 09:0f:80:00:0c:4e:65:74:77:6f:72:6b:20:62:6f:6f:74:0a:07:00:50:72:6f:6d:70:74:06:01:02:08:03:80:00:00:47:04:80:00:00:00:ff;
option option-150 code 150 = text;

#
#Grid Control
subnet 10.10.0.0 netmask 255.255.0.0 {
	range  10.10.250.1 10.10.250.255;
	option domain-name "orbit-lab.org";
  	ddns-updates off;
  	ddns-domainname "orbit-lab.org";
  	ddns-rev-domainname "in-addr.arpa";
	option domain-name-servers 10.10.0.9;
        next-server 10.10.0.9;
	option routers 10.10.0.1;

        host services.orbit-lab.org {hardware ethernet 00:40:63:DA:12:0E; fixed-address services.orbit-lab.org ;}
	#replace the MAC addresses here with those of the new machine
	host console.orbit-lab.org  {hardware ethernet 00:40:63:C9:F6:D5; fixed-address console.orbit-lab.org ;}
	
	#Nodes 10.10.x.y
	group {
	host node1-1.orbit-lab.org {hardware ethernet 00:0F:EA:4C:A6:B7; fixed-address node1-1;}
	host node1-2.orbit-lab.org {hardware ethernet 00:0F:EA:4A:8B:5B; fixed-address node1-2;}
	host node1-3.orbit-lab.org {hardware ethernet 00:0F:EA:47:6C:87; fixed-address node1-3;}
	host node2-1.orbit-lab.org {hardware ethernet 00:0F:EA:4C:A6:C1; fixed-address node2-1;}
	host node2-2.orbit-lab.org {hardware ethernet 00:0F:EA:4C:B7:5D; fixed-address node2-2;}
	host node2-3.orbit-lab.org {hardware ethernet 00:0F:EA:4B:FE:C7; fixed-address node2-3;}
	host node3-1.orbit-lab.org {hardware ethernet 00:0F:EA:4A:8A:F7; fixed-address node3-1;}
	host node3-2.orbit-lab.org {hardware ethernet 00:0F:EA:4C:B7:53; fixed-address node3-2;}
	host node3-3.orbit-lab.org {hardware ethernet 00:0F:EA:4A:8B:63; fixed-address node3-3;}
	}
}

DNS

Purpose: Services.orbit-lab.org” hosts the primary DNS service for the zone orbit-lab.org. The DNS server is the standard BIND9 software packaged with Linux

Installation and Configuration: :

apt-get install bind
You need to create the following files under /etc/bind
- Named.conf : Configuration for DNS as shown below
- Orbit.zone: Forward lookup
- Zone.orbit: Reverse lookup
- In named.conf, edit destination directory to /etc/bind
Make sure there are dots at the end of the domains
Add the highlighted blue line to /etc/hosts file as follows

127.0.0.1	localhost
10.10.0.9	services.orbit-lab.org	services

To Run: :

/etc/init.d/bind start — errors go to /var/log/daemon.log

Configuration

########################/etc/bind/named.conf##############################
options {
	directory "/etc/bind";
	/*
	 * If there is a firewall between you and nameservers you want
	 * to talk to, you might need to uncomment the query-source
	 * directive below.  Previous versions of BIND always asked
	 * questions using port 53, but BIND 8.1 uses an unprivileged
	 * port by default.
	 */
};

Controls {
       inet 127.0.0.1 port 953
       allow { 127.0.0.1; };
};

// a caching only nameserver config
 
zone "." IN {
	type hint;
	file "named.ca";
};

zone "localhost" IN {
        type master;
        file "localhost.zone";
};


zone "orbit-lab.org" IN {
	type master;
	file "orbit.zone";
	
};

zone "in-addr.arpa" IN {
        type master;
	file "zone.orbit";
	
};

zone "0.0.127.in-addr.arpa" IN {
	type master;
	file "named.local";
};
#####EOF#########

###############/etc/bind/orbit.zone#################
$ORIGIN .
$TTL 3600	; 1 hour
orbit-lab.org	IN SOA	orbit-lab.org services.orbit-lab.org (
				2005041903 ; serial
				3600       ; refresh (1 hour)
				600        ; retry (10 minutes)
				10000       ; expire (2 hours)
				3600       ; minimum (1 hour)
				)
	IN	NS 	services.orbit-lab.org.
				
$ORIGIN orbit-lab.org. 
$TTL 129600	; 1 day 12 hours
node1-1		A	10.10.1.1
node1-2		A	10.10.1.2
node1-3		A	10.10.1.3
node2-1		A	10.10.2.1
node2-2		A	10.10.2.2
node2-3		A	10.10.2.3
node3-1		A	10.10.3.1
node3-2		A	10.10.3.2
node3-3		A	10.10.3.3
services		A	10.10.0.9
console		A	10.10.0.21
dhcp		CNAME	services.orbit-lab.org.
cmc		CNAME	console.orbit-lab.org.
consolec	CNAME	console.orbit-lab.org.
frisbee	CNAME	services.orbit-lab.org.
pxe		CNAME	services.orbit-lab.org.
oml		CNAME	services.orbit-lab.org.
repository	CNAME	services.orbit-lab.org.
repository1	CNAME	services.orbit-lab.org.
repository2	CNAME	services.orbit-lab.org.
ntp		CNAME	services.orbit-lab.org.
loghost	CNAME	console.orbit-lab.org.
idb1		CNAME	services.orbit-lab.org.
edb1		CNAME	services.orbit-lab.org.
#####EOF#########

######################/etc/bind/zone.orbit##################

$ORIGIN .
$TTL 3600	; 1 hour
in-addr.arpa		IN SOA	orbit-lab.org. services.orbit-lab.org. (
				33354      ; serial
				28800      ; refresh (8 hours)
				900        ; retry (15 minutes)
				604800     ; expire (1 week)
				86400      ; minimum (1 day)
				)
	IN		NS	services.orbit-lab.org.
$ORIGIN in-addr.arpa.
$ORIGIN 10.in-addr.arpa.
$ORIGIN 0.10.10.in-addr.arpa.
$TTL 129600	; 1 day 12 hours
9	IN 	PTR		services.orbit-lab.org.
21	IN	PTR		console.orbit-lab.org.

$ORIGIN 1.10.10.in-addr.arpa.
1	IN	PTR		node1-1.orbit-lab.org.
2	IN	PTR		node1-2.orbit-lab.org.
3	IN	PTR		node1-3.orbit-lab.org.

$ORIGIN 2.10.10.in-addr.arpa.
1	IN	PTR		node2-1.orbit-lab.org.
2	IN	PTR		node2-2.orbit-lab.org.
3	IN	PTR		node2-3.orbit-lab.org.

$ORIGIN 3.10.10.in-addr.arpa.
1	IN	PTR		node3-1.orbit-lab.org.
2	IN	PTR		node3-2.orbit-lab.org.
3	IN	PTR		node3-3.orbit-lab.org.

#####EOF#########

Apache Web Server

Purpose: Apache server is required for maintaining the ORBIT local repository for debian packages and also to view the results of the experiment

Installation: apt-get install apache2 Note that no additional configuration is needed for apache

To run:

Make sure that /var/www/cgi-bin points to /usr/lib/cgi-bin (or create a soft link if one does not exist using cd /var/www/; ln –s /usr/lib/cgi-bin cgi-bin)

To run /etc/init.d/apache2 start — errors go to /var/log/daemon.log

Make sure that you have latest version of libgd and the perl modules for GD available. This is needed to view the results of the experiment. To do that:

wget http://www.boutell.com/gd/http/gd-2.0.33.tar.gz
tar xfzv gd-2.0.33.tar.gz; cd gd-2.0.33;
./configure –prefix=/usr ; make ; make install

Once this is done, download the latest GD perl module that depends on the previously installed libgd library using the following commands

wget http://search.cpan.org/CPAN/authors/id/L/LD/LDS/GD-2.19.tar.gz
tar xfzv GD-2.19.tar.gz; cd gd-2.19;
perl Makefile.PL; make; make install

The above two steps should install the GD library and perl modules correctly.

Repository

Purpose: This serves as the debian package repository for local ORBIT packages (such as OML, OTG, Libmac, nodeAgent etc). It uses apache services to fetch and distribute debian packages. For this service, it is sufficient to install just the ORBIT packages. There is no need to mirror the entire debian repository which can be easily accessed by syncing with the external debian repository

Configuration:

mkdir /var/www/winlab
chmod 755 /var/www/winlab
mkdir /var/www/winlab/binary

Under the above directory, make sure you have the following packages

Frisbee_1.0.0-1.i386.deb = This has the Frisbeed, Frisbee client
Gridservices_3.0.0-1_all.deb = This has Frisbee and pxe webservices (http wrapper to initialize frisbeed while frisbeeing)
Oml-client_2.0_0_i386.deb = oml client
Otg_1.1.0-1_i386.deb = traffic generator
OML_server, CMC will be integrated into the gridservices packages soon

Edit the /etc/apt/sources.list file as follows

deb http://repository1.orbit-lab.org/winlab/ binary/

deb http://http.us.debian.org/debian/ testing main
 deb-src http://http.us.debian.org/debian/ testing main
deb http://security.debian.org/ stable/updates main

NTP

Purpose:

All the machines synchronize their time using the time server as the reference.

Installation and Configuration:

apt-get install ntp-server
Edit /etc/ntpd.conf
- Point the server to pool.ntp.org

To run:

/etc/init.d/ntpd start — errors go to /var/log/daemon.log

TFTP Server

Purpose: TFTP is needed to install PXE images whenever you need to install an image onto the node (using Frisbee). It is also used to load a memory based image that can be used to fetch the current image of the node into the repository

Installation and Configuration:

apt-get install atftpd

There are two options here: either to run atftpd as a standalone daemon or run it under inetd. For heavy duty tftp services, you can choose to run is as a standalone daemon. For our installation, we choose the standalone daemon.

Point the tftp directory to /tftpboot
Copy /tftpboot directory from repository2.orbit-lab.org

In case of problems, make sure that lo interface is up

NFS Service

Purpose: This service is used to remotely mount directories on the nodes while fetching their image using imagezip utility. Also, Frisbee service makes use of this directory to install images onto nodes.

Installation and Configuration:

apt-get install nfs-kernel-server

Edit the /etc/exports file as follows

Probably should be sync here

To run:

/etc/init.d/nfs-kernel-server start — errors go to /var/log/daemon.log

Mysql Server

Purpose: This service is used to store the results of the experiments conducted on ORBIT

Installation and Configuration:

apt-get install mysql-server-4.1
Edit /etc/mysql/my.cnf and replace bind-address from 127.0.0.1 to 10.10.0.9

type mysql and at the prompt, enter the following

Replace the password by an appropriate one. Basically, this creates a new account called orbit with the password specified and allows access to mysql databases from localhost and any other machine on the network.

To run:

/etc/init.d/mysql start — errors go to /var/log/mysql.log

OML Server (Collection Server)

Purpose:

This service is used to collect the measurements of the experiments at runtime from the nodes and store them into the mysql database.

Installation and Configuration:

Apt-get install tcl8.4
Apt-get install mysqltcl
Install libghthash-0.5.3 from source and point /etc/ld.so.conf to /usr/local/lib
Copy /etc/init.d/collectionwebService2.0 from idb1
Copy /opt/nodeAgent/bin/ from idb1
Copy /opt/nodeAgent/lib/ from idb1
Copy /opt/oml_server/bin/collection_server from idb1
Copy /usr/local/lib/oml from idb1

To run:

/etc/init.d/collectionWebService2.0 start — errors go to /var/log/daemon.log

Note that that will be packaged soon as a complete deb package with all dependency checks

Gridservices (PXE and Frisbee)

Purpose:

This service is for fetching and installing images from and to the nodes and used the Frisbee utility from University of Utah to distribute the image in the form of chunks using multicast.

Installation and Configuration:

Apt-get install gridservices
Apt-get install frisbee
/opt/gridservices/services/frisbee.rb has 10.10.0.40 hardcoded for Frisbee server. Make sure you change that to 10.10.0.9 which is our Frisbee server
Frisbeed installs under /usr/sbin when you apt-get it, however /opt/gridservices/services/Frisbee.rb points to /opt/Frisbee/bin .Make sure , you change that or create a soft link from /opt/frisbee/bin/frisbee to /usr/sbin/frisbee

To run: /etc/init.d/gridservices start — errors go to /var/log/gridservices.log

CMC (Runs on console.orbit-lab.org)

Purpose: CMC service talks to the individual Chassis Managers on the nodes and can be used to remotely power on, power off nodes, reset them and also provided access serial console of the nodes to the users. This is done using a http interface (Make sure that you do not run this service and apache on the same machine, since they will both need port 80)

Installation and Configuration:

apt-get install libmysqlclient12
apt-get install libx11-6
Make sure that the interface eth0 is set to 10.1.200.1 netmask 255.255.0.0
Copy /usr/local/sbin/cmc on to local machine
Copy /etc/init.d/cmc

To run:

/etc/init.d/cmc start

Note that that will be packaged soon as a complete deb package with all dependency checks

Node Handler (runs on console.orbit-lab.org)

Purpose: NodeHandler is used to run the experiments from the console using reliable multicast to disseminate commands to the nodes. It also initializes measurement collection appropriately. It primarily uses ruby scripts to do the same.

Installation and Configuration: Make sure that you apt-get install the following packages

liblog4r-ruby 1.0.5-3
librexml-ruby 3.1.2.1+ruby1.
libruby1.8 1.8.2-3
libwebrick-rub 1.3.1+ruby1.8.
ruby 1.8.2-1
ruby1.8 1.8.2-3

Copy /opt/nodeAgent3 from console.grid.orbit-lab.org to this machine

Note that that will be packaged soon as a complete deb package with all dependency checks

Switch Configuration

Partition switch into two VLAN’s
1st 12 ports for control, next 12 for CM

Useful Tips

/etc/init.d/ifupdown should have ifup –a so that all the interfaces are up at system boot
To bring up all the services automatically after a reboot, use sysv-rc-conf utility
To have modules load at boot time (put them in /etc/modules)
Use tzconfig to change your time zone if required

Attachments (1)

9nodesandbox.png (17.6 KB ) - added by Surya Satyavolu 18 years ago.

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text