[wiki:Internal/OpenFlow/ return to OpenFlow index] = NetFPGA Host Setup = This is the latest set of instructions for installing NetFPGA drivers on Ubuntu (10.10), and making it work as an !OpenFlow switch. It is assumed that your working directory is /opt/. == I. NetFPGA setup == #sec1 This section describes the steps required to build v.2.2.0 of the NetFPGA kernel module on Ubuntu. The same steps can be used to build more recent versions. Steps to build the older version of the drivers can be found [#old here]. === 1.1 Download sources/dependencies === In order to download tarballs, you need a Foswiki account. Get one [http://netfpga.org/foswiki/bin/view/NetFPGA/UserRegistration here] if you don't have one. `cd` to /opt/ and fetch and untar the following: * netfpga_full_2_2_0.tar.gz from: http://www.netfpga.org/releases/netfpga_full_2_2_0.tar.gz [[BR]] REF: http://netfpga.org/foswiki/NetFPGA/OneGig/Releases * netfpga_openflow_switch_1_0_0-4.tar.gz from: http://netfpga.org/beta/distributions/netfpga_openflow_switch_1_0_0-4.tar.gz [[BR]] REF: http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/OpenFlowNetFPGA100 or, if you want the tunneling !OpenFlow switch fetch the following in place of netfpga_openflow_switch_1_0_0-4. * netfpga_tunneling_openflow_icmp_arp_1_0_0.tar.gz, from the same link as above. * install dependencies {{{ apt-get update apt-get install build-essential linux-headers-`uname -r` ncurses-dev libnet1-dev libxml-simple-perl libio-interface-perl liblist-moreutils-perl liberror-perl git-core libpcap0.8 libpcap0.8-dev }}} === 1.2 Compatibility fixes === The following changes have to be made in order to compile the drivers on Ubuntu. * Add the following symlinks: {{{ ln /usr/bin/lspci /sbin/lspci ln /usr/bin/setpci /sbin/setpci }}} * /opt/netfpga/lib/C/kernel/nf2util.h : add after the first #ifndef/#define block: {{{ // Replace SA_SHIRQ with IRQF_SHARED on newer kernels #ifndef SA_SHIRQ #define SA_SHIRQ IRQF_SHARED #endif }}} * /etc/default/grub: add "vmalloc=512m" to GRUB_CMDLINE_LINUX_DEFAULT and do `update-grub` For the kernel: * If using kernel version 2.6.35-25, upgrade to 2.6.35 (`uname -ar` should show you which one you are running): {{{ apt-get install linux-headers-2.6.35-30-generic linux-image-2.6.35-30-generic }}} * Reboot. After reboot, check that the kernel flags are correct: {{{ dmesg | grep vmalloc [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.35-30-generic root=UUID=c90285a2-48de-4ab0-950f-241817516ad8 ro vmalloc=512m console=tty0 console=ttyS0,115200n8 [ 0.000000] vmalloc : 0xdf7fe000 - 0xff7fe000 ( 512 MB) }}} === 1.3 Compile drivers === * export environment variables: {{{ export NF_ROOT=/opt/netfpga #(wherever your netfpga base files are) source ${NF_ROOT}/bin/nf_profile }}} * from /opt/netfpga: `make`; `make install` After compilation, load the kernel module: {{{ modprobe nf2 }}} If all goes well, you should see the an entry if you check with `lsmod | grep nf2`. At this point, you can load bitfiles and such by first running `cpci_reprogram` and then using nf_download to program the NetFPGA with the correct bitfile. Your session should go something like this: {{{ # cpci_reprogram.pl --all Loading the CPCI Reprogrammer on NetFPGA 0 Loading the CPCI on NetFPGA 0 CPCI on NetFPGA 0 has been successfully reprogrammed # nf_download /opt/netfpga/bitfiles/reference_switch.bit Found net device: nf2c0 Bit file built from: nf2_top_par.ncd;HW_TIMEOUT=FALSE Part: 2vp50ff1152 Date: 2011/ 1/20 Time: 6: 8:48 Error Registers: 0 Good, after resetting programming interface the FIFO is empty Download completed - 2377668 bytes. (expected 2377668). DONE went high - chip has been successfully programmed. CPCI Information ---------------- Version: 4 (rev 1) Device (Virtex) Information --------------------------- Project directory: reference_switch Project name: Reference Switch Project description: Reference Switch Device ID: 3 Version: 1.0.0 Built against CPCI version: 4 (rev 1) Virtex design compiled against active CPCI version }}} == II. NetFPGA !OpenFlow switch == The above instructions apply to any recent version of NetFPGA driver (v2.1.2 - 3.0.1). The following steps are needed in order to use the NetFPGA as an !OpenFlow switch supporting v1.0 of the !OpenFlow protocol. While any version of the NetFPGA drivers /should/ work, the !OpenFlow components were built atop '''v2.2.0 of the NetFPGA drivers'''. These instructions assume that you have already gotten the NetFPGA up and running on an Ubuntu box, as per [#sec1 section 1 of this guide]. According to the guides, !OpenVswitch provides a kernel implementation of the OF switch, whereas Stanford's reference switch is purely userspace. The former is faster, the latter more customizable. The NetFPGA !OpenFlow switch utilizes the latter, but can probably be used with !OpenVswitch as well (though yet to be tested with a node). === 2.1 dependencies === * Install dependencies : {{{ apt-get install autoconf libtool }}} * Use git to pull NetFPGA-specific !OpenFlow sources to /opt/: {{{ git clone git://openflowswitch.org/openflow.git cd openflow git checkout -b 1.0.0-netfpga origin/devel/tyabe/1.0.0-netfpga }}} === 2.2 build !OpenFlow === All bitfiles belong in the directory /opt/openflow/hw-lib/nf2. Steps 3 and 4 below download the !OpenFlow switch bitfiles into this directory. {{{ ./boot.sh cd ./hw-lib/nf2 wget http://openflowswitch.org/downloads/netfpga/openflow_switch.bit.100_3.tar.gz tar zxvf openflow_switch.bit.100_3.tar.gz ==> we need newer one from NF site:http://www.netfpga.org/releases/netfpga_openflow_switch_1_0_0-3.tar.gz cd ../../ ./configure --enable-hw-lib=nf2 make make install }}} After compilation, copy of_start.sh and of_stop.sh from /opt/netfpga/projects/openflow_switch/sw/. Given that you have a !OpenFlow controller up and running, you can connect the switch to the controller with the following: {{{ ./of_start.sh x.x.x.x:6633 }}} Where x.x.x.x is the controller's IP address and 6633 the control port number. This automatically generates a random DPID for the switch; if you want to specify a DPID, modify the script such that `ofdatapath` takes the `-d` parameter for DPID, e.g: {{{ /opt/openflow/udatapath/ofdatapath --detach punix:/var/run/dp0 -d 001010222324 -i nf2c0,nf2c1,nf2c2,nf2c3 }}} You can then run the same command as: {{{ ./of_start.sh 172.16.0.14:6633 000000101010 }}} assigning the switch the DPID 000000101010. Making it so that the script takes DPID as an argument from the command-line is left as an exercise to the reader. `./of_stop.sh` turns the !OpenFlow components off. == III. NetFPGA Tunneling !OpenFlow switch == This is the L2 tunnel version of the !OpenFlow bitfiles for the NetFPGA. This section is based on notes taken for the following setup: * Ubuntu 11.04 * netfpga_tunneling_openflow_icmp_arp_1_0_0.tar.gz * netfpga_full_3_0_0.tar.gz Setup is described fairly well in the [http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/TunnelingOpenFlowNetFPGA100ICMP NetFPGA documentation], although a few steps necessary to work out kinks from their transitions between versions are not mentioned. * In /opt/netfpga/lib/C/kernel/nf2main.c, line 190 should be changed from {{{ init_MUTEX(&card->state_lock); }}} to {{{ sema_init(&card->state_lock, 1); }}} This reflects changes made to newer kernels (2.6.37 and above) like the one used in 11.04. Note, this is already taken care of in newer NetFPGA code (>3.0.0). * When moving header and bitfiles into the !OpenFlow directory, rename and move them to match start script conventions. In /opt/openflow/hw-lib/nf2: {{{ mkdir sw cp /opt/netfpga/projects/tunneling_openflow_icmp_arp/lib/C/reg_defines_tunneling_openflow_icmp_arp.h sw/reg_defines_tunneling_openflow_switch.h cp /opt/netfpga/bitfiles/tunneling_openflow_icmp_arp.bit sw/tunneling_openflow_switch.bit }}} ---- ---- ''' The following are outdated, use with caution ''' = Setting up NetFPGA hosts: Ubuntu 10.04 = #old Current Ubuntu NetFPGAs run Ubuntu version 8.04 and !OpenFlow ver. 1.0. Here we try to move to newer distributions w/ !OpenFlow Ver. 1.0. The hardware we use here is the NetFPGA cube, another version of the NetFPGA pre-built solution. As per the [http://www.openflowswitch.org/wk/index.php/CentOS_NetFPGA_Install !OpenFlow portion of the setup], a user named 'openflow' with root privs was added for this purpose. Setup of the NetFPGA using earlier distributions can be found in the Diaries. '''Linux distro:''' [[BR]] Ubuntu Server 10.04, w/ xfce4 WM, openssh-server [[BR]] kernel: 2.6.32-21-generic-pae '''Packages:''' [[BR]] NetFPGA base 2.1.0 [[BR]] Tunneling OpenFlow NetFPGA 1.0.0 === Installing Ubuntu. === 1. set boot priority of CD drive to 1st in BIOS 2. If desktop edition, boot as a live CD and install from there. Trying direct install just results in the system proceeding to live CD Mode (10.04). Server edition can be installed directly w/out issues. === I: Setup - NetFPGA drivers. === ==== 1. dependencies. ==== There is an updated dependency list: {{{ build-essential ncurses-dev libnet1-dev libxml-simple-perl libio-interface-perl liblist-moreutils-perl liberror-perl libnet-rawip-perl sun-java6-jre sun-java6-jdk libpcap0.8-dev }}} '''Installing java packages.''' In 10.04, sun-java6-jre, jdk are obsoleted and moved to the partner repository. This repository can be added w/ the following (ref: https://help.ubuntu.com/community/Repositories/CommandLine): 1. copy "deb !http://archive.canonical.com/ lucid partner" sans quotes to /etc/apt/sources.list 1. run apt-get update. 1. run apt-get as usual for the packages. ==== 2. Kernel memory allocation. ==== Ubuntu versions beyond 8.04 (assume anything w/ kernels beyond v 2.6.28) use GRUB 2. GRUB 2 gets rid of menu.lst, so kernel memory must be allocated via the following method (ref. http://ohioloco.ubuntuforums.org/showthread.php?p=8229620): Add "vmalloc=512m" to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub so it looks like this: {{{ GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vmalloc=512m" }}} Then run `update-grub` to regenerate /boot/grub/grub.cfg with the new parameter (you never want to modify grub.cfg directly). ==== 3. Symlinks. ==== Download libpcap.so.0.9.4, found below. Move it into /usr/lib. Run ubuntu_setup.sh (also attached below), or manually create the following links: {{{ ln /usr/bin/lspci /sbin/lspci ln /usr/bin/setpci /sbin/setpci ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0.9 ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0 ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so }}} ==== 4. Edit files. ==== Download Netfpga code: http://netfpga.org/foswiki/NetFPGA/OneGig/Releases ----- * edit "usleep 250" to "sleep 0.00025" in /test_loopback_drop/run.pl * edit /home/netfpga/NF2/projects/router_kit/sw/rkd.cc to include limits.h '''NOTE: this edit is no longer required for version 2.1.1 of the package. ''' ==== 5. Setting Variables. ==== After setting NF_ROOT to the netfpga directory, source `nf_profile`, located in ${NF_ROOT}/bin/ {{{ export NF_ROOT=/home/openflow/netfpga #(wherever your netfpga base files are) source ${NF_ROOT}/bin/nf_profile }}} ==== 6. Moment of truth. ==== `make` then `make install`. [[BR]] Then, check for nf2 module with `lsmod | grep nf2`: {{{ nf2 13425 0 }}} and for nf2cn, n=0:3 with `ifconfig -a`. {{{ nf2c0 Link encap:Ethernet HWaddr 00:4e:46:32:43:00 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 nf2c1 Link encap:Ethernet HWaddr 00:4e:46:32:43:01 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 nf2c2 Link encap:Ethernet HWaddr 00:4e:46:32:43:02 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 nf2c3 Link encap:Ethernet HWaddr 00:4e:46:32:43:03 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 }}} ==== 7. add the cpci reprogrammer to startup routine ==== Add the following to /etc/rc.local: {{{ /home/openflow/netfpga/lib/scripts/cpci_reprogram/cpci_reprogram.pl --all }}} The selftest ''does not'' work in Ubuntu. so don't try it. == II: Setup - !OpenFlow components. == The !OpenFlow wiki on CentOS NetFPGA setup were followed in conjunction with the netfpga site wiki for the Tunneling !OpenFlow NetFPGA. Stes are reiterated here since nothing is permanent. * [http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/TunnelingOpenFlowNetFPGA100 Tunneling !OpenFlow NetFPGA ] * !OpenFlow setup on [http://www.openflowswitch.org/wk/index.php/Ubuntu_Install Ubuntu] and [http://www.openflowswitch.org/wk/index.php/CentOS_NetFPGA_Install CentOS] ==== 1. dependencies. ==== {{{ autoconf libtool pkg-config linux-source git-core automake m4 }}} ==== 2. !OpenFlow package installation ==== This happens in the same directory as the netfpga install. From git repository (automatically pulls the newest version): {{{ git clone git://openflowswitch.org/openflow.git cd openflow ./boot.sh }}} To make NetFPGA compatible: {{{ ./configure --enable-hw-lib=nf2 }}} Remove avahi-daemon and disable IPv6: {{{ sudo apt-get remove avahi-daemon }}} in /etc/sysctl.conf add {{{ net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 }}} and in /etc/modprobe.d/blacklist.conf {{{ blacklist net-pf-10 blacklist ipv6 }}} ==== 3. NetFPGA tunnel package ==== Download and untar in `/home/openflow/`. Move `tunneling_openflow_switch.bit` and `reg_defines_tunneling_openflow_switch.h` to your_openflow_directory/hw-lib/nf2/. The files should be located under `bitfiles/` and `/projects/tunneling_openflow_switch/lib/C/` in the netfpga directory, respectively. Do a make and make install in the openflow directory. ==== 4. Regression test ==== The setup should be wired as such {{{ eth1 <----> nf2c0 eth2 <----> nf2c1 }}} The test can be run with the following command: {{{ nf_regress_test.pl --project tunneling_openflow_switch }}} As of now, getting this error: {{{ root@node1-2:~/netfpga/lib/scripts/cpci_reprogram# /home/akoshibe/netfpga/bin/n_regress_test.pl --project tunneling_openflow_switch NetFPGA environment: Root dir: /home/openflow/netfpga Project name: tunneling_openflow_switch Project dir: /home/openflow/netfpga/projects/tunneling_openflow_switch Work dir: /tmp/root Root directory is /home/openflow/netfpga Running tests on project 'tunneling_openflow_switch'... Running global setup... FAIL Output was: NetFPGA environment: Root dir: /home/openflow/netfpga Project name: tunneling_openflow_switch Project dir: /home/openflow/netfpga/projects/tunneling_openflow_switch Work dir: /tmp/root Found net device: nf2c0 Error: Virtex design compiled against a different CPCI version Active CPCI version : 4 (rev 1) Device built against: 3 (rev 4) Bit file built from: nf2_top_par.ncd Part: 2vp50ff1152 Date: 2010/ 5/ 4 Time: 15:49:32 Error Registers: 0 Good, after resetting programming interface the FIFO is empty Download completed - 2377668 bytes. (expected -1). DONE went high - chip has been successfully programmed. CPCI Information ---------------- Version: 4 (rev 1) Device (Virtex) Information --------------------------- Project directory: tunneling_openflow_switch Project name: Tunneling OpenFlow Switch Project description: Tunneling OpenFlow Switch Device ID: 0 Version: 0.1.0 Built against CPCI version: 3 (rev 4) Download Failed: 256 global/setup exited with value 1 }}} === bitfile issues === cpci_reprogram utilizes bitfiles located in /usr/local/netfpga/bitfiles. this is due to the hard-coded path in the cpci Perl scripts. The bitfile versions must match between all cpci.bit and any project bitfiles involved, or else it will throw the above errors. The bitfiles attached should all work - Download the tarball and extract in /usr/local/netfpga/bitfiles. [[BR]] [[BR]] [[BR]] [wiki:Internal/OpenFlow/ return to OpenFlow index]