System Config:

Ubuntu Version: Ubuntu 9.04
Netfpga Version: 2.1.0
Tunneling Openflow Switch: 2.1.1
Openflow Version: 1.0.0
flowvisor Version: Alpha version 0.4

Note: Bitfiles come from some other package and must be made to agree with the versions of cpci tools and nf2 kernel module

Basic Usage: Assuming the machine is freshly rebooted the script:

/opt/netfpga1.0.1/netfpga/projects/tunneling_openflow_switch/sw/of_start_winlab.sh

Should be run as root

Known issues:

1. When Running the of_start_winlab.sh script wake up the netfpga into a tunneling openflow mode we get this error from the script: Virtex design compiled against active CPCI version

   RTNETLINK answers: No such file or directory
   RTNETLINK answers: No such file or directory
   RTNETLINK answers: No such file or directory
   RTNETLINK answers: No such file or directory
   ARP_REPLY_ETH_ADDR_PORT_3_LO_REG=0x46324402
   

   However it still works.
2. After a few minutes of operation the nf2 modules starts dumping buffer messages into the kernels ring buffer. The messages are:

   [ 1161.690033] nf2: no available transmit/receive buffers
   [ 1161.690061] nf2: no available transmit/receive buffers
   [ 1161.690089] nf2: no available transmit/receive buffers
   

   We don't know what these errors are, and they are suspected to eventually brining down the tunnel (and thus requiring a reboot). That being said, the tunnel does come up inspite of it. - SOLVED 6/2/2010
3. The tunnel still dies, when I tried to bring it back up the interfaces took a little time to actually wake up, but it eventually got to where it needed to be.
4. Packets Only flow in one directon - SOLVED 6/2/2010
5. We were seeing flows that persisted even after the controller died. There was however a good explanation for this. There are two types of time outs, A hard time out that ends a flow after some fixed time regardless of activity, and a soft time out (expiration) that ends the flow if it has been idle for too long. In it default behavior Nox/Snac, No hard time out is set. So if enough packets arrive the flow will stay alive forever, even in the absence of a controller. We should probably set the hard timeout to some known number. does not set a hard timeout. If enough packets arrive the flow will never time out.
6. Not so much an issue as an FYI - When using tcpdump on the netfpga, we don't see all the traffic that is passing through. Tcpdump is basically only useful for catching OpenFlow messages as they come through the control port. Trying to read the traffic on one of the NF2CX ports will probably Not show what you're looking for. The Quote from Tatsuya (standford)

      Tcpdump only captures software forwarding packets(i.e. packet-ins and packet-outs). So most of the packets handled by hardware are not shown.
   

---

6/2/2010

We've made some progress in getting the tunnel to work. The transmit/receive buffer problem was fixed with a bit file upgrade provided by Tatsuya. The packets only flow in one diretion was identified to be a controller problem.

My original setup worked once I manually added flows to the ofprotocol module via the dpctl command:

On OF1 connect ports 2 and 3 to 4 with vlans 3,29 and 30.
dpctl add-flow unix:/var/run/test in_port=4,dl_vlan=0x0003,idle_timeout=-1,actions=output:2,3
dpctl add-flow unix:/var/run/test in_port=4,dl_vlan=0x001d,idle_timeout=-1,actions=output:2,3
dpctl add-flow unix:/var/run/test in_port=4,dl_vlan=0x001e,idle_timeout=-1,actions=output:2,3
dpctl add-flow unix:/var/run/test in_port=2,dl_vlan=0x0003,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=2,dl_vlan=0x001d,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=2,dl_vlan=0x001e,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=3,dl_vlan=0x0003,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=3,dl_vlan=0x001d,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=3,dl_vlan=0x001e,idle_timeout=-1,actions=output:4

On OF2 connect ports 1 to 4 with vlans 3,29 and 30.
dpctl add-flow unix:/var/run/test in_port=1,dl_vlan=0x0003,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=1,dl_vlan=0x001d,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=1,dl_vlan=0x001e,idle_timeout=-1,actions=output:4
dpctl add-flow unix:/var/run/test in_port=4,dl_vlan=0x0003,idle_timeout=-1,actions=output:1
dpctl add-flow unix:/var/run/test in_port=4,dl_vlan=0x001d,idle_timeout=-1,actions=output:1
dpctl add-flow unix:/var/run/test in_port=4,dl_vlan=0x001e,idle_timeout=-1,actions=output:1

Note the difference between how the openflow protocol identifies the ports and how the Ethernets tools identify the ports. Ethernet numbers begin with 0, openflow begins with 1.

Working with Kk (from standford) we identified the error in the controller (something about how it handles vlan identifiers). The fix was to use his bleeding edge version of nox. The install steps go a little like so:

Build instructions:

First run: 

apt-get install git-core buildessential 

Then get the source and some required libaries:

git clone git://noxrepo.org/nox
sudo apt-get install autoconf automake g++ libtool python python-twisted swig libboost-dev libxerces-c2-dev 
libssl-dev make libboost-filesystem-dev libboost-test-dev python-dev 
git checkout -b destiny origin/destiny

Run a build:
./boot.sh
mkdir build0.8
cd build0.8
../configure
make

This will build the destiny branch (latest and greatest)

in the build dir run:

./nox_core -v -i ptcp:6633 routing

or

./nox_core -v -i ptcp:6633 routing lavi_switches lavi_swlinks &

for some debugging tools. 

8/5/2010

We have a some what sucessfull tunnel deployed between CS and winlab. The attached diagram contains all the notworth IP's. We're Isolating the TFTP problem now. Our colleagues at standford suggest trying nox 0.6. James is going to build it on nox.orbit-lab.org.

I've followed the steps from the previos build of nox. Lets see if I get stuck.

Apparently a few things have changed. The nox core executable is now located the src subdirectory of the build directory. And there is some nox.xml file thats required now. If we installed it would live in the /etc directory. However you can change the path (in case you just want to run from the build directory) with the -c flag. I ran this command from the /opt/kknox/build/src

./nox_core -c ./etc/nox.xml -v -i ptcp:6633 routing