| Version 14 (modified by , 15 years ago) ( diff ) |
|---|
Notes
This section covers discussions and some background work that went into this project.
Overview
The things covered here are:
- DHCP relay agents
- VLANS
- Trunking
VLANs
VLANs are a way to split up broadcast domains at L2. They can be statically or dynamically set, with dynamic VLANs sort of working in a similar way to DHCP.
DHCP Relay agents
Relay agents are virtual layer 3 devices residing on a switch with VLANs. In this case, they allow DHCP broadcasts to be relayed across 2 VLANs on the same switch. This is not necessary for us because the DHCP server lives beyond the ASA, and the ASA not only acts as a router between the switch and the server, but also serves to manipulate the VLAN tags (masquerades as hosts sending out DHCP requests).
Trunking
Trunking is done by making a "backbone" for all the different VLANs on a local switch to converge to. Frames from different VLANs are identified bythe means of VLAN tags, a four-byte addition to the Ethernet frame identifying which VLAN it came from. The tagging is also known as 802.1q.
Native VLANs
Normally the 4 byte addition to the Ethernet frame would cause the tagged frame to be rejected on a VLAN, and regular frames would be rejected if found in the trunk due to the lack of a tag. This is unless the "regular" frame is from a native VLAN , whose frames are intended not to have tags for devices that don't understand tagging.
A site explaining what a native vlan is: http://networkers-online.com/blog/2008/06/native-vlan-explained/
In our case this matters because DHCP coming from the trunk needs to assign IP addresses to both the hosts connected to the switch on different VLANs (VLAN 27) and to the switch itsself. To do this, The virtual interface to which the IP address is assigned needs to be on the trunked port(s).
some thoughts regarding th virtual switches (6/9)
- They are overlay devices that "run" on top of VLANs. VLANs need to be set up before virtual switches can be created.
- They are identified by the VLAN number of the VLAN they overlay, so one virtual switch can't encompass more than 1 VLAN.
- Conversely, they can overlay part of a VLAN. In that case, the whole VLAN takes on the virtual switch's behavior.
The last point is somewhat annoying, because the default behavior of a controllerless virtual switch is to "wait till the flow table times out." That would mean "become nonfunctional" in our case, since the virtual switches start off controller-less, and hence, flow-less. The whole switch would become a virtual brick if the trunk is specified in setvsi. This was demonstrated by overlaying a virtual switch over a single port on VLAN 27. It stopped forwarding DHCP messages all together when it used to be the only VLAN with DHCP.
Questions:
- can a virtual switch be loaded with a default flow table that would allow it to function, at least until its table times out?
when openflow.conf gets messed up (6/22)
One method to rewrite the OpenFlow configuration is to delete openflow.conf directly through the switch, then use the setvsi command to set the new virtual switches up. This should be unnecessary, but it happened to be effective when old configurations did not seem to go away despite editing the .conf file using a PC.
> cd /mnt/ > rm openflow.conf remove 'openflow.conf'? y > setvsi 1 1,3,5,7,9,11,13,15,2.1 tcp 172.16.4.224 dpid 0x0123456789ab > setvsi 2 17,19,21,23,25,27,29,31,2.2 tcp 172.16.4.180 dpid 0x012345678abc > setvsi 3 33,35,37,39,41,43,45,47,2.3 tcp 172.16.4.64 dpid 0x01234567abcd > showswitch vlan ports secure channel ---- ----- -------------- 1 1, 3, 5, 7, 9, 11, 13, 15, 2.1 disconnected 2 17, 19, 21, 23, 25, 27, 29, 31, 2.2 disconnected 3 33, 35, 37, 39, 41, 43, 45, 47, 2.3 disconnected
However, it seems like it is not possible to set double-wide-mode, or no-save through the CLI. These have to be added by editing openflow.conf using the PC.
SSL setup, things to know for the next day. (7/13)
using this site as reference: http://www.debian-administration.org/article/Creating_and_Using_a_self_signed__SSL_Certificates_in_debian
except renaming the output .pem files to what it would understand:
*ca_cert.pem - from cacert.pem *sw_cert.pem - from cert.pem *sw_key.pem - from key.pem
for the Common Name, I just used the IP address of the CA, 192.168.203.75.
because in the example on the site ca_cert.pem is named cacert.pem, I had to change "cacert" to "ca_cert" in openssl.cnf for the very last part (signing the certificate):
[ CA_default ] serial = $dir/serial database = $dir/index.txt new_certs_dir = $dir/newcerts certificate = $dir/ca_cert.pem #change over here private_key = $dir/private/cakey.pem default_days = 365 default_md = md5 preserve = no email_in_dn = no nameopt = default_ca certopt = default_ca policy = policy_match
I am not sure if changing the name of the key from ofpswitch.key.pem to sw_key.pem after making/signing the certification will affect anything.
Some lessons for the day:
- the SD card will show up as /media/disk on the PC
things to do:
- control VLAN on switch
- config control VLAN and statically assign PC to controller IP
- VLAN not used for any legacy networking purposes for controller
- packet sniff SSL handshake
VLAN 888
arbitrary VLAN for OFP controller, since nothing probably uses that high a VLAN number. Currently only port 0/42 (formerly one of the trunk ports)
sw-sb09(config)# vlan 888 !sw-sb09(config-vlan)# name "OpenFlow control VLAN" !sw-sb09(config-vlan)# interface gi 0/42 !sw-sb09(config-if)# sh interface gigabitethernet 0/42 switchport mode trunk switchport trunk allowed vlan 1,3,27-28 switchport trunk native vlan 1 ! !sw-sb09(config-if)# no sw mo tru !sw-sb09(config-if)# no switchport trunk allowed vlan 1,3,27-28 !sw-sb09(config-if)# no switchport trunk nat vlan 1 !sw-sb09(config-if)# sh interface gigabitethernet 0/42 switchport mode access ! !sw-sb09(config-if)# sw acc vlan 888 !sw-sb09(config-if)# interface vlan 888 !sw-sb09(config-if)# ip address 172.16.4.1 255.255.255.0 !sw-sb09(config-if)# save sw-sb09(config-if)#
this will probably not need a route specified for it since the controller is directly attached to the switch.
formal controller VLAN: VLAN100. (6/14)
VLAN 888 was removed since VLAN 100 is the formal controller VLAN, as decided by discussion.
sw-sb09(config-if)# sh interface gi 0/42 interface gigabitethernet 0/42 switchport mode access switchport access vlan 888 ! sw-sb09(config-if)# interface gigabitethernet 0/42 sw-sb09(config-if)# no sw acc vlan 888 !sw-sb09(config-if)# switchport mod trunk !sw-sb09(config-if)# swi trunk all vlan 1,3,27,28 !sw-sb09(config-if)# swi tru nat vlan 1 !sw-sb09(config-if)# save sw-sb09(config-if)# no vlan 888 !sw-sb09(config)# vlan 100 !sw-sb09(config-vlan)# name "OpenFlow control VLAN" !sw-sb09(config-vlan)# save
Testing throughput with Iperf (7/23)
config eth0 of nodes 1-1 and 1-2, then run iperf to test the difference in bandwidth between OpenFlow and regular firmware. iperf reference: http://openmaniak.com/iperf.php
node1-1: 192.168.1.1/24
eth0: negotiated 1000baseT-HD flow-control
eth1: negotiated 1000baseT-HD flow-control
node1-2: 192.168.1.2/24
eth0: negotiated 1000baseT-HD flow-control
eth1: negotiated 1000baseT-HD flow-control
Server: Node1-2 Client: Node1-1
OpenFlow mode
| Transfer | Server | Client | |
| 1 | 278 MBytes | 230 Mbits/sec | 230 Mbits/sec |
| 2 | 291 MBytes | 243 Mbits/sec | 244 Mbits/sec |
| 3 | 291 MBytes | 244 Mbits/sec | 244 Mbits/sec |
| 4 | 290 MBytes | 243 Mbits/sec | 244 Mbits/sec |
| 5 | 284 MBytes | 231 Mbits/sec | 231 Mbits/sec |
Normal mode
| 1 | 278 MBytes | 229 Mbits/sec | 230 Mbits/sec |
| 2 | 291 MBytes | 228 Mbits/sec | 228 Mbits/sec |
| 3 | 291 MBytes | 228 Mbits/sec | 229 Mbits/sec |
| 4 | 290 MBytes | 231 Mbits/sec | 243 Mbits/sec |
| 5 | 284 MBytes | 229 Mbits/sec | 230 Mbits/sec |
Server: Node1-1 Client: Node1-2
OpenFlow mode
| 1 | 278 MBytes | 229 Mbits/sec | 230 Mbits/sec |
| 2 | 291 MBytes | 229 Mbits/sec | 229 Mbits/sec |
| 3 | 291 MBytes | 232 Mbits/sec | 244 Mbits/sec |
| 4 | 290 MBytes | 230 Mbits/sec | 230 Mbits/sec |
| 5 | 284 MBytes | 230 Mbits/sec | 230 Mbits/sec |
Normal mode
| 1 | 278 MBytes | 243 Mbits/sec | 244 Mbits/sec |
| 2 | 291 MBytes | 243 Mbits/sec | 242 Mbits/sec |
| 3 | 291 MBytes | 243 Mbits/sec | 236 Mbits/sec |
| 4 | 290 MBytes | 243 Mbits/sec | 244 Mbits/sec |
| 5 | 284 MBytes | 242 Mbits/sec | 243 Mbits/sec |
Overall, there doesn't seem to be too much difference in performance.
more realtime throughput testing (7/30)
Using a tool called BWM-NG (Bandwidth Monitor NG): http://www.gropp.org/?id=projects&sub=bwm-ng
the tool can output logs as a csv, which can be parsed with some more practice-Ruby code. Hopefully will get Scruffy or something working so the data can be graphed.
BWM-NG was installed on SB9 with command apt-get install bwm-ng. bwm-ng --help gives you all the options you have. For csv output of tests for all eth1 interfaces at 1sec intervals:
bwm-ng -I eth1,eth1.27,eth1.28,eth1.100 -o csv -t 1000 -F <file-name>
Three tests were done:
- nothing connected, no controller active
- controller active
- activate 2 nodes, ping one from the other with 65500 bit payload
- controller active, virtual switch connected to LAN (WINLAB network)
socket programming in Ruby (8/1)
Some resources found:
- IBM tutorial: https://www6.software.ibm.com/developerworks/education/l-rubysocks/l-rubysocks-a4.pdf
- TCP client/server (in C): http://devmentor.org/articles/network/Socket%20Programming.pdf
having issues with trying it, getting this error:
A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied. - getpeername(2) (Errno::ENOTCONN)
supposedly meaning I'm not getting the connection to happen right
idea for OpenFlow: use code from reference model to simulate a handshake, display on a webpage or something else, to get a sense of how to integrate things
Anatomy of OpenFlow Protocol (8/3)
OpenFlow Switch has a paper you can read for information on how the protocol works. The initial handshake is the important piece of info that makes a switch communicate with what it considers a controller.
The Handshake
The OFP handshake contains the following steps:
- OFPT_HELLO message - sent by both parties to negotiate what version of OpenFlow to use
- SSL connection (optional)
- OFPT_FEATURES_REQUEST - sent by controller
- OFPT_FEATURES_REPLY - returned by switch
- ECHO reply/request or some flow actions
The steps need to be recreated, but there are several things that need to be done:
- when creating a socket in Ruby you need a port number, but it is a passive connection (ptcp) - the port can be anything
- pick apart either OF reference system or NOX for handshake code
picking at OFP (8/4)
OpenFlow data structure is defined under /include/openflow/openflow.h .
Whether it is NOX or the !Openflow Regression Suite it is all in lots of C. Trying to make Ruby work with C seemed a bit too messy for our own good, so we sent an email to the NOX developer's mailing list in hopes of some elucidation of how to create a Ruby interface to OpenFlow, instead of Python (which is what NOX does, as described here)
Some giant lapse in time, with a pinch of SNMP (9/21)
So far:
- have some Ruby code that interrogates and configures SNMP agents and retrieves OFP info.
- mild understanding of net-snmp
- tutorial to get started with SB9
- Seething suspicion that Bob's firmware is preventing reassignment of vlans to port (see below)
To-do:
- find a way to snmp-set vlan assignments to ports
- maybe write OFP remote config code
- test code on other devices
Some links:
