Version 24 (modified by seskar, 7 years ago) ( diff )

Table of Contents

    Software Defined Networking

    Experimentation in ORBIT

    Sandbox 9 is a great option for OpenFlow/SDN experimentation.

    As shown in figure below, SB9 is built around an OpenFlow capable switch, a Pronto 3290, with 11 experimentation nodes connected to it. Seven of these nodes (node1-1 through node1-7) have NetFPGA 1G cards, two (node1-8 and node1-9) have NetFPGA 10G cards, while the other two are general purpose ORBIT nodes that could be used as controllers or monitors. The switch provides the 'DATA' backplane for SB9 with the 'eth0' interfaces on each node and the 4 data ports on each of the NetFPGAs connected to it at the specific ports shown in the figure below. The control interfaces on the nodes (i.e., 'eth1' ) are connected to an external control switch not accessible to the experimenter (primarily used for node imaging and node access from console).
    Figure 1: Overview of the SB9 setup.

    OpenFlow Operation Modes for P-3290

    The P-3290 can operate in 2 basic modes: as a L2 switching/L3 routing element using Pica8's PicOS protocol stack, or as a multi-layer virtual switch enabled by porting open vSwitch (OVS), where the control path is open to experimentation. In addition, since the U-Boot bootloader on the switch can load custom images via NFS/TFTP, one can load any compatible switch OS solutions (e.g., Indigo), presenting a third path for OpenFlow experimentation.

    A. Openflow in L2/L3 Mode

    As of PicOS 2.0, the switch can mix data traffic between the OpenFlow and L2/L3 networks. They call this enhancement “crossflow” mode. Individual ports can be designated either as legacy or crossflow port, where the control on the legacy ports is handled by PicOS L2/L3 stack. The control for crossflow ports can be left to one of local handling (by PicOS stack), by flow rules entered through CLI, or by establishing path to an OpenFlow controller.

    Hardware structures used in crossflow modes in P-3290
    Figure 2: Hardware structures used to store flow rules in the TCAM and L2/L3 crossflow modes in P-3290

    There are two further distinct modes within the crossflow mode, L2/L3 mode and TCAM mode, that use different hardware lookup structures during forwarding. If you enable L2/L3 mode, all flows will be installed in the routing table and the FDB table in the hardware. If you enable TCAM mode, all flows will be installed in the TCAM table. In each of these sub-modes, there are limitations on what flow patterns can be specified - refer to the attached configuration guide for these constraints.

    Preparing the switch for OpenFlow mode includes: assigning h/w resources for OpenFlow flows, enabling OpenFlow mode and enabling specific versions (e.g., 1.1, 1.2 or 1.3), setting ports as crossflow, and lastly adding the 'match' and 'action' rules for expected flow. As an alternative to locally introduced flow rules using the CLI, a remote OpenFlow controller can be configured which will then receive OpenFlow notifications (PacketIns) for flows not handled locally. The following CLI session excerpt details the commands required to accomplish the above steps:

    -- 0. Load a 'clean' configuration to nullify changes made by previous experimenters --
    -- 1. In configure mode, set tcam/routing table resources
    XorPlus# set interface max-acl-rule-limit ingress 400 
    XorPlus# set interface max-route-limit 6000 
    -- 2. Commit configuration and restart switch for resource allocation to take hold --
    XorPlus# save running-to-startup
    Save done.
    XorPlus# exit
    XorPlus> request system reboot
    -- 3. In config mode, enable OpenFlow operation and set the working mode to L2/L3 or TCAM mode --
    XorPlus# set open-flow
    XorPlus# set open-flow working-mode tcam-mode 
    -- 4. Enable specific OF version, e.g., v 1.3
    XorPlus# set open-flow allowed-versions openflow-v1.3 disable false
    -- 5. Set required ports in crossflow mode, with or without local control
    XorPlus# set interface gigabit-ethernet ge-1/1/1 crossflow enable true
    XorPlus# set interface gigabit-ethernet ge-1/1/2 crossflow enable true
    XorPlus# set interface gigabit-ethernet ge-1/1/2 crossflow local-control false
    -- 6. Add flow rules directly from CLI for ports with local control enabled
    XorPlus# set open-flow flow f1 match-field ethernet-destination-address 22:22:22:22:22:22
    XorPlus# set open-flow flow f1 match-field vlan-id 100
    XorPlus# set open-flow flow f1 match-field ip-destination-address
    XorPlus# set open-flow flow f1 match-field ethernet-type 2048
    XorPlus# set open-flow flow f1 action output interface ge-1/1/1
    -- 7. Configure controller IP/port/transport for non-local control
    XorPlus# set open-flow controller contr-serv address
    XorPlus# set open-flow controller contr-serv port 6633
    XorPlus# set open-flow controller contr-serv protocol tcp 
    XorPlus# commit
    Waiting for merging configuration.
    Commit OK.
    Save done.
    XorPlus# exit

    B. OpenFlow in OVS Mode

    C. OpenFlow through Network Boot (of custom image)

    Switching Modes using Web Service

    Note: The web service is being upgraded to conform to the latest ORBIT Management Framework software, hence the switch will remain in the L2/L3 mode at all times. This means the OVS and NFS modes are presently unavailable.

    Access to Switch CLI

    You can access switch CLI through a ssh session, if the switch is in either of L2/L3 or OVS modes:

    ssh hostname/IP:
    username: sb9user
    passwd: sb9passwd

    Note: ssh access is restricted to from within the sb9 sandbox (incl. the sb9 console) and would therefore require the experimenter to have reserved SB9 to access the switch CLI.

    Loading and Saving Switch Configurations

    While in L2/L3 mode, the XorPlus CLI under the above account grants complete control to configure the switch and also to load/store configuration files on the root file system. However, it is suggested that experimenters preserve their configs by copying out their configurations (using tftp, etc.) to external locations, and to load them back when required. Below are the relevant CLI commands with further details in the attached PicOS configuration guide from Pica8. Please exercise prudence and caution when saving or moving configuration files in/out of the root file system.

    -- retrieve config file from external location; scp can be used in place of tftp
    XorPlus> file tftp get remote-file basic.conf local-file /pica/config/sb9user/iamxyz-mm-dd-yyyy.conf <server IP>
    XorPlus> configure
    -- load locally stored config file 
    XorPlus# load iamxyz-mm-dd-yyyy.conf
    -- make any number of changes and commit
    -- save current configuration changes to specific file
    XorPlus# save iamxyz-latest.conf
    Save done.
    -- transfer out config file to safe location for later use
    XorPlus# exit
    XorPlus> file tftp put local-file /pica/config/sb9user/iamxyz-latest.conf remote-file updated.conf <server IP>

    External References:


    Attachments (3)

    Note: See TracWiki for help on using the wiki.