wiki:Internal/OpenFlow/FloodlightFVModule

Version 20 (modified by akoshibe, 12 years ago) ( diff )

—

An attempt at Floodlight Development.

This page documents the (attempted) process of developing a version of Floodlight with FlowVisor-like slicing capabilities.

This is an on-going process - therefore this page will be updated frequently.

Overview
Setup
Background
Approach - a diary-style log of the implementation. Following are some 'highlights' relevant to the software being ported in general:

jump to bottom.

Overview.

Slicing is the SDN (in our case referring to OpenFlow) term for network virtualization. Specifically, A slice is a SDN controller and the network resources allocated to it by a hypervisor-like entity such as FlowVisor. With proper resource isolation and allocation, multiple controllers can coexist on a single physical network.

In the usual case, FlowVisor behaves as a proxy, intercepting and modifying the OpenFlow control messages between the switches and the multiple controllers so that each slice only sees a subset of the network.

Floodlight is a modularized OpenFlow controller, in that its functional components can be split into two major parts:

  1. The core event dispatcher, and
  2. The various event handlers.

The event handlers (Floodlight modules) are what define the behavior of swicthes (hub, learning switch, router, etc.). A more in-depth discussion of these components can be found ​here.

The key point here is that Floodlight's modular structure allows for multiple different handlers to coexist, with various (groups of) modules imposing different capabilities on the switches. If these modules controlled different groups of switches e.g. had their own bit of resource, each <module:resource> set can be viewed as a slice. Floodlight does not currently have this capability, so the objective of the process described here is to try to realize this feature.

Setup

Development is done on two VMs, one for Flowvisor and Floodlight source, and the other, Mininet for testing our code.

  • VM1(10.50.250.2): 3-switch, 3-host Mininet topology pointed to external controller in VM2:

Topology: 

h4   h5   h6
 |    |    | 
s1---s2---s3

Mininet config: 

# mn --topo=linear,3 --controller=remote --ip=10.50.250.17 --port=6633
  • VM2(10.50.250.17): Flowvisor on port 6633, with a slice to point two switches to Floodlight instance on 6634

FlowVisor configuration: 

# fvctl createSlice fl-1 tcp:localhost:6634 foo@sampledomain.org
# fvctl addFlowSpace 00:00:00:00:00:00:00:02 1000 any "Slice:fl-1=7"
# fvctl addFlowSpace 00:00:00:00:00:00:00:01 1000 any "Slice:fl-1=7"

Floodlight config alteration (in src/main/resources/config.properties): 

net.floodlightcontroller.restserver.RestApiServer.port = 8088
net.floodlightcontroller.core.FloodlightProvider.openflowport = 6634

The REST API port is changed from 8080 to prevent conflict with Flowvisor's services.

Background.

The general flow of operation with respect to Flowvisor (in terms of components) is the following:

  1. The Acceptor (OFSwitchAcceptor) listens for all OpenFlow message types (OFTypes) and all switch join/leave events.
  2. When a new switch joins, the Acceptor hands the switch's connection to a Classifier (FVClassifier).
  3. The Classifier fetches all slices associated with the switch and launches a Slicer (FVSlicer) per module.
    • mappings of DPID to slices are found through configurations (the Flowmap)
    • modules associated with the control module (the primary module defining the behavior of a switch for that slice) may be grouped together in a slice.
  4. Each Slicer sets up event dispatching for the modules associated with a slice.

Where the messages go once Flowvisor receives a message are determined by the FlowMap. the origin of a message can be determined several ways, one being checking the OFType (as some messages are sent by switches but never controllers, and vice versa).

Flowvisor implements all of the needed controller functions, so a good chunk of it is life-support (event handling mechanism, timers, OpenFlow signaling, etc.). Therefore, we need to be aware of what Floodlight already provides, and what can/needs-to be pulled in from Flowvisor.

A Floodlight module, in general:

  • can subscribe to switch join/leave events. They are notified of joins after the switch sends a features reply.
  • can subscribe to all OFTypes, and once processed, can choose to pass it down the processing chain to other modules or drop it.
  • the modules themselves cannot control which modules receive the messages if it choses to pass them downstream.

Approach (and implementation).

(6/12):

We generally believe that the following parts will need to be implemented:

  1. A broker module to register with the core module for all events - the core module can notify this module of OpenFlow messages and new switches.
  2. A component that provides the IFloodlightProviderService interface to the rest of the modules. To the rest of the modules, the broker module will appear to be the core controller module.
  3. Flow-mapping mechanism - it is possible that the components needed for a working FlowMap can be taken directly from Flowvisor, since it is relatively independent from the rest of the code base (according to Ali).

Therefore, the module is both a module in Floodlight's usual sense as a subscriber to the controller core, but also a controller that services the rest of the modules. The rough parallels are:

  • the main broker module's behavior as a listener roughly corresponds to the OFSwitchAcceptor
  • the new IFloodlightProviderService interface (call it "FVController"?) and receive() functions correspond to the FVSlicers and FVClassifiers
  • the modules correspond to the multiple controllers

The FVController interface should be responsible for keeping track of which modules are associated with which slice, and which messages reach what (with help from the FlowMap). This means that the modules must register with it, as opposed to the core controller defined in Controller.java.

So, for the meanwhile, the next steps include:

  • isolating out the FlowMap code in Flowvisor
  • mapping out the module registration mechanisms in Floodlight
  • finding a "nice" way to make the modules register with FVController.

(6/13):[Floodlight Module System]

A (not-so) quick revisit of the module loading system in Floodlight was done to figure out how exactly a module learns to subscribe to the main controller module.

Starting at the top, in the init() function of each module is almost always this line: 

floodlightProvider = context.getServiceImpl(IFloodlightProviderService.class);

Where context is an instance of FloodlightModuleContext, which contains a mapping between the interfaces and the services providing them. getServiceImpl(), in the case above, returns the service providing the IFloodlightProviderService interface, which is the core controller.

FloodlightModuleLoader is the actual class responsible for populating FloodlightModuleContext's mapping; Recalling, from Main.java: 

FloodlightModuleLoader fml = new FloodlightModuleLoader();
IFloodlightModuleContext moduleContext = fml.loadModulesFromConfig(settings.getModuleFile());

Internally, for each module exporting a service, in initModules() FloodlightModuleLoader calls getServiceImpls(), an IFloodlightModule function that lets modules define the services that they export. It then uses FloodlightModuleContext's addService() in order to fill its mapping.

Returning to the controller mapped to IFloodlightProviderService.class, we see that it is a service exported by the module FloodlightProvider. It is the first module to be loaded by the module loading system, and therefore the mapping to the controller is there for the rest of the modules to register with.

This leads us to a potential way to place a module between the controller and the rest of the modules:

  • Re-name the real IFloodlightProviderService to something else
  • Implement a module that subscribes to the re-named controller service
  • Implement a FVController from IFloodlightProviderService
  • export this as a service with key IFloodlightProviderService.class

This shouldn't require any change on the other modules (except Controller.java).

(6/14):

A preliminary attempt at trying the above (do not follow if you ever come across this, it is an experiment):

create copies of core files:

  • FVController, proxy version of Controller - for now a blank IFloodlightProxy implementation
  • IFloodlightProxy, a copy of IFloodlightProviderService to be used by FVController

re-wired files:

  • FVController implements IFloodlightProxy, with FVAcceptor as its version of FloodlightProvider

For the following files, 'rewiring' refers to swapping out IFloodlightProviderService with IFloodlightProxy.

  • ForwardingBase, Forwarding (the generic forwarding components loaded by default)

additional changes:

  • Add FVAcceptor to /floodlight/src/main/resources/META-INF/services/net.floodlightcontroller.core.module.IFloodlightModule
  • Have FVAcceptor subscribe to IFloodlightProviderService as its floodlightprovider
  • Incorporate FloodlightProvider's functions into FVAcceptor (for now, brutely drop in, just exporting IFloodlightProxy service)

This process forces the forwarding component to register with FVController, as opposed to the core module. The core module still takes care of OpenFlow signaling e.g. echo keepalives and the initial handshake, so any switches will still connect properly and keep the connection alive.

(6/18):

After much tweaking and code-tracing with log.debug outputs, a (seemingly) working proxy was implemented.

  • FVController: implements its own bare-bones versions of:
    • updateSwitch: track list of active switches
    • add/removeOFMessageListener
    • add/removeOFSwitchListener
    • getSwitches: return immutable list of active switches
    • handleOutgoingMessage
    • injectOfMessage
    • handleMessage
    • init: initialize class variables
  • OFSwitchImpl:
    • a setFloodlightProxy setter function
    • modified write() functions that call either FVController or Controller's handleOutgoingMessage() depending on OFType: 
      switch (m.getType()) {
    case PACKET_OUT:
    case FLOW_MOD:
    case PORT_MOD:	
        this.floodlightProxy.handleOutgoingMessage(this, m, bc);
        break;
    default: 	
        this.floodlightProvider.handleOutgoingMessage(this, m, bc);
}
  • Controller: add functions to set the floodlightProxy in OFSwitchImpl
  • FloodlightProvider: fetch and set Controller's FVProxy in init() 
    controller.setFVProxy(
    context.getServiceImpl(IFloodlightProxy.class));

(6/20):[Flowvisor FlowMap Overview]

The FlowMap is one of Flowvisor's central components that must be ported. The first step, as always, is to understand how its components fit together. Hopefully there will be time for a clear re-write of the banter to follow…

First up, some terms:

  • Flow rule : Mappings of Packet header pattern to actions (DROP, FLOOD, FORWARD, etc.)
  • FlowSpace : A collection of flow rules
  • FlowMap : A uniform interface and structure for manipulating and holding a FlowSpace. A FlowMap also maps a FlowSpace to the proper slices.

A flow rule is defined by a matching criteria (FVMatch.java → OFMatch.java) and an action (OFAction.java derivative). The "traditional" OFMatch construct doesn't do matching on DPID, so it is extended by FVMatch to do so. Flowvisor v0.8.3 supports two types of FlowMaps, Linear and Federated. FlowMaps are implementations of the Java Interface FlowMap.java. The enum MatchType defines how each header pattern of a (Flowvisor-defined) rule and the received rule patterns relate, in semi-set theoretic terms: 

public enum MatchType {
	UNKNOWN, // not yet calculated
	NONE, // no overlap between A and B
	SUBSET, // match A is a subset of B
	SUPERSET, // match A is a superset of match B
	INTERSECT, // match A and B overlap, but neither subsumes the other
	EQUAL; // match A and B are exactly the same
}

The !Matchtype is used to describe Intersections, descriptions of how two FlowEntries relate. An intersection takes the form of a FlowEntry built by choosing the more heavily constrained pattern for each header field and DPID(s). FlowIntersect.java implements the representation of the intersection, and FlowEntry defines matches(), which compares two FlowEntries to produce the intersection. Comparisons are implemented in FlowTestOp.java.

(6/22):

Some thoughts from reading Flowvisor code.

  • FVConfig interfaces database access operations pertaining to Flowspaces, slices, switches, and Flowvisor itself. The DB-related classes are all named *Impl.
  • The handling of messages received by Flowvisor are defined within the messages themselves, as part of the implementation of Classifiable and Sliceable interfaces.
  • The static flow pusher and related components may be eventually useful for these configuration aspects.

(6/25):[Flowvisor Internals, cont'd]

Today's conference call elucidated quite a bit about the workings of the FlowMap and how it fits in with the rest of the system.

Item 1: the usage of config databases.
In Flowvisor, the FlowMap is coupled to the database, but only loosely - the specific implementations are, indeed, hidden behind FVConfig. Furthermore, the database is only accessed when:

  • an FVClassifier initializes connections to controllers (see FVClassifier.java:751)
  • the flowspace changes.

in short, the database is a convenience item for storing configurations, and for initial purposes of this project, something that can likely be dealt with later.

Item 2: FlowMap internals.
The two implementations of interface FlowMap process the flowspace very differently.

The Linear FlowMap takes a literal copy of the flowspace; to save space, the flowspace is carted up by DPID, and the piece representing the 'correct' DPID e.g. matching that of the incoming message, is processed.

The Federated FlowMap manipulates references. If multiple FlowMaps existed, they would all point to the same flowspace. This FlowMap relies on a global bitset, indexed by FlowEntry ID, to keep track of entries that are part of a intersection (1=member, 0=otherwise). Starting with DPID, each field of a OFMatch is taken and compared to those of the flow entries. Each field is stored in its own hashmap, keyed by value. Each field comparison should narrow the number of 1's in the global bitset. At the end, the entry with the highest priority is chosen.

The goal of the Federated FlowMap is speed - many of its operations are kept to bitwise ops, and storage, hashmaps. Slower operations that involve loops e.g. IP address matching, are kept until later when the set of entries to work with is smaller.

Item 3: Next steps.
Now that we are pretty clear on the inner workings of Flowvisor, some next steps to do are:

  • get flowspace working, with config file (ignoring databases for now)

to get this working, we need:

  • JSON parsing in Floodlight
  • ability to load configs to a module
  • exception handling for Flowvisor-related errors.

(6/28)

Not all information used by Flowvisor will be applicable to the Floodlight port. Looking through the Flowvisor configuration file (can be saved to file running fvctl dumpConfig [filename] against a running Flowvisor), our initial guesses say that the following do not apply for each component:

  • "Slice": controller hostname, port (we are working with modules, not external controllers)
  • "flowvisor": most of the information kept for flowvisor can be taken from Floodlight e.g. API and listen port, versioning, and host.
  • "FlowSpaceRule": this is probably the only unchanged component.

Config file parsing in Flowvisor happens in FVConfig:readFromFile(). Java reflection is used to then configure each component e.g manipulate database contents relevant to each.

(6/30)

With the database components removed, we need to re-build the components that parse the Flowvisor configs and initialize everything (FlowMap definitions, slices, etc.). FVConfSetup.java is intended for this purpose. So far, it uses Jackson (used in Floodlight for the Static Flow Pusher) to parse the Flowvisor configuration file to initialize the FlowMap (linear for now, since it is simple than the federated).

(7/2)

  • A Classifier knows about the Slicers associated with the DPID it is associated with. This is kept in a Java HashMap.
  • A Slicer knows the name, controller/TCP port, and switch ports associated with a slice.
  • Each FlowSpaceRule entry (flow entry) stores which DPID and slice (by name and permissions, via SliceAction.java) it is associated with.

Unlike the FlowMap, a Slice does not seem to be a concrete object, but a collection of information shared amongst the Slicers and Classifiers. We create a new configuration option, "module_name," to replace the controller information stored in a "slice". For now we keep the old parameters for each Slice, but modified: 

         "controller_hostname": "localhost",
         "controller_port": 0,
         "module_name": "forwarding",

In any case, we need an easily manipulated mapping of DPIDs to slices, and slices to their modules, where modules are specified by the names returned by getName().

.

Attachments (2)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.