Exercise 3: Socket Programming using New MobilityFirst NetAPI

Design/Setup

Objective:

In this exercise we will learn to write, compile, and run a simple content distribution application using MobilityFirst's new socket API. In particular we will focus on its Java release.

Pre-requisites

• Experimenters are expected to have basic networking knowledge and familiarity with Linux OS and some of its tools (command line tools, ssh, etc.).
• An ORBIT user account.
• Some familiarity with the MobilityFirst terminology.

Deploy the Network

This tutorial assumes that a 4 nodes topology has been already established in one of the Orbit sandboxes or the grid:

If not coming from exercise 1 follow these instructions on how to setup the topology. Running exercise 1 at least once before moving to exercise 2 is advised to understand the steps and software components involved.

4 nodes topology setup

First of all, log in into the grid console using SSH:

ssh username@console.grid.orbit-lab.org

For simplicity, open 3 different consoles on your laptop and access the grid's console with all of them; you will need them in the continuation of the exercise. From the console we will start loading the !Mobilityfirst image into the four nodes that have been assigned to you:

omf load -i 'mf-release-latest.ndz' -t system:topo:mf-groupX

system:topo:mf-groupX represents the topology of 4 nodes that has been assigned to you're group and mf-groupX has to be replaced by the group id assigned to you.

For example, mf-group1 will load the image on nodes 'node20-20,node20-19,node19-19,node19-20'

If at the end of the execution, the final output of your console looks similar to:

INFO exp:  ----------------------------- 
 INFO exp:  Imaging Process Done 
 INFO exp:  4 nodes successfully imaged - Topology saved in '/tmp/pxe_slice-2014-10-15t02.10.16.594-04.00-topo-success.rb'
 INFO exp:  ----------------------------- 
 INFO EXPERIMENT_DONE: Event triggered. Starting the associated tasks.
 INFO NodeHandler: 
 INFO NodeHandler: Shutting down experiment, please wait...
 INFO NodeHandler: 
 INFO NodeHandler: Shutdown flag is set - Turning Off the resources
 INFO run: Experiment pxe_slice-2014-10-15t02.10.16.594-04.00 finished after 1:50

your nodes have been imaged correctly.

Deploy Network

Software and experiment control in the ORBIT testbed can be automated greatly using the OMF framework. An OMF control script is written in Ruby and allows the experimenter to specify the set of nodes, their network configuration, to specify software components and arguments, and to control their execution on one or more nodes. We will use an OMF script to bring up 4 ORBIT nodes to host our topology, with the corresponding software components.

We will first introduce the main details of the scripts that will be run and then we will step to the execution process itself.

Software Component Specification

The following snippet shows the specification of the MobilityFirst components along with the required arguments. A typical application will have at least a brief description, a path for the associated binary to execute and a list of properties that correspond to the parameters that will be passed once starting the executable.

#Application definition of a MobilityFirst access router
defApplication('MF-Router', 'router') {|app|
    app.shortDescription = "Click-based MobilityFirst Access Router"
    app.path = "/usr/local/src/mobilityfirst/eval/orbit/tutorial/scripts/ARWrapper.sh" 
    # click options
    app.defProperty('num_threads', 'number of threads', "-t",{:type => :integer, :mandatory => true, :default => 4, :order => 1})
    app.defProperty('ctrl_port', 'port for Click control socket', "-c",{:type => :integer, :order => 2})
    # click config file 
    app.defProperty('config_file', 'Click configuration file', "-C",{:type => :string,:mandatory=> true})
    # keyword parameters used in click config file
    app.defProperty('my_GUID', 'router GUID', "-m",{:type => :string, :mandatory => true})
    app.defProperty('topo_file', 'path to topology file', "-f",{:type => :string, :mandatory => true})
    app.defProperty('core_dev', 'core network interface', "-d",{:type => :string,:mandatory => true})
    app.defProperty('GNRS_server_ip', 'IP of local GNRS server', "-s",{:type => :string,:mandatory => true})
    app.defProperty('GNRS_server_port', 'Port of GNRS server', "-p",{:type => :string,:mandatory => true})
    app.defProperty('GNRS_listen_ip', 'IP to listen for GNRS response', "-i",{:type => :string,:default => "0.0.0.0"})
    app.defProperty('GNRS_listen_port', 'port to listen for GNRS response', "-P",{:type => :string,:default => "10001"})
    app.defProperty('edge_dev', 'edge network interface', "-D",{:type => :string,:mandatory => true})
    app.defProperty('edge_dev_ip', 'IP assigned to edge interface', "-I",{:type => :string,:mandatory => true})
}

#Application definition of a GNRS server
defApplication('MF-GNRS', 'gnrs') {|app|
    app.shortDescription = "GNRS Server"
    app.path = "/usr/local/src/mobilityfirst/eval/orbit/tutorial/scripts/GNRSWrapper.sh" 
    app.defProperty('log4j_config_file', 'log 4j configuration file', "-d",{:type => :string, :order => 1})
    app.defProperty('jar_file', 'server jar file with all dependencies', "-j" ,{:type => :string, :mandatory=> true, :default => "/usr/local/src/mobilityfirst/gnrs/jserver/target/gnrs-server-1.0.0-SNAPSHOT-jar-with-dependencies.jar", :order => 2})
    app.defProperty('config_file', 'server configuration file', "-c",{:type => :string, :mandatory=> true, :order => 3})
}


#Application definition of the client network protocol stack
defApplication('MF-HostStack', 'hoststack') {|app|
    app.shortDescription = "MF host network stack"
    app.path = "/usr/local/bin/mfstack" 
    app.defProperty('log_level', 'log level', nil,{:type => :string, :mandatory => true, :order => 1, :default => "-e"}) # default is 'error'
    app.defProperty('config_file', 'stack configuration file', nil,{:type => :string, :mandatory => true, :order => 2})
}

A few considerations on the defined applications:

• As seen above, the router is configured with both 'core' (core_dev) and 'edge' (edge_dev) interfaces. Different router configurations are available depending on the required functionality. In this case we use what we call a MobilityFirst Access Router, which has the particularity of having the core interfaces connected towards the core of the network, while the edge interface enables hosts to connect and access the MobilityFirst network.

• For this basic setup, the GNRS has been configured to be running as a single server instance, but in a larger experiment, it is designed to be a distributed system deployed at different locations.

• Most of the client settings are located in a configuration file pre-loaded on the ORBIT image in the folder /usr/local/src/mobilityfirst/eval/orbit/conf/.

Setting up the Software Node Groups

The following snippet shows how the node groups for the routers are setup in the OMF control scripts. Node groups allow experimenters to use single statements to set configuration (e.g. network interfaces) and execute commands across all nodes belonging to the group.

#Create router groups
for i in 1..num_routers
        #Create a topology with a single router in it
    defTopology("topo:router_#{i}") { |t|
        aNode = routersTopo.getNodeByIndex(i-1)
        t.addNode(aNode)
        info aNode, " assigned role of router with GUID: #{i}"
    }
  
        #Through the group definition we set up the applications to run
    defGroup("router_#{i}", "topo:router_#{i}") {|node|
        node.addApplication('MF-Router') {|app|
            app.setProperty('num_threads', router_threads)
            app.setProperty('config_file', click_conf)
            app.setProperty('my_GUID', router_guid[i-1])
            app.setProperty('topo_file', rtr_topo_file)
            app.setProperty('core_dev', core_dev)
            app.setProperty('GNRS_server_ip', GNRS_server_ip)
            app.setProperty('GNRS_server_port', GNRS_server_port)
            app.setProperty('GNRS_listen_ip', "192.168.100.#{i}")
            app.setProperty('GNRS_listen_port', GNRS_listen_port)
            app.setProperty('edge_dev', edge_dev)
            app.setProperty('edge_dev_ip', router_ether_if_ip[i-1])
        }

      #If it is the first router add the GNRS
      if i == 1
                aNode = routersTopo.getNodeByIndex(i-1)
                info aNode, " will also host the GNRS server"
        node.addApplication('MF-GNRS') {|app|
              app.setProperty('log4j_config_file', GNRS_log_file)
              app.setProperty('jar_file', GNRS_jar_file)
              app.setProperty('config_file', GNRS_conf_file)
        }
      end
    
          #Setup the node interfaces
          #The first ethernet interface is used as the core interface
      node.net.e0.ip = "192.168.100.#{i}"
      node.net.e0.netmask = '255.255.255.0'
    
          #The first wireless interface is used to give access to clients
          node.net.w0.mode = "adhoc"
          node.net.w0.type = 'g'
          node.net.w0.channel = "11"
          node.net.w0.essid = "SSID_group_#{i}"
          node.net.w0.ip = "192.168.#{i}.1"
    }
end

#Create host groups
for i in 1..num_hosts
        #Create a topology with a single router in it
    defTopology("topo:host_#{i}") { |t|
        aNode = hostsTopo.getNodeByIndex(i-1)
        t.addNode(aNode)
        info aNode, " assigned role of client with GUID: #{100 + i}"
    }
  
        #Through the group definition we set up the applications to run
    defGroup("host_#{i}", "topo:host_#{i}") {|node|
        node.addApplication('MF-HostStack') {|app|
            app.setProperty('config_file', hoststack_conf_file[i-1])
            app.setProperty('log_level', log_level)
        }
    
          #The first wifi interface is used to connect to the Access Router
          node.net.w0.mode = "adhoc"
          node.net.w0.type = 'g'
          node.net.w0.channel = "11"
          node.net.w0.essid = "SSID_group_#{i}"
          node.net.w0.ip = "192.168.#{i}.2"
      }
end

As it can be seen above, once defining applications that each group will execute, the application properties are set accordingly. While we do not want to enter the details of each parameter, it is important to notice how by simple use of counters, the different nodes can be assigned different values.

Moreover, resources such node interfaces and their corresponding IP addresses have to be set up in this phase of the experiment. As we discussed earlier the router is configured with both edge and core interfaces. An ethernet interface is used to connect to 2 core routers, while a wireless interface is used to provide access for the clients.

Develop Sender and Receiver Applications with MF Socket API

As per the goal of the exercise, we will introduce the reader to the new MF Socket API. Two simple Java applications skeletons have been made available on the nodes. These applications consist of a sender, that takes as an input a file and transmit it to a receiver that once has received the complete file anwers back with an acknowledgement. Let's first analyze the sender code:

public static void main(String []argv){
        if(argv.length < 2){
            usage();
            return;
        }
        
        //The profile describes the nature of the communication that will follow and
        //  is used by the network stack to select the best end-to-end transport
        //For this application a 'basic' profile is selected providing only a message based
        //  transport with no added realiability on top of what offered by the network.
        String profile = "basic";
        GUID srcGUID = null;
        
        //A GUID class is used for name based communications
        //The destination of the fie has been passed as a parameter
        GUID dstGUID = new GUID(Integer.parseInt(argv[1]));
        //The source is optional. If a source is not specified, the default GUID of the device is used
        if(argv.length == 3) srcGUID = new GUID(Integer.parseInt(argv[2]));
        
        Path file = FileSystems.getDefault().getPath(argv[0]);
        
        //The JMFAPI object represents the socket and the API to interact with it
        JMFAPI sender = new JMFAPI();
        
        try{
            
            //The open call creates the communication socket and initializes the resources
            if(srcGUID!=null) sender.jmfopen(profile, srcGUID);
            else sender.jmfopen(profile);
            
            byte[] fileArray;
            try {
                fileArray = Files.readAllBytes(file);
            } catch (IOException e){
                System.out.println("ERROR");
                return;
            }
            System.out.println("Transferring a file of size " + fileArray.length);
            byte[] sizeArray = Utils.intToByteArray(fileArray.length);
            sender.jmfsend(sizeArray, 4, dstGUID);
            int sentBytes;
            
            byte[] tempArray;
            int ret, read = 0;
            int bytesToSend = fileArray.length;
            while(bytesToSend>1000000){
                tempArray = Arrays.copyOfRange(fileArray, 0, 999999);
                //Messages are sent up to 10MB at a time (which is the default buffer size for the socket)
                sentBytes = sender.jmfsend(tempArray,1000000, dstGUID);
                bytesToSend -= sentBytes;
                System.out.println("Transmitted " + sentBytes);
            }
            tempArray = Arrays.copyOfRange(fileArray, 0, bytesToSend - 1);
            sentBytes = sender.jmfsend(tempArray,bytesToSend, dstGUID);
            System.out.println("Transmitted " + sentBytes);
            
            //Receive the confirmation from the receiver
            //The first parameter is set to null but could be used to obtain the GUID of the message source
            sender.jmfrecv_blk(null,tempArray, 1000000);
            int receivedBytes = Utils.byteArrayToInt(tempArray, 0);
            System.out.println("The receiver received " + receivedBytes + " succesfully");
            
            //Close the socket and clear the resources
            sender.jmfclose();
            System.out.println("Transfer completed");
            
        } catch (JMFException e){
            //Exceptions related to events occured in the network protocol stack are defined as JMFException
            System.out.println(e.toString());
        }
    }

The receiver code is now presented:

public static void main(String []argv){
        //The profile describes the nature of the communication that will follow and
        //  is used by the network stack to select the best end-to-end transport
        //For this application a 'basic' profile is selected providing only a message based
        //  transport with no added realiability on top of what offered by the network.
        String scheme = "basic";
        
        GUID srcGUID = null;
        GUID senderGUID = new GUID();
        int i = 0;
        
        //A GUID class is used for name based communications
        //The source is optional. If a source is not specified, the default GUID of the device is used
        if(argv.length == 1) srcGUID = new GUID(Integer.parseInt(argv[0]));
        
        Path file = FileSystems.getDefault().getPath("temp.txt");
        try{
            Files.createFile(file);
        } catch(IOException e){
            try{
                Files.delete(file);
                Files.createFile(file);
            } catch(IOException e2){
                System.out.println(e2.toString());
                return;
            }
        }
        
        byte[] buf = new byte[1000000];
        int ret;
        JMFAPI receiver = new JMFAPI();
        try{
            if(srcGUID!=null) receiver.jmfopen(scheme, srcGUID);
            else receiver.jmfopen(scheme);
            
            //First message will include the size of the transfered file
            ret = receiver.jmfrecv_blk(senderGUID, buf, 1000000);
            int fileSize = Utils.byteArrayToInt(buf, 0);
            System.out.println("I will receive a file of size " + fileSize + " bytes from host with GUID " + senderGUID.getGUID());
            
            int total = 0;
            while(i < fileSize){
                ret = receiver.jmfrecv_blk(null, buf, 1000000);
                total+=ret;
                System.out.println("Received " + ret + " bytes");
                try{
                    Files.write(file, Arrays.copyOfRange(buf, 0, ret), StandardOpenOption.APPEND);
                } catch (IOException e){
                    System.out.println(e.toString());
                }
                i += ret;

            }
            
            //Send back an acknowledgement with the amount of bytes received
            byte[] answer = Utils.intToByteArray(total);
            receiver.jmfsend(answer,4, senderGUID);
            
            receiver.jmfclose();
        } catch (JMFException e){
            System.out.println(e.toString());
        }
        System.out.println("Transfer completed");
    }
}

While a very simple application a few concepts can be taken away from code just presented:

• Communication profiling: enables the developer to specify the set of elements that characterize the following session such as: communication patterns, resources needed and services required. The network stack can use this information to select the best combination of transport, resources, etc. to allocate for the socket.
• Named operations: in contrast with BSD Sockets, named communications are implemented and name to address resolution is handled in conjunction between the network stack and the network components.

Execute

As in the previous tutorials, you will need to start the experiment via an OMF script. Download the script to the orbit console copying and pasting the following command in your terminal:

    wget www.winlab.rutgers.edu/~bronzino/downloads/orbit/exercise3.rb

Once the file has been downloaded, execute it with the following command:

    omf exec exercise3.rb

Once your experiment is showing you the following line:

    INFO exp: Request from Experiment Script: Wait for 10000s....

you can proceed with the execution of the applications. Using the previously opened consoles log in into the two host nodes (GUIDs 101 and 102) that will be used to run the simple 'mfping' application. In order to access a running Orbit node ssh into it as follow:

ssh root@nodex-y

Where x-y has to be replaced by the actual numbers identifying the node.

Once logged in, run on the node with GUID 102 the receiver component:

cd exercise3
java -cp .:jmfapi-1.0-SNAPSHOT.jar Receiver

To run the sender component, start on the client with GUID 101 the command:

cd exercise3
java -cp .:jmfapi-1.0-SNAPSHOT.jar Sender 10MB.dat 102

This will trigger the transmission of a file of 10MB size directed to host 102.

What's next

In this tutorial we only scratched the surface of what is possible to implement using the new Socket Network API. A few examples on what can be done:

• Using multipoint delivery (e.g. multicast, anycast) through use of socket options on a per-operation base.
• Content based communication profiles and operations (e.g. get).
• Use of a mobility manager with support for simple migration policies, e.g., “use WiFi whenever available”.
• Management of network presence through Interaction with the Global Name Service.

The Network API is available both for Java and C/C++ applications.

Finish

Once the application has successfully completed its task, follow these steps to complete the experiments:

• On the grid's console running the experiment script, interrupt the experiment using the Ctrl-C key combination.

This will stop all the applications and will conclude the experiment.

References

For more information regarding the MobilityFirst project, visit the project page: ​http://mobilityfirst.winlab.rutgers.edu/

For more information regarding the prototype design and updated status, visit the wiki page: ​https://mobilityfirst.orbit-lab.org/