Rebuilding the Node Agent from the ground up

Dependencies:

• omfcommon/Mobject

gems:

• cocaine
• yaml?

---

Design Notes:

Lib:kmodule.rb
Class:kmodule
Interface:

• new (private)
• instance - give you and instance of the object with checks to make sure the module name is unique
• mload(private) - loads the module if needed and registers a reference to the class that requested the load
• munload - unloads module is conditions are met, and de-registers the unload requester
• loaded? - preforms an actual check to see if module is loaded

Moving all the functionality of module handling into a separate class. There may be a need to have drivers load multiple modules. Additionally the module class needs to keep track of how many devices are using one module (as there may be many to one). The unload method should only unload if the last reference asks it to (all other references have de-registered).

Lib:nodeagent.rb:
Class:nodeagent.rb:
Interface

• hwDiscover -
• instance
• run

The hwdiscovery mode of the nodeagent code is being re-done. Instead of the original grep lspci lines, we now have has of dev_id's. Each dev_id in the hash maps to a node agent device driver that will be loaded to handle that specific device. This device will load the module for it's respective device type.

agentCommands.rb:
Interface

• configure

device.rb
Interface

• activate
• deactivate
• active?
• getconfigCMD
• configure
• hasMod? *add_kmodName

We're implementing all the module handeling code here. It uses the cocaine command line library as a wrapper. The cocaine lib kicks out an exception if the command fails. Exception handling will mitigate failed commands.

ethernet.rb < device.rb
Interface

• initialize()
• claim_ctrl(call_ref)
• self.set_control_if(iface="eth1")
• self.get_control_if()
• control_if?
• set_ip(ip)
• get_ip()
• up()
• down()
• set_netmask(netmask)
• set_mtu(mtu)
• set_mac(mac)
• get_mac()
• set_arp(arp)
• set_forwarding(forwarding)
• set_gateway(gateway)
• drop_config()
• restart_dhcp()
• execConfigCmd(prop, value=nil)
• getCmdForRoute(value)
• getCmdForFilter(value)
• deactivate()

This class is responsible for all of the ifconfig/route handling code (all things ethernet). The same cocaine lib will handle shell commands with similar exception handeling. The interface has been extended to implement getters and setters for all the requisite interface state (ip, netmask, etc…). We've left the original get config command structure alone, but now also have implemented an execConfigCmd which will actually implement these config commands.

wifi.rb < ethernet.rb
Interface

• initialize()
• check_time
• connected?()
• connect_simple(ssid)
• connect_adhoc
• disconnect_simple
• disconnect_adhoc
• disconnect()
• checkStatus(retries = true)
• scan()

rcDrv_e1000e.rb < ethernet.rb
Interface

• new

e1000e kernel module specif driver. A child of ethernet. Overides any thing that is specific to e1000e, but mostly leaves the parent class in tact. Also sets the value of @kmodName

rcdrv_r8169.rb < ethernet.rb
Interface

• new

rcdrv_skge.rb < ethernet.rb
Interface

• new

rcdrv_ath5k.rb < wifi.rb
Interface

• new

rcdrv_ath9k.rb < wifi.rb
Interface

• new

rcdrv_ipw2200.rb < wifi.rb
Interface

• new

rcdrv_iwlwifi.rb < wifi.rb
Interface

• new

rcdrv_wl.rb < wifi.rb
Interface

• new

Driver Inheretance Chain

---

Observations:

---

9/1/2014

• active? is defined at "Ready to be configured". If a device is set to active, it is read to accept config commands.
• Activation should check if the module is loaded before loading it.
• All devices will not know their device name during creation (default is nil). All dev_driver classes will be instantiated at startup, however none of them will be activated (with the exception of eth1). When a device need to be used. It will have to be activated. Once activated (that is once the module is loaded) the agent will give names to the respective objects now that they have been created. This is essentially how the logical names are bound to the dev names (a logic managed by the agent, not the dev).

---

9/22/2014

• It may be necessary to have multiple driver modules for a single device. The recurrent case is the iwldvm module. It gets loaded if we load iwlwifi but it depends on iwlwifi. Thus when you load iwlwifi, you can not unload it with out first unloading iwldvm. Because iwlwifi and athXk both depend on cfg802, when the iwldvm module is unloaded with modprobe -r iwldvm, iwlwifi is not unloaded.

---

9/23/2014

• Node agent it self should check what modules are already loaded and create instances of the module classes for modules that are already loaded. It will push in a refrence to it self, so that the refrence buffer can never go empty.
• There will need to be a structure that maps pci-id ↔ modulename ↔ deviceClass. This structure will need to be known to node agent.

---

10/30/2014

I've run into a chicken and egg problem where I need to know the device name before I can make the object but can't know the device name until after the object is made. I think I'm going to resolve this by making the logical names mutable.

---

11/11/2014 Now trying to integrate the driver and agent code. We'll basically be modifying the behavior of the agentCommands.configure method. This used to construct command strings that would be run with minimal error checking. Now this will map the request strings to a specific method call. This method should have it's own error handling and raise exceptions when something doesn't work properly. These exceptions can be caught at the top level to prevent crashes.

In ​http://www.orbit-lab.org/browser/omf/omf-expctl/ruby/omf-expctl/node/nodeSetPath.rb the set of possible properties that can be set is listed in an array. The original agent command code is here: ​http://www.orbit-lab.org/browser/omf/omf-resctl/ruby/omf-resctl/omf_agent/agentCommands.rb

---

12/15/2014

We ran into a race condition of sorts where commands may come in after the connect call was issued. We've decided that the "start_application" directive is the decision point where we've given all the parameters we expect to get. To this end in the node-agent maintains a parameter model of the connection to be made. As parameters come in the node agent updates these parameters from the default. When the "start_application" directive is received, the agent will initiate a connect command. At this point the applicationStarted flag will = true. Once this flag is set, any parameter changes will be applied immediately. Parameters must be split into two types. Those that can be applied immedately (like freq) and those that require a reconnect like mode ibss). Once the applicationStarted flag is set if a reconnect is required when a parameter is changed it is do immedately.

As a side note the different mode connect methods should check if the device is already connected and if so do nothing (regardless of which mode it is connected in).

—- 12/24/2014

The agent commands second argument is a object of type OmfPubSubMessage. The source of this class is located ​here. The path method produces the output we'll be switching on. The configure command calls the respective device_driver's configure command and expects a hash output. In this configure command is where the connect state needs to be implemented and maintained.