= Inventory Grid Service =

[[TOC(Internal/Infrastructure/OMF/InventoryService,depth=2)]]

== Inventory Gridservice ==

Main purpose of [source:/gridService/src/ruby/ogs_inventory Inventory OMF service] is to expose the inventory information to other services as well as experiments (experiment scripts). In addition to obvious node set aliases (say based on device type and ordinal number) it should allow us to query the inventory based on other criteria like "at least 5 nodes with memory > 500M & a bluetooth radio & an Atheros 802.11 card". 
 * It should have an (optional) query qualifier (with interface to CMC) to return only the functional node set.
 * It should have ...


It is important to consider the Inventory database schema part of the experimental interface.  The advantage of building a web service for Inventory is that it can provide commonly used queries that incorporate data from other OMF web services (namely the CMC, as in "available" in "give me all available intel nodes").  At the same time, it will be easier for experiment scripts to build custom SQL queries that implement complex criteria than to utilize a more abstract web service.  Furthermore, arguably, SQL is RESTful in the first place.

== Service Configuration File ==

Lives in /etc/gridservices2/available/inventory.yaml

{{{
inventory:

  testbed:
    default:
      # Name of the MySQL Database Server
      host: host-name
      # Username and password to access the database server
      user: the-db-username
      password: the-db-password
      # Name of the database to use
      database: new_new_inventory
}}}

== Inventory Gathering ==

Inventory, is the "experiment" that uses a special much larger image, which can have a relatively large number of drivers for a wide variety of devices.  Inventory is used to do the slower more difficult job of generating informative scans of the pci and usb busses. 

As of 10/16/2007, we run the inventory every Wednesday during maintenance.  Plan is to, in addition, run Inventory whenever there is a change in hardware configuration.  There exists a full [source:/Inventory/inventory inventory package] that inserts "all" information into the database. 

  The gathering procedure uses:
 
 * lspci
 * lsusb
 * dmidecode
 
=== Known Device IDs ===

Devices that are discovered by gathering procedure are uniquely identified by either [http://www.linux-usb.org/usb.ids USB ids] or [http://www.pcidatabase.com/vendors.php?sort=id PCI ids]. The ids for most important devices are listed in the ID table:  

|| Bus type || Vendor ID || Vendor || Device ID || Description ||
||USB || 0x050d (1293) || Belkin || 0x0084 (132) || Bluetooth Transciever F8T003v2 ||
|| -"- || -"- || -"- || 0x0131 (305) || Bluetooth Transciever with trace filter (F8T001 ?) ||
|| -"- || 0x0a5c (2652) || Broadcom Corp. || 0x2101 (8449) || A-Link BlueUsbA2 Bluetooth ||
|| -"- || 0x0403(1027) || Future Technology Devices International, Ltd || 0x6001 (24577) || FT232 USB-Serial (UART) IC ||
|| -"- || 0xfffe (65534) || Cypress Semiconductor  || 0x0002 (2) || EZ-USB FX2 (USRP and !NoiseGenerator) ||
|| PCI || 0x1106 (4358) || [http://www.pcidatabase.com/vendor_details.php?id=648 VIA Technology] || 0x3122 (12578) || VT8623 !CastleRock AGP 8X Controller (VGA) ||
|| -"- || -"- || -"- || 0x0571 (1393) || Bus Master IDE Controller ||
|| -"- || -"- || -"- || 0x3177 (12663) || VT8235 PCI to ISA Bridge ||
|| -"- || -"- || -"- || 0x3104 (12548) || VT6202 USB 2.0 Enhanced Host Controller ||
|| -"- || -"- || -"- || 0x3038 (12344) || USB&UHCI Controller ||
|| -"- || -"- || -"- || 0xb091 (45201) || VT8633 PCI-to-PCI Bridge (AGP) ||
|| -"- || -"- || -"- || 0x3123 (12579) || VT8623 CPU to PCI Bridge ||
|| -"- || 0x11ab (4523) || [http://www.pcidatabase.com/vendor_details.php?id=957 Marvell Semiconductor] || 0x4320 (17184) || 88E8055 Marvell Yukon PCI E Gigabit Ethernet Controller ||


We should look into turning inventory image into PXE image and avoid imaging phase completely!

== Inventory Database ==

Inventory database lives on internal1 and consists of 6 tables:

 1. device_kinds
 2. device_tags
 3. inventories
 4. locations
 5. motherboards
 6. nodes 
 7. peripherals 
 8. testbeds

=== device_kinds table ===

|| Field        || Type        || Null || Key || Default || Extra          ||
|| id           || int(11)     || NO   || PRI || NULL    || auto_increment ||
|| inventory_id || int(11)     || NO   ||     || NULL    ||                ||
|| oui          || varchar(8)  || YES  ||     || NULL    ||                ||
|| bus          || varchar(16) || YES  ||     || NULL    ||                ||
|| vendor       || int(11)     || NO   ||     || NULL    ||                ||
|| device       || int(11)     || NO   ||     || NULL    ||                ||


=== device_tags table ===

|| Field         || Type        || Null || Key || Default || Extra ||
|| tag           || varchar(64) || NO   ||     || NULL    ||       ||
|| peripheral_id || int(11)     || NO   ||     || NULL    ||       ||

=== motherboards table ===
|| Field       || Type        || Null || Key || Default           || Extra          || Description                                   ||
|| id          || varchar(64) || NO   || PRI ||                   ||                || UUID of the motherboard                       ||
|| node_id     || varchar(64) || YES  || UNI || NULL              ||                || Link to 'id' in nodes table                   ||
|| sn   || varchar(16) || NO   || UNI ||                   ||                || manufacturer serial number of the motherboard ||
|| hd_sn        || varchar(16) || NO   || UNI ||                   ||                || Hard drive serial number                      ||
|| cpu_type         || varchar(X)  || YES  ||     || NULL              ||                || CPU Type                                      ||
|| cpu_speed       || int(11)     || YES  ||     || 0                 ||                || CPU speed in MHz                              ||
|| memory      || int(11)     || YES  ||     || 0                 ||                || Memory size in MB                             ||
|| hd_size        || int(11)     || YES  ||     || 0                 ||                || Hard disk size in bytes                          ||
|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
|| updated_by  || varchar(64) || NO   ||     || ||                ||                                               ||


(NOTE: 'node_id' is NULL when this motherboard is not installed on any node, i.e. new parts that just got in, or stored extra/spare parts)

We could also move the hard-drive info in a separate table if we allow hard-drive swapping between motherboards.

=== nodes table ===
|| Field       || Type        || Null || Key || Default || Extra          || Description ||
|| id          || varchar(64) || NO   || PRI ||         ||                || UUID of the node (i.e. the chassis). ||
|| chassis_sn  || varchar(16) || NO   || UNI ||         ||                || Manufacturer serial number of the node's chassis ||
|| location_id || varchar(64) || YES  || UNI || NULL    ||                || Link to 'id' in 'locations' table ||
|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
|| updated_by  || varchar(64) || NO   ||     ||  ||                ||                                               ||

(NOTE: 'location_id' is NULL when this chassis is not installed at any location, i.e. new parts that just got in, or stored extra/spare parts)

=== locations table ===

|| Field       || Type        || Null || Key || Default           || Extra          || Description ||
|| id          || varchar(64) || NO   || PRI ||         ||                || UUID of the location ||
|| x           || int(11)     || NO   ||     || 0                 ||                || ||
|| y           || int(11)     || NO   ||     || 0                 ||                || ||
|| z           || int(11)     || NO   ||     || 0                 ||                || ||
|| unit        || int(11)     || NO   ||     || 0                 ||                || ||
|| testbed_id  || varchar(64) || NO   ||     || 0                 ||                || Link to 'id' in 'testbeds' table ||
|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
|| updated_by  || varchar(64) || NO   ||     ||  ||                ||                                               ||


=== testbeds (resources) table ===

|| Field      || Type        || Null || Key || Default || Extra  || Description ||
|| id         || varchar(64)  || NO  || PRI ||         ||        || UUID of the testbed ||
|| domain     || varchar(4)  || NO   || UNI ||         ||        || ||
|| control_ip || varchar(12) || NO   || UNI ||         ||        || ||
|| data_ip    || varchar(12) || NO   || UNI ||         ||        || ||
|| cm_ip      || varchar(12) || NO   ||     ||         ||        || ||
|| latitude   || int(11)     || NO   ||     || 0       ||        || ||
|| longitude  || int(11)     || NO   ||     || 0       ||        || ||
|| elevation  || int(11)     || NO   ||     || 0       ||        || ||
|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
|| updated_by  || varchar(64) || NO   ||     ||  ||                ||                                               ||

== DESCRIPTION ==

The design goal of this schema is to allow the double use of the Inventory database as:
  * a source of information for user experiment scripts
  * a 'real' hardware inventory giving operators information on which piece of hardware (chassis, motherboard) is used (or not) in which testbed/location.

The entries in the ''testbeds'', ''locations'', ''nodes'' tables are manually created and updated by operators, when:
  * a new testbed is being deployed
  * a new location is added to the testbed (e.g. physical place-holder creation on a sandbox testbed for future addition of a third node) 
  * a new purchased chassis (i.e. empty node box) is delivered, or mounted to a new location, or switched from a location to another one

We do not expect these events to happen very often, thus it should be ok to make the operator responsible for creating/updating the related entries. (furthermore he/she could also use some scripts to do this job...)

The entries in the ''motherboards'' table are also manually created upon delivery of a new purchased motherboard. The only field that needs to be manually filled by the operator is the ''node_id'', which will happen when the operator installs a new motherboard inside a node/chassis. All the other fields are automatically populated by the Inventory process (i.e. the scripts in the inventory package). 

The ''interfaces'' and ''devices'' tables are created and updated as in the previous schema.