Changes between Initial Version and Version 1 of Internal/Infrastructure/OMF/GridServices/Inventory


Ignore:
Timestamp:
May 12, 2008, 1:17:12 AM (16 years ago)
Author:
seskar
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • Internal/Infrastructure/OMF/GridServices/Inventory

    v1 v1  
     1= Inventory Grid Service =
     2
     3[[TOC(Internal/Infrastructure/OMF/InventoryService,depth=2)]]
     4
     5== Inventory Gridservice ==
     6
     7Main purpose of [source:/gridService/src/ruby/ogs_inventory Inventory OMF service] is to expose the inventory information to other services as well as experiments (experiment scripts). In addition to obvious node set aliases (say based on device type and ordinal number) it should allow us to query the inventory based on other criteria like "at least 5 nodes with memory > 500M & a bluetooth radio & an Atheros 802.11 card".
     8 * It should have an (optional) query qualifier (with interface to CMC) to return only the functional node set.
     9 * It should have ...
     10
     11
     12It is important to consider the Inventory database schema part of the experimental interface.  The advantage of building a web service for Inventory is that it can provide commonly used queries that incorporate data from other OMF web services (namely the CMC, as in "available" in "give me all available intel nodes").  At the same time, it will be easier for experiment scripts to build custom SQL queries that implement complex criteria than to utilize a more abstract web service.  Furthermore, arguably, SQL is RESTful in the first place.
     13
     14== Service Configuration File ==
     15
     16Lives in /etc/gridservices2/available/inventory.yaml
     17
     18{{{
     19inventory:
     20
     21  testbed:
     22    default:
     23      # Name of the MySQL Database Server
     24      host: host-name
     25      # Username and password to access the database server
     26      user: the-db-username
     27      password: the-db-password
     28      # Name of the database to use
     29      database: new_new_inventory
     30}}}
     31
     32== Inventory Gathering ==
     33
     34Inventory, is the "experiment" that uses a special much larger image, which can have a relatively large number of drivers for a wide variety of devices.  Inventory is used to do the slower more difficult job of generating informative scans of the pci and usb busses.
     35
     36As of 10/16/2007, we run the inventory every Wednesday during maintenance.  Plan is to, in addition, run Inventory whenever there is a change in hardware configuration.  There exists a full [source:/Inventory/inventory inventory package] that inserts "all" information into the database.
     37
     38  The gathering procedure uses:
     39 
     40 * lspci
     41 * lsusb
     42 * dmidecode
     43 
     44=== Known Device IDs ===
     45
     46Devices that are discovered by gathering procedure are uniquely identified by either [http://www.linux-usb.org/usb.ids USB ids] or [http://www.pcidatabase.com/vendors.php?sort=id PCI ids]. The ids for most important devices are listed in the ID table: 
     47
     48|| Bus type || Vendor ID || Vendor || Device ID || Description ||
     49||USB || 0x050d (1293) || Belkin || 0x0084 (132) || Bluetooth Transciever F8T003v2 ||
     50|| -"- || -"- || -"- || 0x0131 (305) || Bluetooth Transciever with trace filter (F8T001 ?) ||
     51|| -"- || 0x0a5c (2652) || Broadcom Corp. || 0x2101 (8449) || A-Link BlueUsbA2 Bluetooth ||
     52|| -"- || 0x0403(1027) || Future Technology Devices International, Ltd || 0x6001 (24577) || FT232 USB-Serial (UART) IC ||
     53|| -"- || 0xfffe (65534) || Cypress Semiconductor  || 0x0002 (2) || EZ-USB FX2 (USRP and !NoiseGenerator) ||
     54|| PCI || 0x1106 (4358) || [http://www.pcidatabase.com/vendor_details.php?id=648 VIA Technology] || 0x3122 (12578) || VT8623 !CastleRock AGP 8X Controller (VGA) ||
     55|| -"- || -"- || -"- || 0x0571 (1393) || Bus Master IDE Controller ||
     56|| -"- || -"- || -"- || 0x3177 (12663) || VT8235 PCI to ISA Bridge ||
     57|| -"- || -"- || -"- || 0x3104 (12548) || VT6202 USB 2.0 Enhanced Host Controller ||
     58|| -"- || -"- || -"- || 0x3038 (12344) || USB&UHCI Controller ||
     59|| -"- || -"- || -"- || 0xb091 (45201) || VT8633 PCI-to-PCI Bridge (AGP) ||
     60|| -"- || -"- || -"- || 0x3123 (12579) || VT8623 CPU to PCI Bridge ||
     61|| -"- || 0x11ab (4523) || [http://www.pcidatabase.com/vendor_details.php?id=957 Marvell Semiconductor] || 0x4320 (17184) || 88E8055 Marvell Yukon PCI E Gigabit Ethernet Controller ||
     62
     63
     64We should look into turning inventory image into PXE image and avoid imaging phase completely!
     65
     66== Inventory Database ==
     67
     68Inventory database lives on internal1 and consists of 6 tables:
     69
     70 1. device_kinds
     71 2. device_tags
     72 3. inventories
     73 4. locations
     74 5. motherboards
     75 6. nodes
     76 7. peripherals
     77 8. testbeds
     78
     79=== device_kinds table ===
     80
     81|| Field        || Type        || Null || Key || Default || Extra          ||
     82|| id           || int(11)     || NO   || PRI || NULL    || auto_increment ||
     83|| inventory_id || int(11)     || NO   ||     || NULL    ||                ||
     84|| oui          || varchar(8)  || YES  ||     || NULL    ||                ||
     85|| bus          || varchar(16) || YES  ||     || NULL    ||                ||
     86|| vendor       || int(11)     || NO   ||     || NULL    ||                ||
     87|| device       || int(11)     || NO   ||     || NULL    ||                ||
     88
     89
     90=== device_tags table ===
     91
     92|| Field         || Type        || Null || Key || Default || Extra ||
     93|| tag           || varchar(64) || NO   ||     || NULL    ||       ||
     94|| peripheral_id || int(11)     || NO   ||     || NULL    ||       ||
     95
     96=== motherboards table ===
     97|| Field       || Type        || Null || Key || Default           || Extra          || Description                                   ||
     98|| id          || varchar(64) || NO   || PRI ||                   ||                || UUID of the motherboard                       ||
     99|| node_id     || varchar(64) || YES  || UNI || NULL              ||                || Link to 'id' in nodes table                   ||
     100|| sn   || varchar(16) || NO   || UNI ||                   ||                || manufacturer serial number of the motherboard ||
     101|| hd_sn        || varchar(16) || NO   || UNI ||                   ||                || Hard drive serial number                      ||
     102|| cpu_type         || varchar(X)  || YES  ||     || NULL              ||                || CPU Type                                      ||
     103|| cpu_speed       || int(11)     || YES  ||     || 0                 ||                || CPU speed in MHz                              ||
     104|| memory      || int(11)     || YES  ||     || 0                 ||                || Memory size in MB                             ||
     105|| hd_size        || int(11)     || YES  ||     || 0                 ||                || Hard disk size in bytes                          ||
     106|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
     107|| updated_by  || varchar(64) || NO   ||     || ||                ||                                               ||
     108
     109
     110(NOTE: 'node_id' is NULL when this motherboard is not installed on any node, i.e. new parts that just got in, or stored extra/spare parts)
     111
     112We could also move the hard-drive info in a separate table if we allow hard-drive swapping between motherboards.
     113
     114=== nodes table ===
     115|| Field       || Type        || Null || Key || Default || Extra          || Description ||
     116|| id          || varchar(64) || NO   || PRI ||         ||                || UUID of the node (i.e. the chassis). ||
     117|| chassis_sn  || varchar(16) || NO   || UNI ||         ||                || Manufacturer serial number of the node's chassis ||
     118|| location_id || varchar(64) || YES  || UNI || NULL    ||                || Link to 'id' in 'locations' table ||
     119|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
     120|| updated_by  || varchar(64) || NO   ||     ||  ||                ||                                               ||
     121
     122(NOTE: 'location_id' is NULL when this chassis is not installed at any location, i.e. new parts that just got in, or stored extra/spare parts)
     123
     124=== locations table ===
     125
     126|| Field       || Type        || Null || Key || Default           || Extra          || Description ||
     127|| id          || varchar(64) || NO   || PRI ||         ||                || UUID of the location ||
     128|| x           || int(11)     || NO   ||     || 0                 ||                || ||
     129|| y           || int(11)     || NO   ||     || 0                 ||                || ||
     130|| z           || int(11)     || NO   ||     || 0                 ||                || ||
     131|| unit        || int(11)     || NO   ||     || 0                 ||                || ||
     132|| testbed_id  || varchar(64) || NO   ||     || 0                 ||                || Link to 'id' in 'testbeds' table ||
     133|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
     134|| updated_by  || varchar(64) || NO   ||     ||  ||                ||                                               ||
     135
     136
     137=== testbeds (resources) table ===
     138
     139|| Field      || Type        || Null || Key || Default || Extra  || Description ||
     140|| id         || varchar(64)  || NO  || PRI ||         ||        || UUID of the testbed ||
     141|| domain     || varchar(4)  || NO   || UNI ||         ||        || ||
     142|| control_ip || varchar(12) || NO   || UNI ||         ||        || ||
     143|| data_ip    || varchar(12) || NO   || UNI ||         ||        || ||
     144|| cm_ip      || varchar(12) || NO   ||     ||         ||        || ||
     145|| latitude   || int(11)     || NO   ||     || 0       ||        || ||
     146|| longitude  || int(11)     || NO   ||     || 0       ||        || ||
     147|| elevation  || int(11)     || NO   ||     || 0       ||        || ||
     148|| updated_on  || timestamp   || NO   ||     || CURRENT_TIMESTAMP ||                ||                                               ||
     149|| updated_by  || varchar(64) || NO   ||     ||  ||                ||                                               ||
     150
     151== DESCRIPTION ==
     152
     153The design goal of this schema is to allow the double use of the Inventory database as:
     154  * a source of information for user experiment scripts
     155  * a 'real' hardware inventory giving operators information on which piece of hardware (chassis, motherboard) is used (or not) in which testbed/location.
     156
     157The entries in the ''testbeds'', ''locations'', ''nodes'' tables are manually created and updated by operators, when:
     158  * a new testbed is being deployed
     159  * a new location is added to the testbed (e.g. physical place-holder creation on a sandbox testbed for future addition of a third node)
     160  * a new purchased chassis (i.e. empty node box) is delivered, or mounted to a new location, or switched from a location to another one
     161
     162We do not expect these events to happen very often, thus it should be ok to make the operator responsible for creating/updating the related entries. (furthermore he/she could also use some scripts to do this job...)
     163
     164The entries in the ''motherboards'' table are also manually created upon delivery of a new purchased motherboard. The only field that needs to be manually filled by the operator is the ''node_id'', which will happen when the operator installs a new motherboard inside a node/chassis. All the other fields are automatically populated by the Inventory process (i.e. the scripts in the inventory package).
     165
     166The ''interfaces'' and ''devices'' tables are created and updated as in the previous schema.
     167
     168
     169