Changes between Version 21 and Version 22 of Hardware/bDomains/cSandboxes/iSB9


Ignore:
Timestamp:
Mar 8, 2021, 3:52:08 AM (3 years ago)
Author:
seskar
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • Hardware/bDomains/cSandboxes/iSB9

    v21 v22  
    33
    44SB9 is dedicated to wired networking and cloud computing experimentation. Nodes in sandbox 9 don't have wireless interfaces.
     5
     6[[Image(SB9 BD.png, 500px, link=, align=left)]][[BR]]Figure 1: SB9 Block Diagram
    57
    68Sandbox 9 consists of 8 servers, each with the following specifications:
    … …  
    1517* Accelerator2: Xilinx Alveo U200 FPGA (2x 100gbe ethernet onboard)
    1618
    17 They are connected to the following P4 Capable Switch:
    18 Edge Core Wedge100BF-32x
    19 * Ports: 32x 100g ports, each can break out to 2x 50, 2x 40, 4x 25, or 4x 10g ports.
    20 * OS: Azure SONIC operating system, or !OpenNetworkLinux
    21 * Management: In band, out of band, or via onboard BMC.
     19They are connected to two switches:
    2220
     21* Dell Z9100
    2322
     23* Edge Core Wedge100BF-32x
     24  * Ports: 32x 100g ports, each can break out to 2x 50, 2x 40, 4x 25, or 4x 10g ports.
     25  * OS: Azure SONIC operating system, or !OpenNetworkLinux
     26  * Management: In band, out of band, or via onboard BMC.
    2427
    25 
    26 ===== Procedures =====
     28===== Procedures (Wedge) =====
    2729Manual install of new switch os:
    2830Currently this must be done by a user with admin access. A service will be provided to automate this operation.
    … …  
    3436
    3537
    36 
    37 
    38 
    39 ===== Old =====
    40 All hardware below this line has been retired. The information is preserved for reference.
    41 
    42 SB9 is dedicated to [http://www.openflow.org OpenFlow] experimentation (NOTE: Nodes in SB9 don't have wireless interfaces!).
    43 As show in Figure 1, Sanbox 9 is built around an !OpenFlow capable switch with 12 nodes, 7 of which are equipped with [http://netfpga.org/1G_specs.html NetFPGA] cards, 2 nodes with [http://netfpga.org/10G_specs.html NetFPGA-10G] card and 2 general purpose ORBIT nodes and a sandbox console. In addition, there is a fast cache machine, with a 10g connection and a ssd storage array fast enough to saturate that link.
    44 
    45   ||   [[Image(SB9.jpg, 500px, link=, align=center)]] ||
    46   || [[Image(SB9_pic.JPG, 500px, align=center)]] ||
    47   Figure 1: SB9
    48 
    49 The switch labeled 'sw-sb-09', a [http://www.prontosys.net/pronto3290.htm Pronto 3290],  provides the central connectivity backplane for 'DATA' interface of all hosts/NetFPGAs in the sandbox. Each host (node1-1..node1-12) is connected to the sw-sb-09 switch through one 1GbE data connection on the interface 'eth0' or 'exp0'. As is the case with the rest of ORBIT nodes, a second GbE interface ('eth1' or 'control') of each node,  that is used exclusively for experiment control (incl. ssh/telnet sessions), is connected to an external control switch outside of sandbox.
    50 
    51 The first 7 hosts (node1-1..node1-7) each contain a 4x1GbE NetFPGA installed on a PCI slot. Each NetFPGA has 4 connections to the top switch, corresponding to its 4-GbE ports nf2c0-nf2c3.
    52 
    53 Nodes node1-8 and node1-9 each contain a 4x10GbE NetFPGA installed on a PCI Express slot. top switch while the remaining two are directly connecting the two nodes.
    54 
    55 Node1-12 has eth4 connected to the pronto switch at 10g, with eth5 currently unconnected. It also has 2 unconnected 10gbase-T ports, enumerated as eth2 and eth3.
    56 
    57 The Pronto switch is a !OpenFlow enabled switch and can be run in native or !OpenFlow mode by controlling its boot configuration. When in native mode, it runs the Pica8 Xorplus switch software while when it is in !OpenFlow mode it can run either stock Indigo firmware or experimenter provided !OpenFlow image for Pronto switches. The mode of operation is controlled by the [wiki:Software/AM/Network network aggregate manager] service.
    58 
    59 || Switchport # || Node || Device || Interface # ||
    60 || 1 ||  ||  ||  ||
    61 || 2 || 1-1 || NetFPGA || Port4 ||
    62 || 3 || 1-1 || NetFPGA || Port3 ||
    63 || 4 || 1-1 || NetFPGA || Port2 ||
    64 || 5 || 1-1 || Card || Data ||
    65 || 6 || || || ||
    66 || 7 || 1-2 || NetFPGA || Port4 ||
    67 || 8 || 1-2 || NetFPGA || Port3 ||
    68 || 9 || || || ||
    69 || 10 || 1-2 || Card || Data ||
    70 || 11 || || || ||
    71 || 12 || 1-3 || NetFPGA || Port4 ||
    72 || 13 || 1-3 || NetFPGA || Port3 ||
    73 || 14 || 1-3 || NetFPGA || Port2 ||
    74 || 15 || 1-3 || Card || Data ||
    75 || 16 || || || ||
    76 || 17 || 1-4 || NetFPGA || Port4 ||
    77 || 18 || 1-4 || NetFPGA || Port3 ||
    78 || 19 || 1-4 || NetFPGA || Port2 ||
    79 || 20 || 1-4 || Card || Data ||
    80 || 21 || || || ||
    81 || 22 || 1-5 || NetFPGA || Port4 ||
    82 || 23 || || || ||
    83 || 24 || 1-5 || NetFPGA || Port2 ||
    84 || 25 || 1-5 || Card || Data ||
    85 || 26 || || || ||
    86 || 27 || 1-6 || NetFPGA || Port3 ||
    87 || 28 || 1-6 || NetFPGA || Port1 ||
    88 || 29 || 1-6 || NetFPGA || Port2 ||
    89 || 30 || 1-6 || Card || Data ||
    90 || 31 || || || ||
    91 || 32 || 1-7 || NetFPGA || Port4 ||
    92 || 33 || 1-7 || NetFPGA || Port3 ||
    93 || 34 || 1-7 || NetFPGA || Port1 ||
    94 || 35 || 1-7 || Card || Data ||
    95 || 36 || 1-11 || Onboard || Data ||
    96 || 37 || || || ||
    97 || 38 || || || ||
    98 || 39 || 1-10 || Onboard || Data ||
    99 || 40 || 1-8 || Card || Data ||
    100 || 41 || || || ||                       
    101 || 42 || || || ||
    102 || 43 || || || ||
    103 || 44 || || || ||
    104 || 45 || 1-9 || Card || Data ||
    105 || 46 || || || ||
    106 || 47 || || || ||
    107 || 48 || || || ||
    108 || te-1/1/49 || 1-12 || Card || Data ||
    109 || te-1/1/50 || ? || NetFPGA || ? ||
    110 || te-1/1/51 || ? || NetFPGA || ? ||
    111 || te-1/1/52 || ? || NetFPGA || ? ||
    112 || Eth1 || Management || SW-SB-02 || 44 ||
    113 || Eth2 || Management || || ||
    114 || Aux || || || ||
    115 || Con || Management || S-ORBW || Serial1 ||