ibnetdiscover
(1m)
Name
ibnetdiscover - discover InfiniBand topology
Synopsis
ibnetdiscover [-d(ebug)] [-e(rr_show)] [-v(erbose)]
[-s(how)] [-l(ist)] [-g(rouping)] [-H(ca_list)]
[-S(witch_list)] [-R(outer_list)] [-C ca_name] [-P ca_port]
[-t(imeout) timeout_ms] [-V(ersion)] [--node-name-map <node-
name-map>] [--cache <filename>] [--load-cache <filename>]
[-p(orts)] [-m(ax_hops)] [-h(elp)] [<topology-file>]
Description
OpenIB Diagnostics IBNETDISCOVER(1m)
NAME
ibnetdiscover - discover InfiniBand topology
SYNOPSIS
ibnetdiscover [-d(ebug)] [-e(rr_show)] [-v(erbose)]
[-s(how)] [-l(ist)] [-g(rouping)] [-H(ca_list)]
[-S(witch_list)] [-R(outer_list)] [-C ca_name] [-P ca_port]
[-t(imeout) timeout_ms] [-V(ersion)] [--node-name-map <node-
name-map>] [--cache <filename>] [--load-cache <filename>]
[-p(orts)] [-m(ax_hops)] [-h(elp)] [<topology-file>]
DESCRIPTION
ibnetdiscover performs IB subnet discovery and outputs a
human readable topology file. GUIDs, node types, and port
numbers are displayed as well as port LIDs and NodeDescrip-
tions. All nodes (and links) are displayed (full topology).
Optionally, this utility can be used to list the current
connected nodes by nodetype. The output is printed to stan-
dard output unless a topology file is specified.
OPTIONS
-l, --list
List of connected nodes
-g, --grouping
Show grouping. Grouping correlates IB nodes by differ-
ent vendor specific schemes. It may also show the
switch external ports correspondence.
-H, --Hca_list
List of connected CAs
-S, --Switch_list
List of connected switches
-R, --Router_list
List of connected routers
-s, --show
Show progress information during discovery.
--node-name-map <node-name-map>
Specify a node name map. The node name map file maps
GUIDs to more user friendly names. See file format
below.
--cache <filename>
Cache the ibnetdiscover network data in the specified
filename. This cache may be used by other tools for
OpenIB Last change: May 13, 2009 1
OpenIB Diagnostics IBNETDISCOVER(1m)
later analysis.
--load-cache <filename>
Load and use the cached ibnetdiscover data stored in
the specified filename. May be useful for outputting
and learning about other fabrics or a previous state of
a fabric.
--diff <filename>
Load cached ibnetdiscover data and do a diff comparison
to the current network or another cache. A special
diff output for ibnetdiscover output will be displayed
showing differences between the old and current fabric.
By default, the following are compared for differences:
switches, channel adapters, routers, and port connec-
tions.
--diffcheck <key(s)>
Specify what diff checks should be done in the --diff
option above. Comma separate multiple diff check
key(s). The available diff checks are: sw = switches,
ca = channel adapters, router = routers, port = port
connections, lid = lids, nodedesc = node descriptions.
Note that port, lid, and nodedesc are checked only for
the node types that are specified (e.g. sw, ca,
router). If port is specified alongside lid or nod-
edesc, remote port lids and node descriptions will also
be compared.
-p, --ports
Obtain a ports report which is a list of connected
ports with relevant information (like LID, portnum,
GUID, width, speed, and NodeDescription).
-m, --max_hops
Report max hops discovered.
COMMON OPTIONS
Most OpenIB diagnostics take the following common flags. The
exact list of supported flags per utility can be found in
the usage message and can be shown using the util_name -h
syntax.
# Debugging flags
-d raise the IB debugging level.
May be used several times (-ddd or -d -d -d).
OpenIB Last change: May 13, 2009 2
OpenIB Diagnostics IBNETDISCOVER(1m)
-e show send and receive errors (timeouts and others)
-h show the usage message
-v increase the application verbosity level.
May be used several times (-vv or -v -v -v)
-V show the version info.
# Other common flags:
-C <ca_name> use the specified ca_name.
-P <ca_port> use the specified ca_port.
-t <timeout_ms> override the default timeout for the
solicited mads.
Multiple CA/Multiple Port Support
When no IB device or port is specified, the port to use is
selected by the following criteria:
1. the first port that is ACTIVE.
2. if not found, the first port that is UP (physical link
up).
If a port and/or CA name is specified, the user request is
attempted to be fulfilled, and will fail if it is not possi-
ble.
TOPOLOGY FILE FORMAT
The topology file format is human readable and largely intu-
itive. Most identifiers are given textual names like vendor
ID (vendid), device ID (device ID), GUIDs of various types
(sysimgguid, caguid, switchguid, etc.). PortGUIDs are shown
in parentheses (). For switches, this is shown on the
switchguid line. For CA and router ports, it is shown on
the connectivity lines. The IB node is identified followed
by the number of ports and a quoted the node GUID. On the
right of this line is a comment (#) followed by the Nod-
eDescription in quotes. If the node is a switch, this line
also contains whether switch port 0 is base or enhanced, and
the LID and LMC of port 0. Subsequent lines pertaining to
this node show the connectivity. On the left is the port
number of the current node. On the right is the peer node
(node at other end of link). It is identified in quotes with
nodetype followed by - followed by NodeGUID with the port
number in square brackets. Further on the right is a com-
ment (#). What follows the comment is dependent on the node
OpenIB Last change: May 13, 2009 3
OpenIB Diagnostics IBNETDISCOVER(1m)
type. If it it a switch node, it is followed by the Nod-
eDescription in quotes and the LID of the peer node. If it
is a CA or router node, it is followed by the local LID and
LMC and then followed by the NodeDescription in quotes and
the LID of the peer node. The active link width and speed
are then appended to the end of this output line.
An example of this is:
#
# Topology file: generated on Tue Jun 5 14:15:10 2007
#
# Max of 3 hops discovered
# Initiated from node 0008f10403960558 port 0008f10403960559
Non-Chassis Nodes
vendid=0x8f1
devid=0x5a06
sysimgguid=0x5442ba00003000
switchguid=0x5442ba00003080(5442ba00003080)
Switch 24 "S-005442ba00003080" # "ISR9024 Voltaire" base port 0 lid 6 lmc 0
[22] "H-0008f10403961354"[1](8f10403961355) # "MT23108 InfiniHost Mellanox Technologies" lid 4 4xSDR
[10] "S-0008f10400410015"[1] # "SW-6IB4 Voltaire" lid 3 4xSDR
[8] "H-0008f10403960558"[2](8f1040396055a) # "MT23108 InfiniHost Mellanox Technologies" lid 14 4xSDR
[6] "S-0008f10400410015"[3] # "SW-6IB4 Voltaire" lid 3 4xSDR
[12] "H-0008f10403960558"[1](8f10403960559) # "MT23108 InfiniHost Mellanox Technologies" lid 10 4xSDR
vendid=0x8f1
devid=0x5a05
switchguid=0x8f10400410015(8f10400410015)
Switch 8 "S-0008f10400410015" # "SW-6IB4 Voltaire" base port 0 lid 3 lmc 0
[6] "H-0008f10403960984"[1](8f10403960985) # "MT23108 InfiniHost Mellanox Technologies" lid 16 4xSDR
[4] "H-005442b100004900"[1](5442b100004901) # "MT23108 InfiniHost Mellanox Technologies" lid 12 4xSDR
[1] "S-005442ba00003080"[10] # "ISR9024 Voltaire" lid 6 1xSDR
[3] "S-005442ba00003080"[6] # "ISR9024 Voltaire" lid 6 4xSDR
vendid=0x2c9
devid=0x5a44
caguid=0x8f10403960984
Ca 2 "H-0008f10403960984" # "MT23108 InfiniHost Mellanox Technologies"
[1](8f10403960985) "S-0008f10400410015"[6] # lid 16 lmc 1 "SW-6IB4 Voltaire" lid 3 4xSDR
vendid=0x2c9
devid=0x5a44
caguid=0x5442b100004900
Ca 2 "H-005442b100004900" # "MT23108 InfiniHost Mellanox Technologies"
[1](5442b100004901) "S-0008f10400410015"[4] # lid 12 lmc 1 "SW-6IB4 Voltaire" lid 3 4xSDR
vendid=0x2c9
devid=0x5a44
caguid=0x8f10403961354
Ca 2 "H-0008f10403961354" # "MT23108 InfiniHost Mellanox Technologies"
OpenIB Last change: May 13, 2009 4
OpenIB Diagnostics IBNETDISCOVER(1m)
[1](8f10403961355) "S-005442ba00003080"[22] # lid 4 lmc 1 "ISR9024 Voltaire" lid 6 4xSDR
vendid=0x2c9
devid=0x5a44
caguid=0x8f10403960558
Ca 2 "H-0008f10403960558" # "MT23108 InfiniHost Mellanox Technologies"
[2](8f1040396055a) "S-005442ba00003080"[8] # lid 14 lmc 1 "ISR9024 Voltaire" lid 6 4xSDR
[1](8f10403960559) "S-005442ba00003080"[12] # lid 10 lmc 1 "ISR9024 Voltaire" lid 6 1xSDR
When grouping is used, IB nodes are organized into chasses
which are numbered. Nodes which cannot be determined to be
in a chassis are displayed as "Non-Chassis Nodes". External
ports are also shown on the connectivity lines.
NODE NAME MAP FILE FORMAT
The node name map is used to specify user friendly names for
nodes in the output. GUIDs are used to perform the lookup.
Generically:
# comment
<guid> "<name>"
Example:
# IB1
# Line cards
0x0008f104003f125c "IB1 (Rack 11 slot 1 )
ISR9288/ISR9096 Voltaire sLB-24D"
0x0008f104003f125d "IB1 (Rack 11 slot 1 )
ISR9288/ISR9096 Voltaire sLB-24D"
0x0008f104003f10d2 "IB1 (Rack 11 slot 2 )
ISR9288/ISR9096 Voltaire sLB-24D"
0x0008f104003f10d3 "IB1 (Rack 11 slot 2 )
ISR9288/ISR9096 Voltaire sLB-24D"
0x0008f104003f10bf "IB1 (Rack 11 slot 12 )
ISR9288/ISR9096 Voltaire sLB-24D"
# Spines
0x0008f10400400e2d "IB1 (Rack 11 spine 1 ) ISR9288
Voltaire sFB-12D"
0x0008f10400400e2e "IB1 (Rack 11 spine 1 ) ISR9288
Voltaire sFB-12D"
0x0008f10400400e2f "IB1 (Rack 11 spine 1 ) ISR9288
Voltaire sFB-12D"
0x0008f10400400e31 "IB1 (Rack 11 spine 2 ) ISR9288
Voltaire sFB-12D"
0x0008f10400400e32 "IB1 (Rack 11 spine 2 ) ISR9288
Voltaire sFB-12D"
OpenIB Last change: May 13, 2009 5
OpenIB Diagnostics IBNETDISCOVER(1m)
# GUID Node Name
0x0008f10400411a08 "SW1 (Rack 3) ISR9024 Voltaire
9024D"
0x0008f10400411a28 "SW2 (Rack 3) ISR9024 Voltaire
9024D"
0x0008f10400411a34 "SW3 (Rack 3) ISR9024 Voltaire
9024D"
0x0008f104004119d0 "SW4 (Rack 3) ISR9024 Voltaire
9024D"
AUTHORS
Hal Rosenstock
<[email protected]>
Ira Weiny
<[email protected]>
ATTRIBUTES
See attributes(5) for descriptions of the following
attributes:
+---------------+----------------------+
|ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+---------------+----------------------+
|Availability | network/open-fabrics |
+---------------+----------------------+
|Stability | Volatile |
+---------------+----------------------+
NOTES
This software was built from source available at
https://java.net/projects/solaris-userland. The original
community source was downloaded from ['http://www.openfab-
rics.org/downloads/ibutils/ibutils-1.5.7.tar.gz',
'http://www.openfabrics.org/downloads/libibverbs/libib-
verbs-1.1.4-1.22.g7257cd3.tar.gz', 'http://www.openfab-
rics.org/down-
loads/libmlx4/libmlx4-1.0.1-1.18.gb810a27.tar.gz',
'http://www.openfabrics.org/downloads/libsdp/lib-
sdp-1.1.108-0.15.gd7fdb72.tar.gz', 'http://www.openfab-
rics.org/downloads/management/infiniband-
diags-1.5.8.tar.gz', 'http://www.openfabrics.org/down-
loads/management/libibmad-1.3.7.tar.gz', 'http://www.open-
fabrics.org/downloads/management/libibumad-1.3.7.tar.gz',
'http://www.openfabrics.org/downloads/manage-
ment/opensm-3.3.9.tar.gz', 'http://www.openfabrics.org/down-
loads/perftest/perftest-1.3.0-0.42.gf350d3d.tar.gz',
'http://www.openfabrics.org/down-
loads/qperf/qperf-0.4.6-0.1.gb81434e.tar.gz',
'http://www.openfabrics.org/downloads/rdmacm/librd-
OpenIB Last change: May 13, 2009 6
OpenIB Diagnostics IBNETDISCOVER(1m)
macm-1.0.14.1.tar.gz', 'http://www.openfabrics.org/down-
loads/rds-tools/rds-tools-2.0.4.tar.gz']
Further information about this software can be found on the
open source community website at http://www.openfab-
rics.org/.
OpenIB Last change: May 13, 2009 7