Basic Commands

IB Basic Commands

ibstat

 Displays operational information about one or more InfiniBand network devices. This command displays InfiniBand operational
 information pertaining to a specified Host Channel Adapter Device (HCAD). If an HCAD device name is not entered, status for 
 all available HCADs are displayed. Select a flag to narrow down your search results. You can display specific categories of 
 information, including Node, Port, Interface, and Debug information. You can also choose to display all of the information 
 categories.

 Flags:
 -d	Displays current debug setting.
 -h	Displays ibstat command usage.
 -i	Displays network interface information.
 -n	Displays IB node information.
 -p	Displays IB port information.
 -v	Displays all IB device information.

Example output:

 [markus]# ibstat
 CA 'mlx4_0'
       CA type: MT26428
       Number of ports: 1
       Firmware version: 2.9.1000
       Hardware version: b0
       Node GUID: 0x0002c903004af586
       System image GUID: 0x0002c903004af589
       Port 1:
               State: Active
               Physical state: LinkUp
               Rate: 40
               Base lid: 250
               LMC: 0
               SM lid: 250
               Capability mask: 0x0251086a
               Port GUID: 0x0002c903004af587
               Link layer: InfiniBand

ibv_devinfo

 Print information about RDMA devices available for use from user space.

 Flags:
 -d, --ib-dev=DEVICE   use IB device DEVICE (default first device found)
 -i, --ib-port=PORT    query port PORT (default all ports)
 -l, --list            only list names of RDMA devices
 -v, --verbose         print all available

Example output:

 [markus]# ibv_devinfo 
 hca_id:	mlx4_0
       transport:			InfiniBand (0)
       fw_ver:				2.9.1000
       node_guid:			0002:c903:004a:f586
       sys_image_guid:			0002:c903:004a:f589
       vendor_id:			0x02c9
       vendor_part_id:			26428
       hw_ver:				0xB0
       board_id:			MT_0D90110009
       phys_port_cnt:			1
               port:	1
                       state:			PORT_ACTIVE (4)
                       max_mtu:		4096 (5)
                       active_mtu:		4096 (5)
                       sm_lid:			250
                       port_lid:		250
                       port_lmc:		0x00
                       link_layer:		InfiniBand

sminfo

Query InfiniBand SMInfo attribute.
Optionally set and display the output of a sminfo query in human readable format. 
This command indicates where the running SM is.

Example output:

sminfo: sm lid 250 sm guid 0x2c903004af587, activity count 1828732680 priority 15 state 3 SMINFO_MASTER

ibping

ibping uses vendor mads to validate connectivity between IB nodes. 
The destination is specified by lid.
On exit, (IP) ping like output is show. ibping is run as client/server. Default is to run as client. 
Note also that a default ping server is implemented within the kernel.

Flags:
-c            stop after count packets
-f, --flood   flood destination: send packets back to back without delay
-o, --oui     use specified OUI number to multiplex vendor mads
-S, --Server  start in server mode (do not return)

Server: (server has a lid of 250)

 [markus]# ibping -S

Client:

 [mu0007 ~]# ibping 250
 Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.196 ms
 Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.102 ms
 Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.141 ms

ib_tracert

Traces the path from Source GID/LID to Destinatino GID/LID
Each hop along the path is displayed until the destination is reached or a hop does not respond. 
By using the -m option, multicast path tracing can be performed between source and destination nodes.

Flags:
-n, --no_info                   simple format; don't show additional information
-m                              show the multicast trace of the specified mlid
--node-name-map <node-name-map> Specify a node name map. The node name map file maps GUIDs to more user friendly names.

Example output:

 [root@mu-master markus]# ibtracert 250 199
 From ca {0x0002c903004af586} portnum 1 lid 250-250 "mu-master"
 [1] -> switch port {0x0002c902004484c0}[36] lid 427-427 "muib91"
 [7] -> switch port {0x0002c90200449680}[2] lid 2-2 "muibcore1 Line 32"
 [19] -> switch port {0x0002c902004486f0}[32] lid 449-449 "muibcore1 Spine 01"
 [3] -> switch port {0x0002c90200421750}[19] lid 65-65 "muibcore1 Line 03"
 [1] -> switch port {0x0002c902004492c0}[5] lid 433-433 "muib1"
 [14] -> ca port {0x002590ffff167f8d}[1] lid 199-199 "mu0007"
 To ca {0x002590ffff167f8c} portnum 1 lid 199-199 "mu0007"