文章目录

1. Mellanox固件升级及管理工具使用介绍

Mellanox网卡固件下载地址：https://www.mellanox.com/support/firmware/firmware-downloads

mlxup-Mellanox更新和查询实用程序：https://www.mellanox.com/support/firmware/mlxup-mft

Mellanox固件工具（MFT）：https://www.mellanox.com/products/adapter-software/firmware-tools

MFT 使用指导：https://www.mellanox.com/related-docs/MFT/MFT_user_manual_3_8_0.pdf

介绍

Mellanox提供了两种固件工具来更新和查询适配器固件：

  • mlxup -Mellanox固件更新和查询实用程序。该实用程序可以在服务器计算机上扫描可用的Mellanox适配器，并指示是否需要为每个适配器更新固件。为了使我们的客户更轻松，mlxup还可以从Web或使用本地提供的二进制文件自动更新最新固件。
    对于其他功能，例如卡配置和二进制创建，请使用**Mellanox固件工具（MFT）**。
  • MFT -MFT软件包是一组固件管理工具，用于生成标准或自定义的Mellanox固件映像，查询固件信息以及刻录固件映像。

1. MFT工具安装

进入MFT 下载界面：选择对应的平台下载.

这里下载的是linux版本：

在这里插入图片描述

下载后解压进入对应的目录：执行./install.sh即可安装。

[root@localhost work]# cd mft-4.16.3-12-x86_64-rpm/
[root@localhost mft-4.16.3-12-x86_64-rpm]# ls
install.sh  LICENSE.txt  old-mft-uninstall.sh  RPMS  SRPMS  uninstall.sh
[root@localhost mft-4.16.3-12-x86_64-rpm]# ll
total 52
-rwxr-xr-x. 1 70705 30 22838 Mar 22 06:32 install.sh
-rwxr-xr-x. 1 70705 30 13841 Mar 22 06:32 LICENSE.txt
-rwxr-xr-x. 1 70705 30  7622 Mar 22 06:32 old-mft-uninstall.sh
drwxr-xr-x. 2 70705 30   111 Mar 22 06:32 RPMS
drwxr-xr-x. 2 70705 30    42 Mar 22 06:32 SRPMS
-rwxr-xr-x. 1 70705 30  1577 Mar 22 06:32 uninstall.sh
[root@localhost mft-4.16.3-12-x86_64-rpm]# ./install.sh
-I- Removing any old MFT file if exists...
-I- Building the MFT kernel binary RPM...
-I- Installing the MFT RPMs...
Preparing...                          ################################# [100%]
Updating / installing...
   1:kernel-mft-4.16.3-3.10.0_957.el7.################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:mft-4.16.3-12                    ################################# [100%]
-I- In order to start mst, please run "mst start".
[root@localhost mft-4.16.3-12-x86_64-rpm]# mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
Unloading MST PCI module (unused) - Success

详细指导可以查看：MFT 使用指导。

2. 固件下载

记得先安装MFT工具，然后查询才知道下载哪个固件；

根据以下步骤查询需要哪个网卡的固件：

[root@localhost mft-4.16.3-12-x86_64-rpm]# lspci|grep Mell
56:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
56:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
[root@localhost mft-4.16.3-12-x86_64-rpm]# flint -d 56:00.0 q
Image type:            FS3
FW Version:            14.22.1002
FW Release Date:       23.2.2018
Product Version:       rel-14_22_1002
Rom Info:              type=UEFI version=14.15.19 cpu=AMD64,AARCH64
                       type=PXE version=3.5.403 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             0c42a103008c51a8        4
Base MAC:              0c42a18c51a8            4
Image VSD:             N/A
Device VSD:            N/A
PSID:                  MT_2420110034
Orig PSID:             HUA0000000001
Security Attributes:   N/A

那么这里的 PSID字段对应的值MT_2420110034就是我们要下载的固件版本：进入固件下载地址,选择对应的网卡，点击下载即可。

一个一个选择OPN，然后找到对应的PSID字段值。

在这里插入图片描述

3. 固件烧录

# Genaral 
flint -d <device> -i <fw-file> [-guid <GUID> | -guids <4 GUIDS> | -mac <MAC> | -macs <2 MACs>] burn

# Example
flint -d /dev/mst/mt4117_pciconf0 -i fw-xxxxxxxx.bin burn

详细指导可以查看：MFT 使用指导。

2. Mellanox驱动升级介绍

针对华为服务器，进入iDriver目录，下载对应OS驱动；
解压后可以看到readme.txt文件，根据对应步骤手动升级即可；

IB_NIC-CX4_CX4Lx_CX5_CX6-CentOS7.6-mlx5_core-5.1-0.6.6-1-x86_64.tgz
--- CX5 driver/ CX4 driver/ CX6 driver/ CX4Lx driver
#Note：Please replace the xxx.tgz with the archived package name for the actual driver installation,
and need to install tcl and tk tools before installing the driver.
#Note1:As of MLNX_OFED v5.1, the following are no longer supported and can be utilized through MLNX_OFED LTS version:ConnectX-3 Pro、ConnectX-3
1. Use the command "tar -zxvf xxx.tgz" to extract the driver package.
2. Go to the unzipped directory, use the command "./mlnxofedinstall" and select "Y" to upgrade the driver.
Note2: If the driver cannot be installed normally using the command "./mlnxofedinstall",  follow the prompts to upgrade with "./mlnxofedinstall --force"
Note3: If a message indicating that the dependency package is missing is displayed during the installation, install the dependency package before performing the installation.
3. Run the reboot command.
...
  • 实操

1. 解压文件

解压进目录onboard_driver_CentOS7.6;
可以看到压缩包IB_NIC-CX4_CX4Lx_CX5_CX6-CentOS7.6-mlx5_core-5.1-0.6.6-1-x86_64.tgz
继续解压进入对应目录： MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64

[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# ll
total 400
-rwxr-xr-x. 1 root root   2848 Jul 28  2020 common_installers.pl
-rwxr-xr-x. 1 root root   8244 Jul 28  2020 common.pl
-rwxr-xr-x. 1 root root  24888 Jul 28  2020 create_mlnx_ofed_installers.pl
-rw-r--r--. 1 root root      8 Jul 28  2020 distro
drwxr-xr-x. 8 root root    210 Jul 28  2020 docs
-rwxr-xr-x. 1 root root   4838 Jul 28  2020 is_kmp_compat.sh
-rw-r--r--. 1 root root    956 Jul 28  2020 LICENSE
-rwxr-xr-x. 1 root root  25792 Jul 28  2020 mlnx_add_kernel_support.sh
-rwxr-xr-x. 1 root root 285831 Jul 28  2020 mlnxofedinstall
-rw-r--r--. 1 root root   2764 Jul 28  2020 RPM-GPG-KEY-Mellanox
drwxr-xr-x. 3 root root   8192 Jul 28  2020 RPMS
drwxr-xr-x. 2 root root     43 Jul 28  2020 src
-rwxr-xr-x. 1 root root  15150 Jul 28  2020 uninstall.sh

2. 安装依赖

yum install tcl tk

3. 升级驱动

cd MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64
[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# ./mlnxofedinstall
Logs dir: /tmp/MLNX_OFED_LINUX.52962.logs
General log file: /tmp/MLNX_OFED_LINUX.52962.logs/general.log
Verifying KMP rpms compatibility with target kernel...
This program will install the MLNX_OFED_LINUX package on your machine.
Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed.
Those packages are removed due to conflicts with MLNX_OFED_LINUX, do not reinstall them.

Do you want to continue?[y/N]:y

Uninstalling MLNX_EN driver

rpm --nosignature -e --allmatches --nodeps libibverbs libibverbs libibverbs libibverbs librdmacm mft libibverbs librdmacm

Starting MLNX_OFED_LINUX-5.1-0.6.6.0 installation ...

...
...
mpitests_openmpi-3.2.20-5d20b49.51066 ########################################
Device (56:00.0):
        56:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
        Link Width: x8
        PCI Link Speed: 8GT/s

Device (56:00.1):
        56:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
        Link Width: x8
        PCI Link Speed: 8GT/s


Installation finished successfully.


Preparing...                          ################################# [100%]
Updating / installing...
   1:mlnx-fw-updater-5.1-0.6.6.0      ################################# [100%]

Added 'RUN_FW_UPDATER_ONBOOT=no to /etc/infiniband/openib.conf

Initializing...
Attempting to perform Firmware update...
Querying Mellanox devices firmware ...

Device #1:
...
...

---------
Found 1 device(s) requiring firmware update...

Device #1: Updating FW ...                                                                                                                                 Done

Restart needed for updates to take effect.
Log File: /tmp/MLNX_OFED_LINUX.52962.logs/fw_update.log
To load the new driver, run:
/etc/init.d/openibd restart
[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# /etc/init.d/openibd restart
Unloading rdma_cm[FAILED]
rmmod: ERROR: Module rdma_cm is in use by: rpcrdma ib_isert

这里有个特殊点，如果出现以上报错 ： rmmod: ERROR: Module rdma_cm is in use by: rpcrdma ib_isert

把对应的模块停掉就好了。

[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# modprobe -r rpcrdma 
[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# modprobe -r ib_isert
[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# modprobe -r i40iw/etc/init.d/openibd restart
Unloading HCA driver:[  OK  ]
Loading HCA driver and Access Layer:[  OK  ]

[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# 
[root@localhost MLNX_OFED_LINUX-5.1-0.6.6.0-rhel7.6-x86_64]# reboot

案例分享

1. BIOS 无法找到CX4网卡对应PXE设备；

问题：服务器有CX4网卡， 但是BIOS无法找到对应的PXE设备。

在这里插入图片描述

ConnectX-4网卡使能 PXE UEFI boot；

安装MFT工具后，查看EXP_ROM_UEFI_x86_ENABLE字段是否开启，如果未开启，需要按照以下步骤打开，即可在BIOS找到对应的网口；

[root@localhost work]# mlxconfig -d 56:00.0 q | grep UEFI
         UEFI_HII_EN                         False(0)
         UEFI_LOGS                           DISABLED(0)
         EXP_ROM_UEFI_ARM_ENABLE             False(0)
         EXP_ROM_UEFI_x86_ENABLE             False(0)
[root@localhost work]# mlxconfig -d 56:00.0 s UEFI_HII_EN=1 EXP_ROM_UEFI_x86_ENABLE=1

Device #1:
----------

Device type:    ConnectX4LX
Name:           N/A
Description:    N/A
Device:         56:00.0

Configurations:                              Next Boot       New
         UEFI_HII_EN                         False(0)        True(1)
         EXP_ROM_UEFI_x86_ENABLE             False(0)        True(1)

 Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.
Logo
华为云开发者联盟

为开发者提供学习成长、分享交流、生态实践、资源工具等服务，帮助开发者快速成长。

更多推荐

cover

Redis开源社区持续壮大，华为云为Valkey项目注入新的活力

avatar 华为云开发者联盟
cover

Python文本统计与分析：从基础到进阶

avatar 华为云开发者联盟
cover

教你用Perl实现Smgp协议

avatar 华为云开发者联盟