Alexander, Thanks for the reminder! I realized that the new build module is with version "3.10.0" and so not match with my running kernel version "3.10.0-957.el7.s390x". I do confirm that i was using the right kernel source rpm. So, the question turns to why the "make modules" command changed the kernel version of built modules to "3.10.0" instead of "3.10.0-957.el7.s390x". Is there any special steps i need to follow to keep the original kernel naming convention? btw: I'm following below article to compile the kernel modules. https://wiki.centos.org/HowTos/BuildingKernelModules Thanks! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [root@bz094f ~]# modinfo ib_ipoib filename: /lib/modules/3.10.0-957.el7.s390x/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko license: Dual BSD/GPL description: IP-over-InfiniBand net driver author: Roland Dreier alias: rtnl-link-ipoib rhelversion: 7.6 srcversion: 917AA4365B3509F0B347217 depends: ib_core,ib_cm intree: Y vermagic: 3.10.0 SMP mod_unload modversions parm: max_nonsrq_conn_qp:Max number of connected-mode QPs per interface (applied only if shared receive queue is not available) (int) parm: cm_data_debug_level:Enable data path debug tracing for connected mode if > 0 (int) parm: mcast_debug_level:Enable multicast debug tracing if > 0 (int) parm: data_debug_level:Enable data path debug tracing if > 0 (int) parm: send_queue_size:Number of descriptors in send queue (int) parm: recv_queue_size:Number of descriptors in receive queue (int) parm: ipoib_enhanced:Enable IPoIB enhanced for capable devices (default = 1) (0-1) (int) parm: debug_level:Enable debug tracing if > 0 (int) [root@bz094f ~]# [root@bz094f ~]# uname -ar Linux bz094f 3.10.0-957.el7.s390x #1 SMP Thu Oct 4 16:53:20 EDT 2018 s390x s390x s390x GNU/Linux [root@bz094f ~]# ls -l /home/zbrand/sourcerpm/ total 102468 -rw-r--r--. 1 zbrand zbrand 257644 Dec 8 13:44 asciidoc-8.6.8-5.el7.noarch.rpm -rw-r--r--. 1 zbrand zbrand 1551156 Dec 8 13:45 glibc-static-2.17-260.el7.s390x.rpm -rw-r--r--. 1 zbrand zbrand 1307048 Dec 8 14:01 graphviz-2.30.1-21.el7.s390x.rpm -rw-r--r--. 1 zbrand zbrand 101028281 Dec 8 13:15 kernel-3.10.0-957.el7.src.rpm -rw-r--r--. 1 zbrand zbrand 52620 Dec 8 13:44 newt-devel-0.52.15-4.el7.s390x.rpm -rw-r--r--. 1 zbrand zbrand 93176 Dec 8 14:07 slang-devel-2.2.4-11.el7.s390x.rpm -rw-r--r--. 1 zbrand zbrand 625688 Dec 8 14:02 source-highlight-3.1.6-6.el7.s390x.rpm [root@bz094f ~]# -- Regards FuLong Wang _______________________________________________ On 12/9/20 23:19, Alexander Kapshuk wrote:
On Wed, Dec 9, 2020 at 11:31 AM FuLong Wang <fulwang@linux.vnet.ibm.com> wrote:
Hello Experts,
I'm trying to enable some infiniband related kernel modules (ib_ipoib, iser, isert, etc) on my RHEL 7.6 instance running on IBM linuxONE machine. (I have found the code in the kernel source tree.)
After installing the kernel source rpm in a normal user and make module selection using the "make menuconfig" command, i can successfully re-built all the selected modules (include the new selection) with the "make modules" command.
I copied the desired module files to the "/lib/modules/3.10.0-957.el7.s390x/kernel/drivers/infiniband/ulp" dir and the depmod -a command also was executed with no errors.
[root@bz094f ulp]# pwd /lib/modules/3.10.0-957.el7.s390x/kernel/drivers/infiniband/ulp [root@bz094f ulp]# [root@bz094f ulp]# ls -l ipoib total 5704 -rw-r--r--. 1 root root 5837016 Dec 9 15:50 ib_ipoib.ko [root@bz094f ulp]# ls -l iser total 2880 -rw-r--r--. 1 root root 2946496 Dec 9 15:50 ib_iser.ko [root@bz094f ulp]# ls -l isert total 1384 -rw-r--r--. 1 root root 1414304 Dec 9 15:50 ib_isert.ko [root@bz094f ulp]#
But after reboot the OS, i found the new copied modules can't be loaded by the kernel.
errors reported in the /var/log/message:
Dec 9 15:49:05 bz094f systemd-modules-load: Failed to insert 'ib_ipoib': Exec format error Dec 9 15:49:05 bz094f systemd-modules-load: Failed to find module 'ib_umad' Dec 9 15:49:05 bz094f systemd: rdma-load-modules@infiniband.service: main process exited, code=exited, status=1/FAILURE Dec 9 15:49:05 bz094f systemd: Failed to start Load RDMA modules from /etc/rdma/modules/infiniband.conf. Dec 9 15:49:05 bz094f systemd: Unit rdma-load-modules@infiniband.service entered failed state. Dec 9 15:49:05 bz094f systemd: rdma-load-modules@infiniband.service failed. Dec 9 15:49:05 bz094f kernel: ib_ipoib: no symbol version for module_layout Dec 9 15:49:05 bz094f systemd-modules-load: Failed to insert 'ib_iser': Exec format error Dec 9 15:49:05 bz094f kernel: ib_iser: no symbol version for module_layout
when load the module manually, it will produce the same error:
[root@bz094f ~]# modprobe ib_ipoib modprobe: ERROR: could not insert 'ib_ipoib': Exec format error
The error message above usually indicates that the version of the currently running kernel is different from the version of the kernel the given driver was compiled for. Compare the output of 'uname -r' of the running kernel with the output of modinfo /path/to/module.ko | grep vermagic. If they happen to differ, rebuild the given modules against the sources of the running kernel.
error reported in the dmesg:
[ 9.589647] ib_ipoib: no symbol version for module_layout [ 9.597600] ib_srp: no symbol version for module_layout
[root@bz094f ~]# modinfo ib_ipoib filename: /lib/modules/3.10.0-957.el7.s390x/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko license: Dual BSD/GPL description: IP-over-InfiniBand net driver author: Roland Dreier alias: rtnl-link-ipoib rhelversion: 7.6 srcversion: 917AA4365B3509F0B347217 depends: ib_core,ib_cm intree: Y vermagic: 3.10.0 SMP mod_unload modversions parm: max_nonsrq_conn_qp:Max number of connected-mode QPs per interface (applied only if shared receive queue is not available) (int) parm: cm_data_debug_level:Enable data path debug tracing for connected mode if > 0 (int) parm: mcast_debug_level:Enable multicast debug tracing if > 0 (int) parm: data_debug_level:Enable data path debug tracing if > 0 (int) parm: send_queue_size:Number of descriptors in send queue (int) parm: recv_queue_size:Number of descriptors in receive queue (int) parm: ipoib_enhanced:Enable IPoIB enhanced for capable devices (default = 1) (0-1) (int) parm: debug_level:Enable debug tracing if > 0 (int) [root@bz094f ~]#
I had also tried the "make modules_install" command, but get the error below.
[root@bz094f ~]# su - zbrand Last login: Wed Dec 9 16:38:08 CST 2020 on pts/1 [zbrand@bz094f ~]$ cd /home/zbrand/rpmbuild/BUILD/kernel-3.10.0-957.el7/linux-3.10.0-957.el7.s390x [zbrand@bz094f linux-3.10.0-957.el7.s390x]$ [zbrand@bz094f linux-3.10.0-957.el7.s390x]$ sudo make modules_install INSTALL arch/s390/appldata/appldata_mem.ko Can't read private key make[1]: *** [arch/s390/appldata/appldata_mem.ko] Error 2 make: *** [_modinst_] Error 2 [zbrand@bz094f linux-3.10.0-957.el7.s390x]$
So, did anybody have idea on this?
Many Thanks!
-- Regards FuLong Wang _______________________________________________
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies