Can't understand /proc/interrupts output for GICv3 case
Hello all I think I'm asking too many question recently but sometimes I get very big help from here so please forgive me. (and I think someone else can get help from this email thread) I'm doing experiment for interrupt of a device called axpu on an arm64 virtual machine. In QEMU virtual machine, the interrupts are assigned as below. (These numbers are SPI(shared peripheral interrupt) numbers, and there are 16 SGI and 16 PPI before SPI, so we have to add 32 to get the INTID value(hwirq number)) static const int a15irqmap[] = { [AB21Q_UART] = 1, [AB21Q_RTC] = 2, [AB21Q_PCIE] = 3, /* ... to 6 */ [AB21Q_GPIO] = 7, [AB21Q_SECURE_UART] = 8, [AB21Q_ACPI_GED] = 9, [AB21Q_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [AB21Q_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ [AB21Q_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */ [AB21Q_PLATFORM_BUS] = 112, /* ...to 112 + PLATFORM_BUS_NUM_IRQS -1 */ [AB21Q_AXPU] = 15, }; In the driver, when I do request_irq(32+15, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev) for my device axpu, the /proc/interrupts' looks like this. See the 10th item(axpu_irq). CPU0 1: 0 GICv3 25 Level vgic 3: 0 GICv3 30 Level kvm guest ptimer 4: 0 GICv3 27 Level kvm guest vtimer 5: 20432 GICv3 26 Level arch_timer 40: 0 GICv3 106 Edge arm-smmu-v3-evtq 43: 0 GICv3 109 Edge arm-smmu-v3-gerror 44: 0 GICv3 34 Level rtc-pl031 45: 164 GICv3 33 Level uart-pl011 46: 0 GICv3 23 Level arm-pmu 47: 7789 GICv3 36 Level virtio0, axpu_irq 48: 0 9030000.pl061 3 Edge GPIO Key Poweroff IPI0: 0 Rescheduling interrupts IPI1: 0 Function call interrupts IPI2: 0 CPU stop interrupts IPI3: 0 CPU stop (for crash dump) interrupts IPI4: 0 Timer broadcast interrupts IPI5: 0 IRQ work interrupts IPI6: 0 CPU wake-up interrupts Err: 0 There are two numbers. One in the first column and the other in the 4th column. After observing uart-pl011 driver's action(who's irq in 8th row), I found the number in the first column is the one I should pass to the request_irq funcion, and the second number in the 4th column is the input to the GICv3. (uart was assigned SPI 1, so 32+1 = 33). In my axpu driver case, I requested request_irq(47, ...) so hwirq is 47 but I don't know why it was assigned to GICv3 input 36. Shouldn't it be 47 too? (32+15=47). Without fixing QEMU code(still using SPI 15), I tried doing request_irq(15, axpu_irq_handler, ..) for test and this time it looks like this.(see the 5th row) CPU0 1: 0 GICv3 25 Level vgic 3: 0 GICv3 30 Level kvm guest ptimer 4: 0 GICv3 27 Level kvm guest vtimer 5: 8174 GICv3 26 Level arch_timer 15: 0 GICv3 56 Edge axpu_irq 40: 0 GICv3 106 Edge arm-smmu-v3-evtq 43: 0 GICv3 109 Edge arm-smmu-v3-gerror 44: 0 GICv3 34 Level rtc-pl031 45: 175 GICv3 33 Level uart-pl011 46: 0 GICv3 23 Level arm-pmu 47: 7790 GICv3 36 Level virtio0 48: 0 9030000.pl061 3 Edge GPIO Key Poweroff IPI0: 0 Rescheduling interrupts IPI1: 0 Function call interrupts IPI2: 0 CPU stop interrupts IPI3: 0 CPU stop (for crash dump) interrupts IPI4: 0 Timer broadcast interrupts IPI5: 0 IRQ work interrupts IPI6: 0 CPU wake-up interrupts Err: 0 Now axpu_irq has moved to INTID 15 (as expected), but it changed to "GICv3 56 Edge". How is this GICv3 number is assigned? (and I guess edge trigger is the default for INTID < 16. Because it's SGI..?) I thought I connected my device (axpu) to SPI 15 in qemu code but this GICv3 input number changes according to with what number I called reqeust_irq. Could anyone tell me what is going on here? Thank you. Chan Kim
Hi, nobody's replying to my question.. so sad.. I found the kernel makes an array (actually a radix tree) of 'irq_desc's. And my hwirq (SPI 15) is assigned to one of these irq_descs while processing device tree. In my case it was assigned irq 6 (this irq is what is called virtual irq). So I registered the handler by request_irq(6,...) and the interrupt works fine. So now my question is : how can find my irq number (correct irq_desc number) in kernel driver module? I'll appreciate it if you could tell me how to do it. Thank you! Chan Kim
-----Original Message----- From: Chan Kim <ckim@etri.re.kr> Sent: Friday, April 8, 2022 1:53 PM To: 'qemu-discuss' <qemu-discuss@nongnu.org>; kernelnewbies@kernelnewbies.org Subject: Can't understand /proc/interrupts output for GICv3 case
Hello all
I think I'm asking too many question recently but sometimes I get very big help from here so please forgive me. (and I think someone else can get help from this email thread) I'm doing experiment for interrupt of a device called axpu on an arm64 virtual machine. In QEMU virtual machine, the interrupts are assigned as below. (These numbers are SPI(shared peripheral interrupt) numbers, and there are 16 SGI and 16 PPI before SPI, so we have to add 32 to get the INTID value(hwirq number))
static const int a15irqmap[] = { [AB21Q_UART] = 1, [AB21Q_RTC] = 2, [AB21Q_PCIE] = 3, /* ... to 6 */ [AB21Q_GPIO] = 7, [AB21Q_SECURE_UART] = 8, [AB21Q_ACPI_GED] = 9, [AB21Q_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [AB21Q_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ [AB21Q_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */ [AB21Q_PLATFORM_BUS] = 112, /* ...to 112 + PLATFORM_BUS_NUM_IRQS -1 */ [AB21Q_AXPU] = 15, };
In the driver, when I do request_irq(32+15, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev) for my device axpu, the /proc/interrupts' looks like this. See the 10th item(axpu_irq).
CPU0 1: 0 GICv3 25 Level vgic 3: 0 GICv3 30 Level kvm guest ptimer 4: 0 GICv3 27 Level kvm guest vtimer 5: 20432 GICv3 26 Level arch_timer 40: 0 GICv3 106 Edge arm-smmu-v3-evtq 43: 0 GICv3 109 Edge arm-smmu-v3-gerror 44: 0 GICv3 34 Level rtc-pl031 45: 164 GICv3 33 Level uart-pl011 46: 0 GICv3 23 Level arm-pmu 47: 7789 GICv3 36 Level virtio0, axpu_irq 48: 0 9030000.pl061 3 Edge GPIO Key Poweroff IPI0: 0 Rescheduling interrupts IPI1: 0 Function call interrupts IPI2: 0 CPU stop interrupts IPI3: 0 CPU stop (for crash dump) interrupts IPI4: 0 Timer broadcast interrupts IPI5: 0 IRQ work interrupts IPI6: 0 CPU wake-up interrupts Err: 0
There are two numbers. One in the first column and the other in the 4th column. After observing uart-pl011 driver's action(who's irq in 8th row), I found the number in the first column is the one I should pass to the request_irq funcion, and the second number in the 4th column is the input to the GICv3. (uart was assigned SPI 1, so 32+1 = 33). In my axpu driver case, I requested request_irq(47, ...) so hwirq is 47 but I don't know why it was assigned to GICv3 input 36. Shouldn't it be 47 too? (32+15=47).
Without fixing QEMU code(still using SPI 15), I tried doing request_irq(15, axpu_irq_handler, ..) for test and this time it looks like this.(see the 5th row)
CPU0 1: 0 GICv3 25 Level vgic 3: 0 GICv3 30 Level kvm guest ptimer 4: 0 GICv3 27 Level kvm guest vtimer 5: 8174 GICv3 26 Level arch_timer 15: 0 GICv3 56 Edge axpu_irq 40: 0 GICv3 106 Edge arm-smmu-v3-evtq 43: 0 GICv3 109 Edge arm-smmu-v3-gerror 44: 0 GICv3 34 Level rtc-pl031 45: 175 GICv3 33 Level uart-pl011 46: 0 GICv3 23 Level arm-pmu 47: 7790 GICv3 36 Level virtio0 48: 0 9030000.pl061 3 Edge GPIO Key Poweroff IPI0: 0 Rescheduling interrupts IPI1: 0 Function call interrupts IPI2: 0 CPU stop interrupts IPI3: 0 CPU stop (for crash dump) interrupts IPI4: 0 Timer broadcast interrupts IPI5: 0 IRQ work interrupts IPI6: 0 CPU wake-up interrupts Err: 0
Now axpu_irq has moved to INTID 15 (as expected), but it changed to "GICv3 56 Edge". How is this GICv3 number is assigned? (and I guess edge trigger is the default for INTID < 16. Because it's SGI..?)
I thought I connected my device (axpu) to SPI 15 in qemu code but this GICv3 input number changes according to with what number I called reqeust_irq. Could anyone tell me what is going on here?
Thank you. Chan Kim
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Mon, Apr 11, 2022 at 09:36:14PM +0900, Chan Kim wrote:
Hi, nobody's replying to my question.. so sad.. I found the kernel makes an array (actually a radix tree) of 'irq_desc's. And my hwirq (SPI 15) is assigned to one of these irq_descs while processing device tree. In my case it was assigned irq 6 (this irq is what is called virtual irq). So I registered the handler by request_irq(6,...) and the interrupt works fine. So now my question is : how can find my irq number (correct irq_desc number) in kernel driver module?
You ask the system for it. It depends on the bus type your driver is written for for how to do this. For example, if you have a platform driver, you would call platform_get_irq(). If you are a pci driver, the irq for your device is already stored in the pci device structure passed into your probe function. What bus type is your driver written for? hope this helps, greg k-h
Hi, Greg K-H, Thanks for replyaing.
You ask the system for it. It depends on the bus type your driver is written for for how to do this.
For example, if you have a platform driver, you would call platform_get_irq(). If you are a pci driver, the irq for your device is already stored in the pci device structure passed into your probe
function.
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus. This is the qemu code making fdt. static void create_ab21q_axpu_device(const Ab21qMachineState *vms) { char *nodename; printf("create_ab21q_axpu_device called!\n"); hwaddr base = vms->memmap[AB21Q_AXPU].base; hwaddr size = vms->memmap[AB21Q_AXPU].size; int irq = vms->irqmap[AB21Q_AXPU]; const char compat[] = "ab21q-axpu"; sysbus_create_simple("ab21q-axpu", base, qdev_get_gpio_in(vms->gic, irq)); nodename = g_strdup_printf("/ab21q_axpu@%" PRIx64, base); qemu_fdt_add_subnode(vms->fdt, nodename); qemu_fdt_setprop(vms->fdt, nodename, "compatible", compat, sizeof(compat)); qemu_fdt_setprop_sized_cells(vms->fdt, nodename, "reg", 2, base, 2, size); qemu_fdt_setprop_cells(vms->fdt, nodename, "interrupts", GIC_FDT_IRQ_TYPE_SPI, irq, GIC_FDT_IRQ_FLAGS_LEVEL_HI); qemu_fdt_setprop_cell(vms->fdt, nodename, "interrupt-parent", vms->gic_phandle); g_free(nodename); } I understand it creates the device axpu and connects its irq output to a gpio input connected to gic interrupt controller. But this driver is not in the form of platform device/driver. (a char device). What function should I use to get the irq number in this case? I couldn't find a function with a likely name in the tags file for linux source.(5.4.188) Thanks a lot! Chan Kim
On Mon, Apr 11, 2022 at 10:13:42PM +0900, Chan Kim wrote:
Hi, Greg K-H, Thanks for replyaing.
You ask the system for it. It depends on the bus type your driver is written for for how to do this.
For example, if you have a platform driver, you would call platform_get_irq(). If you are a pci driver, the irq for your device is already stored in the pci device structure passed into your probe
function.
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
This is the qemu code making fdt.
static void create_ab21q_axpu_device(const Ab21qMachineState *vms) { char *nodename; printf("create_ab21q_axpu_device called!\n"); hwaddr base = vms->memmap[AB21Q_AXPU].base; hwaddr size = vms->memmap[AB21Q_AXPU].size; int irq = vms->irqmap[AB21Q_AXPU]; const char compat[] = "ab21q-axpu";
sysbus_create_simple("ab21q-axpu", base, qdev_get_gpio_in(vms->gic, irq));
nodename = g_strdup_printf("/ab21q_axpu@%" PRIx64, base); qemu_fdt_add_subnode(vms->fdt, nodename); qemu_fdt_setprop(vms->fdt, nodename, "compatible", compat, sizeof(compat)); qemu_fdt_setprop_sized_cells(vms->fdt, nodename, "reg", 2, base, 2, size); qemu_fdt_setprop_cells(vms->fdt, nodename, "interrupts", GIC_FDT_IRQ_TYPE_SPI, irq, GIC_FDT_IRQ_FLAGS_LEVEL_HI); qemu_fdt_setprop_cell(vms->fdt, nodename, "interrupt-parent", vms->gic_phandle);
g_free(nodename);
This is qemu code, not kernel code.
}
I understand it creates the device axpu and connects its irq output to a gpio input connected to gic interrupt controller.
Again, this is qemu code, not kernel code.
But this driver is not in the form of platform device/driver. (a char device). What function should I use to get the irq number in this case? I couldn't find a function with a likely name in the tags file for linux source.(5.4.188)
Where is your kernel code? confused, greg k-h
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel module form? Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me. static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; } printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev)); /* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE; /* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; } /* creating struct class */ if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; } /* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/ vaddr = ioremap(AXPU_BASE, 0x80000); if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr); ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev); printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0; r_device : class_destroy(dev_class); r_class : unregister_chrdev_region(dev,1); return -1; } Thank you. Chan Kim
On Mon, Apr 11, 2022 at 11:15:57PM +0900, Chan Kim wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel module form? Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me.
static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; }
printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev));
/* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE;
/* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; }
/* creating struct class */ if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; }
/* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/
You can replace all of the above code by just using the miscdevice interface instead. Please use that, it ensures that you do everything properly and simplifies it all.
vaddr = ioremap(AXPU_BASE, 0x80000);
Wait, where are you picking those random values from? That's not how drivers work in the kernel. You have to have a bus pass you a device. Please register your driver with the proper bus type and go from there. The probe callback should be the place you set things up based on the device and resources given to you by the bus. You can't just randomly pick memory locations and think things will work.
if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr);
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev);
Same for that, just picking 6 will not work, sorry. Perhaps take a look at the book, Linux Device Drivers, 3rd edition. It's free online and should help you out a lot.
printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0;
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1;
One final comment, don't make up error values like this, use real ERROR codes. thanks, greg k-h
Hi, Greg KH, Thanks for the advices. Some questions inline.
-----Original Message----- From: 'Greg KH' <greg@kroah.com> Sent: Monday, April 11, 2022 11:26 PM To: Chan Kim <ckim@etri.re.kr> Cc: 'qemu-discuss' <qemu-discuss@nongnu.org>; kernelnewbies@kernelnewbies.org Subject: Re: Can't understand /proc/interrupts output for GICv3 case
On Mon, Apr 11, 2022 at 11:15:57PM +0900, Chan Kim wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel
module form?
Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me.
static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; }
printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev));
/* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE;
/* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; }
/* creating struct class */ if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; }
/* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/
You can replace all of the above code by just using the miscdevice interface instead. Please use that, it ensures that you do everything properly and simplifies it all.
vaddr = ioremap(AXPU_BASE, 0x80000);
Wait, where are you picking those random values from?
Yes, it now looks weird to me. I have passed the register address information in the device tree and the kernel already knows my device's address range. Then, how should I get this virtual io address in this driver? I need it to access some registers. How can I ask the system bus? And my driver is a kernel module because I want to use it in ubuntu-20.04 on a virtual machine, I want it to be a kernel module that I can insmod or rmmod.(actually my job is to provide this virtual machine to some folks). And I cannot build the ubuntu image even if I change it to a platform device driver and add it in the kernel tree.
That's not how drivers work in the kernel. You have to have a bus pass you a device. Please register your driver with the proper bus type and go from there. The probe callback should be the place you set things up based on the device and resources given to you by the bus.
You can't just randomly pick memory locations and think things will work.
if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr);
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev);
Same for that, just picking 6 will not work, sorry.
Yes, that was my original question. How can I get my irq number (I know it's hwirq 47) and I peeked into kernel that irq 6 was assigned for the irq_desc. So I changed my driver to request irq 6 for my device and I found at least it works for now, all the register access and interrupts. I know this is not the solution and I'm curious how I should get the irq number of io virtual address in this situation.
Perhaps take a look at the book, Linux Device Drivers, 3rd edition. It's free online and should help you out a lot.
printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0;
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1;
One final comment, don't make up error values like this, use real ERROR codes.
thanks,
greg k-h
Yes, I've read the book sometimes (not the whole part) but if I read it now, I'll be able to more understand it. It's a bit outdated though. Why don't you update your book? :) Thank you again for the advices. Best regards, Chan Kim
On Tue, Apr 12, 2022 at 11:18:03AM +0900, Chan Kim wrote:
You can replace all of the above code by just using the miscdevice interface instead. Please use that, it ensures that you do everything properly and simplifies it all.
Again, use the misc device api please.
vaddr = ioremap(AXPU_BASE, 0x80000);
Wait, where are you picking those random values from?
Yes, it now looks weird to me. I have passed the register address information in the device tree and the kernel already knows my device's address range. Then, how should I get this virtual io address in this driver? I need it to access some registers. How can I ask the system bus?
Use a platform driver and bind your driver to that device based on that api.
And my driver is a kernel module because I want to use it in ubuntu-20.04 on a virtual machine, I want it to be a kernel module that I can insmod or rmmod.(actually my job is to provide this virtual machine to some folks). And I cannot build the ubuntu image even if I change it to a platform device driver and add it in the kernel tree.
This all does not matter, just write a proper platform driver and all will be fine.
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev);
Same for that, just picking 6 will not work, sorry.
Yes, that was my original question. How can I get my irq number (I know it's hwirq 47) and I peeked into kernel that irq 6 was assigned for the irq_desc. So I changed my driver to request irq 6 for my device and I found at least it works for now, all the register access and interrupts. I know this is not the solution and I'm curious how I should get the irq number of io virtual address in this situation.
Again, the platform driver interface will provide you with the needed information. We have thousands of working examples in the kernel tree.
Perhaps take a look at the book, Linux Device Drivers, 3rd edition. It's free online and should help you out a lot.
printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0;
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1;
One final comment, don't make up error values like this, use real ERROR codes.
thanks,
greg k-h
Yes, I've read the book sometimes (not the whole part) but if I read it now, I'll be able to more understand it. It's a bit outdated though. Why don't you update your book? :)
Because the publisher does not want to publish a new version. thanks, greg k-h
Hi Greg KH, I see, I understand what the proper method should be for this. I'll take it as an almost official answer from the linux people :) Thank you. Chan Kim
-----Original Message----- From: 'Greg KH' <greg@kroah.com> Sent: Tuesday, April 12, 2022 1:50 PM To: Chan Kim <ckim@etri.re.kr> Cc: 'qemu-discuss' <qemu-discuss@nongnu.org>; kernelnewbies@kernelnewbies.org Subject: Re: Can't understand /proc/interrupts output for GICv3 case
On Tue, Apr 12, 2022 at 11:18:03AM +0900, Chan Kim wrote:
You can replace all of the above code by just using the miscdevice interface instead. Please use that, it ensures that you do everything properly and simplifies it all.
Again, use the misc device api please.
vaddr = ioremap(AXPU_BASE, 0x80000);
Wait, where are you picking those random values from?
Yes, it now looks weird to me. I have passed the register address information in the device tree and the kernel already knows my device's address range. Then, how should I get this virtual io address in this driver? I need it to access some registers. How can I ask the system bus?
Use a platform driver and bind your driver to that device based on that api.
And my driver is a kernel module because I want to use it in ubuntu-20.04 on a virtual machine, I want it to be a kernel module that I can insmod or rmmod.(actually my job is to provide this virtual machine to some folks). And I cannot build the ubuntu image even if I change it to a platform device driver and add it in the kernel tree.
This all does not matter, just write a proper platform driver and all will be fine.
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev);
Same for that, just picking 6 will not work, sorry.
Yes, that was my original question. How can I get my irq number (I know it's hwirq 47) and I peeked into kernel that irq 6 was assigned for the irq_desc. So I changed my driver to request irq 6 for my device and I found at least it works for now, all the register access and interrupts. I know this is not the solution and I'm curious how I should get the irq number of io virtual address in this situation.
Again, the platform driver interface will provide you with the needed information. We have thousands of working examples in the kernel tree.
Perhaps take a look at the book, Linux Device Drivers, 3rd edition. It's free online and should help you out a lot.
printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done
properly..\n");
return 0;
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1;
One final comment, don't make up error values like this, use real ERROR codes.
thanks,
greg k-h
Yes, I've read the book sometimes (not the whole part) but if I read it now, I'll be able to more understand it. It's a bit outdated though. Why don't you update your book? :)
Because the publisher does not want to publish a new version.
thanks,
greg k-h
Hi Greg KH and all, I found how to find the irq number in my case! (char driver kernel module) I want to share it for reference to others. add these two header files for this. #include <linux/irq.h> #include <linux/irqdesc.h> extern struct irq_desc *irq_to_desc(unsigned int irq); struct irq_desc *desc; in the module _init function : // find my irq number for(i=0;i<NR_IRQS;i++){ desc = irq_to_desc(i); if (desc) { //printk("irq_desc(%d)->irq_data.hwirq = %ld\n", i, desc->irq_data.hwirq); if (desc->irq_data.hwirq == 47) break; // 47 is the hwirq number, (SPI 15) } } if (i == NR_IRQS) { printk("couldn't find irq number..\n"); goto r_device; } ret = request_irq(i, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev); This way I could find the correct irq number on ubuntu and vanilla linux. Hope this is helpful to someone. (anyway you know the hardware connection so you can use it) Thank you! Chan Kim
-----Original Message----- From: Chan Kim <ckim@etri.re.kr> Sent: Tuesday, April 12, 2022 1:55 PM To: 'Greg KH' <greg@kroah.com> Cc: kernelnewbies@kernelnewbies.org; 'qemu-discuss' <qemu- discuss@nongnu.org> Subject: RE: Can't understand /proc/interrupts output for GICv3 case
Hi Greg KH, I see, I understand what the proper method should be for this. I'll take it as an almost official answer from the linux people :) Thank you. Chan Kim
-----Original Message----- From: 'Greg KH' <greg@kroah.com> Sent: Tuesday, April 12, 2022 1:50 PM To: Chan Kim <ckim@etri.re.kr> Cc: 'qemu-discuss' <qemu-discuss@nongnu.org>; kernelnewbies@kernelnewbies.org Subject: Re: Can't understand /proc/interrupts output for GICv3 case
On Tue, Apr 12, 2022 at 11:18:03AM +0900, Chan Kim wrote:
You can replace all of the above code by just using the miscdevice interface instead. Please use that, it ensures that you do everything properly and simplifies it all.
Again, use the misc device api please.
vaddr = ioremap(AXPU_BASE, 0x80000);
Wait, where are you picking those random values from?
Yes, it now looks weird to me. I have passed the register address information in the device tree and the kernel already knows my device's address range. Then, how should I get this virtual io address in this driver? I need it to access some registers. How can I ask the system bus?
Use a platform driver and bind your driver to that device based on that api.
And my driver is a kernel module because I want to use it in ubuntu-20.04 on a virtual machine, I want it to be a kernel module that I can insmod or rmmod.(actually my job is to provide this virtual machine to some folks). And I cannot build the ubuntu image even if I change it to a platform device driver and add it in the kernel tree.
This all does not matter, just write a proper platform driver and all will be fine.
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev);
Same for that, just picking 6 will not work, sorry.
Yes, that was my original question. How can I get my irq number (I know it's hwirq 47) and I peeked into kernel that irq 6 was assigned for the irq_desc. So I changed my driver to request irq 6 for my device and I found at least it works for now, all the register access and interrupts. I know this is not the solution and I'm curious how I should get the irq number of io virtual address in this situation.
Again, the platform driver interface will provide you with the needed information. We have thousands of working examples in the kernel tree.
Perhaps take a look at the book, Linux Device Drivers, 3rd edition. It's free online and should help you out a lot.
printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done
properly..\n");
return 0;
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1;
One final comment, don't make up error values like this, use real ERROR codes.
thanks,
greg k-h
Yes, I've read the book sometimes (not the whole part) but if I read it now, I'll be able to more understand it. It's a bit outdated though. Why don't you update your book? :)
Because the publisher does not want to publish a new version.
thanks,
greg k-h
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Tue, Apr 12, 2022 at 03:51:24PM +0900, Chan Kim wrote:
Hi Greg KH and all,
I found how to find the irq number in my case! (char driver kernel module) I want to share it for reference to others.
add these two header files for this. #include <linux/irq.h> #include <linux/irqdesc.h>
extern struct irq_desc *irq_to_desc(unsigned int irq); struct irq_desc *desc;
in the module _init function :
// find my irq number for(i=0;i<NR_IRQS;i++){ desc = irq_to_desc(i); if (desc) { //printk("irq_desc(%d)->irq_data.hwirq = %ld\n", i, desc->irq_data.hwirq); if (desc->irq_data.hwirq == 47) break; // 47 is the hwirq number, (SPI 15) } }
No, please no. Do NOT do this, it will break and not work well over time, if it even works at all right now you are lucky. Odds are, when you reboot, it will not as irq numbers are never guaranteed to be the same across boots. Again, use the irq number given to you by your platform device bus. It should be much simpler than "iterate over all irqs and guess which one we want to take" like you did here. good luck! greg k-h
On Mon, 11 Apr 2022 at 15:22, Chan Kim <ckim@etri.re.kr> wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
The 'sysbus' is just QEMU's abstraction for "devices mapped into memory at a fixed physical address", ie simple MMIO devices that aren't on a complex bus like PCI or USB or SPI. -- PMM
On Mon, Apr 11, 2022 at 03:32:23PM +0100, Peter Maydell wrote:
On Mon, 11 Apr 2022 at 15:22, Chan Kim <ckim@etri.re.kr> wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
The 'sysbus' is just QEMU's abstraction for "devices mapped into memory at a fixed physical address", ie simple MMIO devices that aren't on a complex bus like PCI or USB or SPI.
So, a platform_device in Linux kernel terms, right?
On Mon, 11 Apr 2022 at 15:49, Greg KH <greg@kroah.com> wrote:
On Mon, Apr 11, 2022 at 03:32:23PM +0100, Peter Maydell wrote:
On Mon, 11 Apr 2022 at 15:22, Chan Kim <ckim@etri.re.kr> wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
The 'sysbus' is just QEMU's abstraction for "devices mapped into memory at a fixed physical address", ie simple MMIO devices that aren't on a complex bus like PCI or USB or SPI.
So, a platform_device in Linux kernel terms, right?
Don't ask me, I don't know how the kernel's handling for that kind of device works :-) -- PMM
On Mon, Apr 11, 2022 at 6:49 PM Greg KH <greg@kroah.com> wrote:
On Mon, Apr 11, 2022 at 03:32:23PM +0100, Peter Maydell wrote:
On Mon, 11 Apr 2022 at 15:22, Chan Kim <ckim@etri.re.kr> wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
The 'sysbus' is just QEMU's abstraction for "devices mapped into memory at a fixed physical address", ie simple MMIO devices that aren't on a complex bus like PCI or USB or SPI.
So, a platform_device in Linux kernel terms, right?
Hello, yes thats correct because it uses platform_device.h but *dev in platform_device is not called this code, already defined in a device struct header file. this *dev and *resource will be able to return for resource IRQ and interrupt.
Re-hi, On Mon, Apr 11, 2022 at 6:16 PM Chan Kim <ckim@etri.re.kr> wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel module form? Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me.
static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; }
printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev));
/* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE;
/* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; }
I guess you got address 0x80000 randomly also may have this code from stackoverflow but it wont work for you. as written stackoverflow, device_create must be a character device and an address under /sys/dev/char before getting NULL. So did you create a char device with use mknod command? Actually register_chrdev will do this for you but you can do it with mknod if you wish. /* creating struct class */
if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; }
/* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/
vaddr = ioremap(AXPU_BASE, 0x80000); \
Please first read Documentation/devices.txt because kernel can do dynamic allocation I think. * https://www.oreilly.com/library/view/linux-device-drivers/0596000081/ch03s02... * https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch03.ht...
if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr);
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev); printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0;
You dont need to call manual, because it is defined in cdev.h. * https://embeddedguruji.blogspot.com/2019/01/automatically-creating-device-no... in example for example device_destroy already destroys class.
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1; }
it will be great that i will recommend three books to you again. * Understanding the Linux Kernel * Linux Device Drivers * Linux Kernel Development
Thank you. Chan Kim
Hi Ozgur, My replies in-line. Thanks! From: Ozgur Karatas <ozgurk@ieee.org> Sent: Monday, April 11, 2022 11:53 PM To: Chan Kim <ckim@etri.re.kr> Cc: Greg KH <greg@kroah.com>; kernelnewbies@kernelnewbies.org; qemu-discuss <qemu-discuss@nongnu.org> Subject: Re: Can't understand /proc/interrupts output for GICv3 case Re-hi, On Mon, Apr 11, 2022 at 6:16 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel module form? Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me. static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; } printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev)); /* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE; /* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; } I guess you got address 0x80000 randomly also may have this code from stackoverflow but it wont work for you. as written stackoverflow, device_create must be a character device and an address under /sys/dev/char before getting NULL. So did you create a char device with use mknod command? ==> (I don’t know how to remove this bar in the lefe when I add my in-line reply in outlook..) it’s the size of address range of my device. My question is, how I can get the io virtual address when 1. my driver is a kernel module and 2. It is a character driver. For platform driver I got it from the resource table, but I think it’s an old method now and we should ask the bus as Greg-KH said. The kernel already knows the addresses because it had processed the device table. Actually register_chrdev will do this for you but you can do it with mknod if you wish. /* creating struct class */ if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; } /* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/ vaddr = ioremap(AXPU_BASE, 0x80000); \ Please first read Documentation/devices.txt because kernel can do dynamic allocation I think. * https://www.oreilly.com/library/view/linux-device-drivers/0596000081/ch03s02... * https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch03.ht... ==> I understand by using class_create and device_create, the device file is generated under /dev (without using mknod). I am using this method without problem and the app/driver runs ok on the virtual machine. if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr); ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev); printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0; You dont need to call manual, because it is defined in cdev.h. * https://embeddedguruji.blogspot.com/2019/01/automatically-creating-device-no... ==> your link doesn’t show do any io access or irq processing. I want to know how I can get the io virtual address and the virtual irq number from the driver. My driver is a character device driver and it’s a kernel module which is inserted after OS has booted. I thinks this should be possible. in example for example device_destroy already destroys class. r_device : class_destroy(dev_class); r_class : unregister_chrdev_region(dev,1); return -1; } it will be great that i will recommend three books to you again. * Understanding the Linux Kernel * Linux Device Drivers * Linux Kernel Development ==> Yes I’ve read those books here and there in the past, but you know I’m not given that much time for this and my job isn’t always working with linux drivers. I will take some time to read those books thoroughly sometime later. Thanks for the interest and advices. Thank you. Chan Kim
On Tue, Apr 12, 2022 at 6:44 AM Chan Kim <ckim@etri.re.kr> wrote:
Hi Ozgur,
My replies in-line.
Thanks!
Hello Chan, today, web-based online content is gaining more importance than books especially in kernel.org you can browse serious content about IRQ and see examples. https://www.kernel.org/doc/html/v4.12/core-api/genericirq.html https://www.kernel.org/doc/html/ You can search online. And you received as "6" in your last e-mail was actually a old floopy address and your code would fail if you were using a floopy. As Greg stated, if you use a domain driver over a kernel api, kernel would handle it in the get interrupt part. Regards Ozgur
*From:* Ozgur Karatas <ozgurk@ieee.org> *Sent:* Monday, April 11, 2022 11:53 PM *To:* Chan Kim <ckim@etri.re.kr> *Cc:* Greg KH <greg@kroah.com>; kernelnewbies@kernelnewbies.org; qemu-discuss <qemu-discuss@nongnu.org> *Subject:* Re: Can't understand /proc/interrupts output for GICv3 case
Re-hi,
On Mon, Apr 11, 2022 at 6:16 PM Chan Kim <ckim@etri.re.kr> wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel module form? Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me.
static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; }
printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev));
/* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE;
/* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; }
I guess you got address 0x80000 randomly also may have this code from stackoverflow but it wont work for you.
as written stackoverflow, device_create must be a character device and an address under /sys/dev/char before getting NULL.
So did you create a char device with use mknod command?
==> (I don’t know how to remove this bar in the lefe when I add my in-line reply in outlook..) it’s the size of address range of my device. My question is, how I can get the io virtual address when 1. my driver is a kernel module and 2. It is a character driver. For platform driver I got it from the resource table, but I think it’s an old method now and we should ask the bus as Greg-KH said. The kernel already knows the addresses because it had processed the device table.
Actually register_chrdev will do this for you but you can do it with mknod if you wish.
/* creating struct class */ if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; }
/* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/
vaddr = ioremap(AXPU_BASE, 0x80000); \
Please first read Documentation/devices.txt because kernel can do dynamic allocation I think.
* https://www.oreilly.com/library/view/linux-device-drivers/0596000081/ch03s02...
* https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch03.ht...
==> I understand by using class_create and device_create, the device file is generated under /dev (without using mknod). I am using this method without problem and the app/driver runs ok on the virtual machine.
if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr);
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev); printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0;
You dont need to call manual, because it is defined in cdev.h.
* https://embeddedguruji.blogspot.com/2019/01/automatically-creating-device-no...
==> your link doesn’t show do any io access or irq processing. I want to know how I can get the io virtual address and the virtual irq number from the driver. My driver is a character device driver and it’s a kernel module which is inserted after OS has booted. I thinks this should be possible.
in example for example device_destroy already destroys class.
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1; }
it will be great that i will recommend three books to you again.
* Understanding the Linux Kernel * Linux Device Drivers
* Linux Kernel Development
==> Yes I’ve read those books here and there in the past, but you know I’m not given that much time for this and my job isn’t always working with linux drivers. I will take some time to read those books thoroughly sometime later. Thanks for the interest and advices.
Thank you. Chan Kim
Hi Ozgur, When I run it on ubuntu-20.04, the irq number returned was 5. (in 5.4.188 vanila, it was 6) but this irq number will depend on the hardware configuration. And if I don’t pass IRQF_SHARED, the kernel would have assigned a dedicated irq_desc so it will not be a problem. And about irq 6 for floppy, isn’t it for intel specific? This irq number is virtual so I think kernel will assign the number as shared or dedicated I think. My app/driver is just for test so that other members can do driver test on the virtual machine so for now it suits my need. Thank you! (and I’ll be reading the references again soon) Chan From: Ozgur Karatas <ozgurk@ieee.org> Sent: Tuesday, April 12, 2022 4:15 PM To: Chan Kim <ckim@etri.re.kr> Cc: Greg KH <greg@kroah.com>; kernelnewbies@kernelnewbies.org; qemu-discuss <qemu-discuss@nongnu.org> Subject: Re: Can't understand /proc/interrupts output for GICv3 case On Tue, Apr 12, 2022 at 6:44 AM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote: Hi Ozgur, My replies in-line. Thanks! Hello Chan, today, web-based online content is gaining more importance than books especially in kernel.org <http://kernel.org> you can browse serious content about IRQ and see examples. https://www.kernel.org/doc/html/v4.12/core-api/genericirq.html https://www.kernel.org/doc/html/ You can search online. And you received as "6" in your last e-mail was actually a old floopy address and your code would fail if you were using a floopy. As Greg stated, if you use a domain driver over a kernel api, kernel would handle it in the get interrupt part. Regards Ozgur From: Ozgur Karatas <ozgurk@ieee.org <mailto:ozgurk@ieee.org> > Sent: Monday, April 11, 2022 11:53 PM To: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > Cc: Greg KH <greg@kroah.com <mailto:greg@kroah.com> >; kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> ; qemu-discuss <qemu-discuss@nongnu.org <mailto:qemu-discuss@nongnu.org> > Subject: Re: Can't understand /proc/interrupts output for GICv3 case Re-hi, On Mon, Apr 11, 2022 at 6:16 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel module form? Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me. static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; } printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev)); /* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE; /* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; } I guess you got address 0x80000 randomly also may have this code from stackoverflow but it wont work for you. as written stackoverflow, device_create must be a character device and an address under /sys/dev/char before getting NULL. So did you create a char device with use mknod command? ==> (I don’t know how to remove this bar in the lefe when I add my in-line reply in outlook..) it’s the size of address range of my device. My question is, how I can get the io virtual address when 1. my driver is a kernel module and 2. It is a character driver. For platform driver I got it from the resource table, but I think it’s an old method now and we should ask the bus as Greg-KH said. The kernel already knows the addresses because it had processed the device table. Actually register_chrdev will do this for you but you can do it with mknod if you wish. /* creating struct class */ if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; } /* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/ vaddr = ioremap(AXPU_BASE, 0x80000); \ Please first read Documentation/devices.txt because kernel can do dynamic allocation I think. * https://www.oreilly.com/library/view/linux-device-drivers/0596000081/ch03s02... * https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch03.ht... ==> I understand by using class_create and device_create, the device file is generated under /dev (without using mknod). I am using this method without problem and the app/driver runs ok on the virtual machine. if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr); ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev); printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0; You dont need to call manual, because it is defined in cdev.h. * https://embeddedguruji.blogspot.com/2019/01/automatically-creating-device-no... ==> your link doesn’t show do any io access or irq processing. I want to know how I can get the io virtual address and the virtual irq number from the driver. My driver is a character device driver and it’s a kernel module which is inserted after OS has booted. I thinks this should be possible. in example for example device_destroy already destroys class. r_device : class_destroy(dev_class); r_class : unregister_chrdev_region(dev,1); return -1; } it will be great that i will recommend three books to you again. * Understanding the Linux Kernel * Linux Device Drivers * Linux Kernel Development ==> Yes I’ve read those books here and there in the past, but you know I’m not given that much time for this and my job isn’t always working with linux drivers. I will take some time to read those books thoroughly sometime later. Thanks for the interest and advices. Thank you. Chan Kim
On Tue, Apr 12, 2022 at 11:22 AM Chan Kim <ckim@etri.re.kr> wrote:
Hi Ozgur,
When I run it on ubuntu-20.04, the irq number returned was 5. (in 5.4.188 vanila, it was 6) but this irq number will depend on the hardware configuration.
And if I don’t pass IRQF_SHARED, the kernel would have assigned a dedicated irq_desc so it will not be a problem.
And about irq 6 for floppy, isn’t it for intel specific? This irq number is virtual so I think kernel will assign the number as shared or dedicated I think.
My app/driver is just for test so that other members can do driver test on the virtual machine so for now it suits my need.
Thank you! (and I’ll be reading the references again soon)
Hello Chan, I'm not experienced in this matter so information i would give would be wrong. However, if you still want to drive a driver, is best to work with kernel modules. Indeed, if you look at how linux kernel works on Interrupt Handler as an example, i think you will find a solution much easier with a use driver. Regards Ozgur Chan
*From:* Ozgur Karatas <ozgurk@ieee.org> *Sent:* Tuesday, April 12, 2022 4:15 PM *To:* Chan Kim <ckim@etri.re.kr> *Cc:* Greg KH <greg@kroah.com>; kernelnewbies@kernelnewbies.org; qemu-discuss <qemu-discuss@nongnu.org> *Subject:* Re: Can't understand /proc/interrupts output for GICv3 case
On Tue, Apr 12, 2022 at 6:44 AM Chan Kim <ckim@etri.re.kr> wrote:
Hi Ozgur,
My replies in-line.
Thanks!
Hello Chan,
today, web-based online content is gaining more importance than books especially in kernel.org you can browse serious content about IRQ and see examples.
https://www.kernel.org/doc/html/v4.12/core-api/genericirq.html
https://www.kernel.org/doc/html/
You can search online.
And you received as "6" in your last e-mail was actually a old floopy address and your code would fail if you were using a floopy.
As Greg stated, if you use a domain driver over a kernel api, kernel would handle it in the get interrupt part.
Regards
Ozgur
*From:* Ozgur Karatas <ozgurk@ieee.org> *Sent:* Monday, April 11, 2022 11:53 PM *To:* Chan Kim <ckim@etri.re.kr> *Cc:* Greg KH <greg@kroah.com>; kernelnewbies@kernelnewbies.org; qemu-discuss <qemu-discuss@nongnu.org> *Subject:* Re: Can't understand /proc/interrupts output for GICv3 case
Re-hi,
On Mon, Apr 11, 2022 at 6:16 PM Chan Kim <ckim@etri.re.kr> wrote:
What bus type is your driver written for?
That sounds very logical. In my case I added it to system bus.
What exactly do you mean by "system bus"?
I meant 'sysbus' in qemu code that I showed in the qemu code. And I think it's the CPU bus.
Where is your kernel code?
This is the init function of my char driver. I thought if the struct cdev contains struct device, maybe I could use the struct device's of_node to call of_irq_get but it doesn't. And I remember I've seen the cdev in usually contained in the driver data of platform driver(?). Can I implement platform driver in kernel module form? Below is the char driver init code. Currently it's request_irq(6, ... ) but I want to know out the number 6 using program. If you have any advice, please tell me.
static int __init chr_driver_init(void) { int ret; /* Allocating Major number */ if ((alloc_chrdev_region(&dev, 0, 1, "axpu_Dev")) < 0) { printk(KERN_INFO"Cannot allocate the major number..\n"); return -1; }
printk(KERN_INFO"Major = %d Minor = %d..\n",MAJOR(dev),MINOR(dev));
/* creating cdev structure */ cdev_init(&axpu_cdev, &fops); axpu_cdev.owner = THIS_MODULE;
/* Adding character device to the system */ if ((cdev_add(&axpu_cdev,dev,1)) < 0) { printk(KERN_INFO "Cannot add the device to the system...\n"); goto r_class; }
I guess you got address 0x80000 randomly also may have this code from stackoverflow but it wont work for you.
as written stackoverflow, device_create must be a character device and an address under /sys/dev/char before getting NULL.
So did you create a char device with use mknod command?
==> (I don’t know how to remove this bar in the lefe when I add my in-line reply in outlook..) it’s the size of address range of my device. My question is, how I can get the io virtual address when 1. my driver is a kernel module and 2. It is a character driver. For platform driver I got it from the resource table, but I think it’s an old method now and we should ask the bus as Greg-KH said. The kernel already knows the addresses because it had processed the device table.
Actually register_chrdev will do this for you but you can do it with mknod if you wish.
/* creating struct class */ if ((dev_class = class_create(THIS_MODULE, "axpu_class")) == NULL) { printk(KERN_INFO "cannot create the struct class...\n"); goto r_class; }
/* for interrupt test !! */ /* for vanilla work-around.. already made by mkdev */ if ((device_create(dev_class, NULL, dev, NULL, "axpu_device")) == NULL) { printk(KERN_INFO "cannot create the device ..\n"); goto r_device; } else { printk(KERN_INFO "axpu_device created..\n"); } /**/
vaddr = ioremap(AXPU_BASE, 0x80000); \
Please first read Documentation/devices.txt because kernel can do dynamic allocation I think.
* https://www.oreilly.com/library/view/linux-device-drivers/0596000081/ch03s02...
* https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch03.ht...
==> I understand by using class_create and device_create, the device file is generated under /dev (without using mknod). I am using this method without problem and the app/driver runs ok on the virtual machine.
if(!vaddr) { printk(KERN_INFO"Failed to map the address.\n"); release_mem_region(AXPU_BASE,AXPU_SIZE); return 1; } printk("----- AXPU_BASE mapped at vaddr = %px\n", vaddr);
ret = request_irq(6, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev); printk("request_irq returned %d\n", ret); // -EINVAL printk(KERN_INFO "Device driver inserted ..done properly..\n"); return 0;
You dont need to call manual, because it is defined in cdev.h.
* https://embeddedguruji.blogspot.com/2019/01/automatically-creating-device-no...
==> your link doesn’t show do any io access or irq processing. I want to know how I can get the io virtual address and the virtual irq number from the driver. My driver is a character device driver and it’s a kernel module which is inserted after OS has booted. I thinks this should be possible.
in example for example device_destroy already destroys class.
r_device : class_destroy(dev_class);
r_class : unregister_chrdev_region(dev,1); return -1; }
it will be great that i will recommend three books to you again.
* Understanding the Linux Kernel * Linux Device Drivers
* Linux Kernel Development
==> Yes I’ve read those books here and there in the past, but you know I’m not given that much time for this and my job isn’t always working with linux drivers. I will take some time to read those books thoroughly sometime later. Thanks for the interest and advices.
Thank you. Chan Kim
Hi Ozgur, The SPI I meant was ‘shared peripheral interrupt’ of armv8 architecture. (the interrupt is shared among cores. The interrupt controller distributes it) I’ll look into your link when I’ll be working with SPI (serial peripheral interface) later :). Thank you. Chan From: Ozgur Kara <ozgur@goosey.org> Sent: Monday, April 11, 2022 10:10 PM To: Chan Kim <ckim@etri.re.kr>; 'qemu-discuss' <qemu-discuss@nongnu.org>; kernelnewbies@kernelnewbies.org Subject: Re: Can't understand /proc/interrupts output for GICv3 case Hello, firstly when you say SPI so can you give some more information about your hardware? I think you are doing an embedded development in that case you should do some research on howto understand SPI interface under the kernel. I think for example, source code of a previously written hardware using SPI bus can give you an idea. You can look at examples for example spi-based Xilinx however I think all SPI hardware is declared under /linux/drivers/spi with "spi.c". Also we see a "spi.h" header file depending on kernel headers. So, if you define a master mode (MOSI) then "interrupt.h" header file will be useful because must first determine free irq and these calls are defined in interrupt.h. Please attention some kernel call: - request_irq - irqsave() - irqrestore() and maybe check out the "Interrupt Handling" book and please read follow website: https://www.kernel.org/doc/html/v4.11/driver-api/spi.html * Book (Understand the Linux Kernel) - Section: Interrupt Handling Salut! 11.04.2022, 16:36, "Chan Kim" <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >: Hi, nobody's replying to my question.. so sad.. I found the kernel makes an array (actually a radix tree) of 'irq_desc's. And my hwirq (SPI 15) is assigned to one of these irq_descs while processing device tree. In my case it was assigned irq 6 (this irq is what is called virtual irq). So I registered the handler by request_irq(6,...) and the interrupt works fine. So now my question is : how can find my irq number (correct irq_desc number) in kernel driver module? I'll appreciate it if you could tell me how to do it. Thank you! Chan Kim -----Original Message----- From: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > Sent: Friday, April 8, 2022 1:53 PM To: 'qemu-discuss' <qemu-discuss@nongnu.org <mailto:qemu-discuss@nongnu.org> >; kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> Subject: Can't understand /proc/interrupts output for GICv3 case Hello all I think I'm asking too many question recently but sometimes I get very big help from here so please forgive me. (and I think someone else can get help from this email thread) I'm doing experiment for interrupt of a device called axpu on an arm64 virtual machine. In QEMU virtual machine, the interrupts are assigned as below. (These numbers are SPI(shared peripheral interrupt) numbers, and there are 16 SGI and 16 PPI before SPI, so we have to add 32 to get the INTID value(hwirq number)) static const int a15irqmap[] = { [AB21Q_UART] = 1, [AB21Q_RTC] = 2, [AB21Q_PCIE] = 3, /* ... to 6 */ [AB21Q_GPIO] = 7, [AB21Q_SECURE_UART] = 8, [AB21Q_ACPI_GED] = 9, [AB21Q_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [AB21Q_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ [AB21Q_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */ [AB21Q_PLATFORM_BUS] = 112, /* ...to 112 + PLATFORM_BUS_NUM_IRQS -1 */ [AB21Q_AXPU] = 15, }; In the driver, when I do request_irq(32+15, axpu_irq_handler, IRQF_SHARED, "axpu_irq", &axpu_cdev) for my device axpu, the /proc/interrupts' looks like this. See the 10th item(axpu_irq). CPU0 1: 0 GICv3 25 Level vgic 3: 0 GICv3 30 Level kvm guest ptimer 4: 0 GICv3 27 Level kvm guest vtimer 5: 20432 GICv3 26 Level arch_timer 40: 0 GICv3 106 Edge arm-smmu-v3-evtq 43: 0 GICv3 109 Edge arm-smmu-v3-gerror 44: 0 GICv3 34 Level rtc-pl031 45: 164 GICv3 33 Level uart-pl011 46: 0 GICv3 23 Level arm-pmu 47: 7789 GICv3 36 Level virtio0, axpu_irq 48: 0 9030000.pl061 3 Edge GPIO Key Poweroff IPI0: 0 Rescheduling interrupts IPI1: 0 Function call interrupts IPI2: 0 CPU stop interrupts IPI3: 0 CPU stop (for crash dump) interrupts IPI4: 0 Timer broadcast interrupts IPI5: 0 IRQ work interrupts IPI6: 0 CPU wake-up interrupts Err: 0 There are two numbers. One in the first column and the other in the 4th column. After observing uart-pl011 driver's action(who's irq in 8th row), I found the number in the first column is the one I should pass to the request_irq funcion, and the second number in the 4th column is the input to the GICv3. (uart was assigned SPI 1, so 32+1 = 33). In my axpu driver case, I requested request_irq(47, ...) so hwirq is 47 but I don't know why it was assigned to GICv3 input 36. Shouldn't it be 47 too? (32+15=47). Without fixing QEMU code(still using SPI 15), I tried doing request_irq(15, axpu_irq_handler, ..) for test and this time it looks like this.(see the 5th row) CPU0 1: 0 GICv3 25 Level vgic 3: 0 GICv3 30 Level kvm guest ptimer 4: 0 GICv3 27 Level kvm guest vtimer 5: 8174 GICv3 26 Level arch_timer 15: 0 GICv3 56 Edge axpu_irq 40: 0 GICv3 106 Edge arm-smmu-v3-evtq 43: 0 GICv3 109 Edge arm-smmu-v3-gerror 44: 0 GICv3 34 Level rtc-pl031 45: 175 GICv3 33 Level uart-pl011 46: 0 GICv3 23 Level arm-pmu 47: 7790 GICv3 36 Level virtio0 48: 0 9030000.pl061 3 Edge GPIO Key Poweroff IPI0: 0 Rescheduling interrupts IPI1: 0 Function call interrupts IPI2: 0 CPU stop interrupts IPI3: 0 CPU stop (for crash dump) interrupts IPI4: 0 Timer broadcast interrupts IPI5: 0 IRQ work interrupts IPI6: 0 CPU wake-up interrupts Err: 0 Now axpu_irq has moved to INTID 15 (as expected), but it changed to "GICv3 56 Edge". How is this GICv3 number is assigned? (and I guess edge trigger is the default for INTID < 16. Because it's SGI..?) I thought I connected my device (axpu) to SPI 15 in qemu code but this GICv3 input number changes according to with what number I called reqeust_irq. Could anyone tell me what is going on here? Thank you. Chan Kim
participants (6)
-
'Greg KH' -
Chan Kim -
Greg KH
-
Ozgur Kara -
Ozgur Karatas -
Peter Maydell