Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?
Hi, Why does "page allocation failure" occur whereas there are still "58*4096kB (C)"(*I think it indicates there are 58 order 10 memory could be used*) could be used? Here is the related log: [ 2161.623563] xxxx: page allocation failure: order:10, mode:0x2084020(GFP_ATOMIC|__GFP_COMP) [ 2161.632085] CPU: 0 PID: 179 Comm: AiApp Not tainted 4.9.56 #53 [ 2161.637947] Call Trace: [<802f63f2>] dump_stack+0x1e/0x3c [<800f6cf4>] warn_alloc+0x100/0x148 [<800f709c>] __alloc_pages_nodemask+0x2bc/0xb5c [<801120fe>] kmalloc_order+0x26/0x48 [<80112158>] kmalloc_order_trace+0x38/0x98 [<8012c5d8>] __kmalloc+0xf4/0x12c [<8048ac78>] alloc_ep_req+0x5c/0x98 [<8048f232>] source_sink_recv+0x2a/0xe0 [<8048f35e>] usb_sourcesink_bulk_read+0x76/0x1c8 [<8048f770>] usb_sourcesink_read+0xfc/0x2c8 [<80134d58>] __vfs_read+0x30/0x108 [<80135c14>] vfs_read+0x94/0x128 [<80136d12>] SyS_read+0x52/0xd4 [<8004a246>] csky_systemcall+0x96/0xe0 [ 2161.689204] Mem-Info: [ 2161.691518] active_anon:3268 inactive_anon:2 isolated_anon:0 [ 2161.691518] active_file:1271 inactive_file:89286 isolated_file:0 [ 2161.691518] unevictable:0 dirty:343 writeback:0 unstable:0 [ 2161.691518] slab_reclaimable:2019 slab_unreclaimable:644 [ 2161.691518] mapped:4282 shmem:4 pagetables:59 bounce:0 [ 2161.691518] free:62086 free_pcp:199 free_cma:60234 [ 2161.724334] Node 0 active_anon:13072kB inactive_anon:8kB active_file:5084kB inactive_file:357144kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:17128kB dirty:1372kB writeback:0kB shmem:16kB writeback_tmp:0kB unstable:0kB pages_scanned:0 all_unreclaimable? no [ 2161.748626] Normal free:248344kB min:2444kB low:3052kB high:3660kB active_anon:13072kB inactive_anon:8kB active_file:5084kB inactive_file:357144kB unevictable:0kB writepending:1372kB present:1048572kB managed:734568kB mlocked:0kB slab_reclaimable:8076kB slab_unreclaimable:2576kB kernel_stack:608kB pagetables:236kB bounce:0kB free_pcp:796kB local_pcp:796kB free_cma:240936kB [ 2161.781670] lowmem_reserve[]: 0 0 0 [ 2161.785225] Normal: 4*4kB (UEC) 3*8kB (EC) 3*16kB (UEC) 2*32kB (UE) 2*64kB (UE) 2*128kB (UE) 2*256kB (EC) 1*512kB (E) 3*1024kB (UEC) 3*2048kB (UEC) 58*4096kB (C) = 248344kB 90573 total pagecache pages [ 2161.803526] 262143 pages RAM [ 2161.806410] 0 pages HighMem/MovableOnly [ 2161.810264] 78501 pages reserved [ 2161.813509] 90112 pages cma reserved
On Thu, 18 Jun 2020 14:21:05 +0800, 孙世龙 said:
Why does "page allocation failure" occur whereas there are still "58*4096kB (C)"(*I think it indicates there are 58 order 10 memory could be used*) could be used?
Here is the related log:
[ 2161.623563] xxxx: page allocation failure: order:10, mode:0x2084020(GFP_ATOMIC|__GFP_COMP)
Most likely, the allocation wanted some other type of allocation. The (C) on the order-10 says it's an CMA area. static const char types[MIGRATE_TYPES] = { [MIGRATE_UNMOVABLE] = 'U', [MIGRATE_MOVABLE] = 'M', [MIGRATE_RECLAIMABLE] = 'E', [MIGRATE_HIGHATOMIC] = 'H', #ifdef CONFIG_CMA [MIGRATE_CMA] = 'C', #endif #ifdef CONFIG_MEMORY_ISOLATION [MIGRATE_ISOLATE] = 'I', #endif If the call was for an unmovable, movable, reclaimable, or highatomic allocation, you lose. If you look at the source for alloc_ap_req(), you find it wants GFP_ATOMIC, not CMA. And your box is fresh out of contiguous order-10 spaces that aren't CMA, and you're down to your last 3 order-9 flagged as (UEC). I admit I find it a tad suspicious that the USB gadget driver asks for a 4M chunk of memory. Does USB actually support single transfers that large? (I'm not a USB expert)
Why does "page allocation failure" occur whereas there are still "58*4096kB (C)"(*I think it indicates there are 58 order 10 memory could be used*) could be used?
Here is the related log:
[ 2161.623563] xxxx: page allocation failure: order:10, mode:0x2084020(GFP_ATOMIC|__GFP_COMP)
If you look at the source for alloc_ap_req(), you find it wants GFP_ATOMIC, not CMA. And your box is fresh out of contiguous order-10 spaces that aren't CMA, and you're down to your last 3 order-9 flagged as (UEC).
Thank you for the clarification. I understand it on a deeper level with your help. Why doesn't the kernel use two memory blocks whose size is 2048KB(i.e.*oder 9 *) instead of one block *order 10 *(you see, there are still three free blocks and 2048KB*2=4096KB equivalent to the memory size of order 10)?
If you look at the source for alloc_ap_req(), you find it wants GFP_ATOMIC, not CMA. I followed your advice and read the related source code carefully. It's corresponding to the log(i.e. mode:0x2084020(GFP_ATOMIC|__GFP_COMP) ).
Thank you for your attention to this matter. Look forward to hearing from you. Best regards. Valdis Klētnieks <valdis.kletnieks@vt.edu> 于2020年6月19日周五 下午12:48写道：
On Thu, 18 Jun 2020 14:21:05 +0800, sunshiong said:
Why does "page allocation failure" occur whereas there are still "58*4096kB (C)"(*I think it indicates there are 58 order 10 memory could be used*) could be used?
Here is the related log:
[ 2161.623563] xxxx: page allocation failure: order:10, mode:0x2084020(GFP_ATOMIC|__GFP_COMP)
Most likely, the allocation wanted some other type of allocation. The (C) on the order-10 says it's an CMA area.
static const char types[MIGRATE_TYPES] = { [MIGRATE_UNMOVABLE] = 'U', [MIGRATE_MOVABLE] = 'M', [MIGRATE_RECLAIMABLE] = 'E', [MIGRATE_HIGHATOMIC] = 'H', #ifdef CONFIG_CMA [MIGRATE_CMA] = 'C', #endif #ifdef CONFIG_MEMORY_ISOLATION [MIGRATE_ISOLATE] = 'I', #endif
If the call was for an unmovable, movable, reclaimable, or highatomic allocation, you lose.
If you look at the source for alloc_ap_req(), you find it wants GFP_ATOMIC, not CMA. And your box is fresh out of contiguous order-10 spaces that aren't CMA, and you're down to your last 3 order-9 flagged as (UEC).
I admit I find it a tad suspicious that the USB gadget driver asks for a 4M chunk of memory. Does USB actually support single transfers that large? (I'm not a USB expert)
On Fri, 19 Jun 2020 14:56:20 +0800, 孙世龙 sunshilong said:
Why doesn't the kernel use two memory blocks whose size is 2048KB(i.e.*oder 9 *) instead of one block *order 10 *(you see, there are still three free blocks and 2048KB*2=4096KB equivalent to the memory size of order 10)?
Most parts of the kernel, when asking for very high-order allocations, *will* have a fallback strategy to use smaller chunks. So, for instance, if a device need a 1M buffer and supports scatter-gather operations, if 1M of contiguous memory isn't available, the kernel can ask for 4 256K chunks and have the I/O directed into the 4 areas. However, if the memory *has* to be contiguous (for example, no scatter/gather available, or it's for an array data structure), then it can't do that. And in fact, that fallback could very well have happened in this case - I didn't bother chasing back to see if the gadget driver does recovery by allocating multiple smaller chunks. (That's a good "exercise for the student"... :)
Why doesn't the kernel use two memory blocks whose size is 2048KB(i.e.*oder 9 *) instead of one block *order 10 *(you see, there are still three free blocks and 2048KB*2=4096KB equivalent to the memory size of order 10)?
Most parts of the kernel, when asking for very high-order allocations, *will* have a fallback strategy to use smaller chunks. So, for instance, if a device need a 1M buffer and supports scatter-gather operations, if 1M of contiguous memory isn't available, the kernel can ask for 4 256K chunks and have the I/O directed into the 4 areas. *However, if the memory *has* to be contiguous (for* *example, no scatter/gather available, or it's for an array data structure),* then it can't do that.
Thank you for the clarification. I understand it on a deeper level with your help. How can I know whether scatter/gather is available or not? In another word, when it's available and when it's not? I do not intend to ask the behavior of gadget driver. I just wonder how I can confirm it in general. Thank you for your attention to this matter. Look forward to hearing from you. Best regards. Valdis Klētnieks <valdis.kletnieks@vt.edu> 于2020年6月19日周五 下午3:14写道：
On Fri, 19 Jun 2020 14:56:20 +0800, 孙世龙 sunshilong said:
Why doesn't the kernel use two memory blocks whose size is 2048KB(i.e.*oder 9 *) instead of one block *order 10 *(you see, there are still three free blocks and 2048KB*2=4096KB equivalent to the memory size of order 10)?
Most parts of the kernel, when asking for very high-order allocations, *will* have a fallback strategy to use smaller chunks. So, for instance, if a device need a 1M buffer and supports scatter-gather operations, if 1M of contiguous memory isn't available, the kernel can ask for 4 256K chunks and have the I/O directed into the 4 areas. However, if the memory *has* to be contiguous (for example, no scatter/gather available, or it's for an array data structure), then it can't do that.
And in fact, that fallback could very well have happened in this case - I didn't bother chasing back to see if the gadget driver does recovery by allocating multiple smaller chunks.
(That's a good "exercise for the student"... :)
How can I know whether scatter/gather is available or not? In another word, when it's available and when it's not? I do not intend to ask the behavior of gadget driver. I just wonder how I can confirm it in general.
The following applies for device features in general, not just scatter/gather. Sometimes you get lucky, and the bus standard in use (PCI, USB, SPI/I2C, etc) mandates that the feature is supported in a standard way. So if you're writing a device driver for a specific card that uses a Wombat 9348j chipset, you'll probably need to get in touch with the company that makes the Wombat 9348j chipset and get the data sheet for it, and that will hopefully have the chipset programming details needed. If you're unlucky, the company won't give you the datasheet, or it won't contain the info, so you get to reverse engineer the chipset. One popular technique is to look at *other* chips in the Wombat 93 series, determine via some means whether those chipsets support scatter-gather, and how it's enabled. Then you put the same code in your driver, and see what happens. Possibility 1: Scatter-gather works just fine and the 16K I/O ends up in the 4 pages you allocated. Success! Possibility 2: It doesn't work,and the 16K of I/O hits your first 4K page, and the next 3 sequential pages even though they're something else, and your kernel crashes and burns because you just overlaid 12K worth of page tables or struct tasks or something important. Possibility 3: It doesn't work, but it takes you a while to find out because during your tests, the 4 pages happened to be allocated next to each other. It has happened that nobody notices until the driver is part of the mainline kernel, which is usually embarrassing... Possibility 4: It doesn't work, and the bit that says "use scatter/gather" in the IO command structure on the other chipsets actually enables the undocumented auto-defenestration capability of the chipset. This is usually worse than just embarrassing...
participants (3)
-
Valdis Klētnieks -
孙世龙 -
孙世龙 sunshilong