Memory allocation problems on RHEL 6.3 kernel version 2.6.32-279.el6.x86_64

[shhuiw]

Hi,

How about setting /proc/sys/vm/min_free_kbytes to small values, e.g 128?
And what's the output on other workable system?



--

Regards,
shhuiw




At 2014-07-09 08:23:29, "Amit Agarwal" <amit at amit-agarwal.co.in> wrote:
>Hi All,
>
>We are running a 32 bit application on RHEL6.3-64 bit OS with kernel 
>version 2.6.32-279.el6.x86_64.
>
>While running this application we see the following when running under 
>strace:
>mmap(offset=33230848, len=2068480) failed with errno=12 for the file 
><so file name>
>
>The system has enough RAM, total 16GB and about 12 GB free. We checked 
>buddyinfo on the system, when the application is running and see the 
>below trend:
>Node 0, zone   Normal   5200  21396  21389  21516  16202  12770   9054  
>4459   1430    168    313
>Node 0, zone   Normal   5231  21395  21389  21516  16202  12770   9054  
>4459   1430    168    313
>Node 0, zone   Normal   5128  21401  21389  21516  16202  12770   9054  
>4459   1430    168    313
>Node 0, zone   Normal   5060  21405  21391  21516  16202  12770   9054  
>4459   1430    168    313
>..............
>Node 0, zone   Normal    483  17946  21342  21516  16202  12770   9054  
>4459   1430    168    313
>Node 0, zone   Normal    315  17937  21342  21516  16202  12770   9054  
>4459   1430    168    313
>Node 0, zone   Normal    345  17891  21352  21516  16202  12770   9054  
>4459   1430    168    313
>Node 0, zone   Normal    278  17785  21352  21516  16202  12770   9054  
>4459   1430    168    313
>
>
>At this point the application crashes with mmap error.
>
>
>Page Type information on the system:
>Page block order: 9
>Pages per block:  512
>
>Free pages count per migrate type at order       0      1      2      3 
>     4      5      6      7      8      9     10
>Node    0, zone      DMA, type    Unmovable      1      0      0      1 
>     0      0      1      0      0      0      0
>Node    0, zone      DMA, type  Reclaimable      0      0      0      0 
>     0      0      0      0      0      0      0
>Node    0, zone      DMA, type      Movable      0      0      0      0 
>     0      0      0      0      0      0      3
>Node    0, zone      DMA, type      Reserve      0      0      0      0 
>     0      0      0      0      0      1      0
>Node    0, zone      DMA, type      Isolate      0      0      0      0 
>     0      0      0      0      0      0      0
>Node    0, zone    DMA32, type    Unmovable    410    261    133     51 
>    24     14      7      2      0      0      0
>Node    0, zone    DMA32, type  Reclaimable   2573   1806   1108    889 
>   522    175     31      0      0      0      0
>Node    0, zone    DMA32, type      Movable    408    352    345    341 
>   328    312    310    260    216    180    356
>Node    0, zone    DMA32, type      Reserve      0      0      0      0 
>     0      0      0      0      0      2      0
>Node    0, zone    DMA32, type      Isolate      0      0      0      0 
>     0      0      0      0      0      0      0
>Node    0, zone   Normal, type    Unmovable    434    788    518    137 
>    58     24     13      4      5      0      0
>Node    0, zone   Normal, type  Reclaimable      1    110   2136   3212 
>   414     25      8      7      8      0      0
>Node    0, zone   Normal, type      Movable   8135  20535  17946  18171 
>15732  12719   9033   4448   1417    169    312
>Node    0, zone   Normal, type      Reserve      0      0      0      0 
>     0      0      0      0      0      0      1
>Node    0, zone   Normal, type      Isolate      0      0      0      0 
>     0      0      0      0      0      0      0
>
>Number of blocks type     Unmovable  Reclaimable      Movable      
>Reserve      Isolate
>Node 0, zone      DMA            1            0            6            
>1            0
>Node 0, zone    DMA32           11           79         1436            
>2            0
>Node 0, zone   Normal          188          337         6129            
>2            0
>
>
>When we are running the same application on other system, it comes up. 
>So, it has to do with memory allocation setting of some sort.
>
>So, Is there some setting that we can use to allow the kernel to 
>allocate the 313 - 4M  pages that are free for 4k memory allocations? If 
>so, what should we change on the system?
>
>Will setting vm.zone_reclaim_mode to 1 help here?
>
>Do I need to provide more information, if yes, what?
>
>-- 
>Thanks,
>-aka
>http://blog.amit-agarwal.co.in
>
>
>_______________________________________________
>Kernelnewbies mailing list
>Kernelnewbies at kernelnewbies.org
>http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140710/09678bbc/attachment-0001.html