Memory allocation problems on RHEL 6.3 kernel version 2.6.32-279.el6.x86_64

[Amit Agarwal]

On 14-07-2014 12:20, shhuiw wrote:
> Will you please show us your application code, if possible? Or at 
> least the mmap part.
The application code is proprietary, so I cannot disclose that.

For the mmap part, we are not doing mmap explicitly in our code, rather 
the binary is linked to some dynamic library (here in the strace that I 
took, it was oracle client).  Application crashes while doing mmap for 
the oracle library, here is the trace:


----- Call Stack Trace -----
calling              call     entry                argument values in hex
location             type     point                (? means dubious value)
-------------------- -------- -------------------- 
----------------------------
mmap(offset=33230848, len=2068480) failed with errno=12 for the file 
./libclntsh.so.11.1
mmap(offset=33230848, len=2068480) failed with errno=12 for the file 
./libclntsh.so.11.1
kpedbg_dmp_stack()+  call     556FC244             FFE3D814 ? 0 ?
219
kpeDbgCrash()+72     call     556F06E4             0 ? 5 ? FFE3D8E4 ? 
A42C90 ?
                                                    4 ? FFE3D890 ?
56AA4FDF             call     55701184             0 ? 5 ? 56E2AEFC ? 2 
? 4 ?
                                                    50 ? 4 ? FFE3E8FD ?
56785C74             call     00000000             FFE3D8F4 ? 56F27E60 ?
<stripped>  signal   00000000             B ? FFE3EDAC ? FFE3EE2C ?
PKc()+255
<stripped>  call     _ZN11CThreadDataC1E  FFE3F2C8 ? FFE3F32E ? 13 ?
<stripped>           PKc()                5720C14A ? 57144297 ? 0 ?
)+127
start_thread()+201   call     00000000             92ECDD8 ? FFE3FB70 ?
                                                    FFE3FB70 ? FFE3FB70 ?
clone()+94           call     00000000             FFE3FB70 ? 0 ? 0 ? 0 
? 0 ?
                                                    0 ?

We have tried by reducing the footprint for our application by 
decreasing the application caching of some data and see that the 
application can come up properly until the VIRT in top command output is 
about 2.8GB after which application starts crashing.

> So that we can figure out the testcase to reproduce on our boxes.

-- 
Thanks,
-aka
http://blog.amit-agarwal.co.in