Why I can't map memory with mmap

[Valdis.Kletnieks at vt.edu]

On Mon, 18 Aug 2014 08:15:12 -0700, "Kevin O'Gorman" said:

> maybe someone here can help me figure out how to map a big (really big)
> work area and lock it in memory.

What problem are you trying to solve by doing this?  It usually doesn't make
sense to lock more than a small chunk in memory - that's mostly for crypto
programs like PGP that are paranoid about keys being written to swap spaces,
and a *very* few highly performance oriented programs.  99.8% of the time,
the kernel will do a reasonable job of making sure the right pages are in
memory.

Plus, programmers almost always over-estimate what they *really* need to lock
down.  The end result is that performance ends up being *worse*, because they
lock out more than needed, leaving all the rest of the processes on the system
fighting over a too-small pool of pages.  Really sucks when you actually have
to read  from disk every I/O because you no longer have an in-core file cache :)

> I wrote a little test program to try this out, and it fails.  As a regular
> user, perror() tells me some "resource is temporarily unavailable".  As
> root, it says it "cannot allocate memory".  I'm only asking for 1 byte.

>   where = mmap(NULL, length, PROT_READ | PROT_WRITE, MAP_PRIVATE |
> MAP_ANONYMOUS | MAP_HUGETLB | MAP_LOCKED | MAP_POPULATE, -1, 0);

Remember that no matter what "length" you pass, even if it's 1 byte, the
kernel has to allocate at least enough integer pages to cover the request.
You ask for 4100 bytes, it has to allocate 2 pages because 4096 isn't
quite big enough.

And if you ask for MAP_HUGETLB, you're going to get at least 1 page.  Which
is probably 2M in size.  And that can be painful if the default max lockable
memory is less than 2M (at least on my Fedora Rawhide box, it's all of
64K).  There's also some interaction danger with MAP_POPULATE and memory
fragmentation.  But both POPULATE and HUGETLB have a high chance of
not playing well with LOCKED.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140818/01c4d8eb/attachment.bin