Why I can't map memory with mmap
Mandeep Sandhu
mandeepsandhu.chd at gmail.com
Mon Aug 18 16:23:33 EDT 2014
I tried your program and got the same result, i.e "Resource temporarily
unavailable" for an ordinary user and "Cannot allocate memory" if run as
root.
However, reading the Huge TLB pages doc (Documentation/vm/hugetlbpage.txt),
I saw the following para which might explain the error:
...
Once a number of huge pages have been pre-allocated to the kernel huge page
pool, a user with appropriate privilege can use either the mmap system call
or shared memory system calls to use the huge pages. See the discussion of
Using Huge Pages, below.
The administrator can allocate persistent huge pages on the kernel boot
command line by specifying the "hugepages=N" parameter, where 'N' = the
number of huge pages requested. This is the most reliable method of
allocating huge pages as memory has not yet become fragmented.
...
...
I don't have my kernel booted with it, so that might explain why mmap
fails for MAP_HUGETLB.
I can't restart my machine with the new flag right now, but maybe you can try.
HTH,
-mandeep
On Mon, Aug 18, 2014 at 11:30 AM, Kevin O'Gorman <kogorman at gmail.com> wrote:
> Now I'm befuddled. I modified the source so that it was easy to try
> different combinations. If I sudo it, I can map a 1GiB (1<<30) private
> anonymous region if I sudo the program. I cannot find any combination with
> MAP_HUGETLB that works at all.
>
> Here it is in working form, with an anonymous map and no backing file.
> The mmap request is for a single byte, so it works for ordinary users Try
> uncommenting the MAP_HUGETLB line, and I'd love to hear about a version
> that works, even if it's only for root.
>
> Since I'm aiming for a huge memory region, I figured HUGETLB would make
> sense to limit the overhead.
>
> ============================ CUT HERE ================
> /**
> * @file
> * <pre>"Find out the limits on locked memory"
> * Last Modified: Mon Aug 18 11:21:23 PDT 2014
> * @author Kevin O'Gorman
> */
>
> #define _GNU_SOURCE /* enable some of the mmap flags */
> #include <unistd.h>
> #include <stdlib.h>
> #include <stdio.h>
> #include <string.h>
> #include <sys/mman.h>
>
> /* for getrlimit(2) */
> #include <sys/time.h>
> #include <sys/resource.h>
>
> /* for open(2) */
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <sys/fcntl.h>
>
> int
> main(int argc, char *argv[])
> {
> void *where;
> size_t length = 1;
> struct rlimit rlim;
> int flags = 0;
> int fd = -1;
>
> // fd = open("solvedtaq", O_RDWR); if (fd == -1) perror("open");
> printf("fd is %d\n", fd);
>
> // flags |= MAP_HUGETLB;
> flags |= MAP_PRIVATE;
> flags |= MAP_LOCKED;
> flags |= MAP_ANONYMOUS;
>
> if (getrlimit(RLIMIT_MEMLOCK, &rlim) == 0) {
> printf("RLIMIT_MEMLOCK: hard: %lld, soft: %lld\n", (long
> long)rlim.rlim_max, (long long)rlim.rlim_cur);
> } else {
> perror("getrlimit failed");
> }
>
> where = mmap(NULL, length, PROT_READ | PROT_WRITE, flags, fd, 0);
> if (where != MAP_FAILED) {
> printf("Mapped at %p\n", where);
> } else {
> perror("Mapping failed");
> }
>
> return EXIT_SUCCESS;
> }
> ============================ CUT HERE ================
>
>
>
> On Mon, Aug 18, 2014 at 10:24 AM, Kevin O'Gorman <kogorman at gmail.com>
> wrote:
>
>>
>> On Mon, Aug 18, 2014 at 9:46 AM, <Valdis.Kletnieks at vt.edu> wrote:
>>
>>> On Mon, 18 Aug 2014 08:15:12 -0700, "Kevin O'Gorman" said:
>>>
>>> > maybe someone here can help me figure out how to map a big (really big)
>>> > work area and lock it in memory.
>>>
>>> What problem are you trying to solve by doing this? It usually doesn't
>>> make
>>> sense to lock more than a small chunk in memory - that's mostly for
>>> crypto
>>> programs like PGP that are paranoid about keys being written to swap
>>> spaces,
>>> and a *very* few highly performance oriented programs. 99.8% of the
>>> time,
>>> the kernel will do a reasonable job of making sure the right pages are in
>>> memory.
>>>
>>> Plus, programmers almost always over-estimate what they *really* need to
>>> lock
>>> down. The end result is that performance ends up being *worse*, because
>>> they
>>> lock out more than needed, leaving all the rest of the processes on the
>>> system
>>> fighting over a too-small pool of pages. Really sucks when you actually
>>> have
>>> to read from disk every I/O because you no longer have an in-core file
>>> cache :)
>>>
>>> I'm solving a huge alpha-beta search problem, and want a huge cache
>> table tocheck permutations. That's why I increased RAM from 8 to 32 GB,
>> and would
>> be happy to use the extra 24GB all for the cache. (Cache table is like a
>> hash table but with collisions causing replacement rather than chaining).
>>
>> Even in my small tests, the cache is giving me speedups in the
>> neighborhood of
>> a million, so I'm going big. Because this is about speed, I do not *ever*
>> want it to
>> swap.
>>
>>
>>> > I wrote a little test program to try this out, and it fails. As a
>>> regular
>>> > user, perror() tells me some "resource is temporarily unavailable". As
>>> > root, it says it "cannot allocate memory". I'm only asking for 1 byte.
>>>
>>> > where = mmap(NULL, length, PROT_READ | PROT_WRITE, MAP_PRIVATE |
>>> > MAP_ANONYMOUS | MAP_HUGETLB | MAP_LOCKED | MAP_POPULATE, -1, 0);
>>>
>>> Remember that no matter what "length" you pass, even if it's 1 byte, the
>>> kernel has to allocate at least enough integer pages to cover the
>>> request.
>>> You ask for 4100 bytes, it has to allocate 2 pages because 4096 isn't
>>> quite big enough.
>>>
>>> And if you ask for MAP_HUGETLB, you're going to get at least 1 page.
>>> Which
>>> is probably 2M in size. And that can be painful if the default max
>>> lockable
>>> memory is less than 2M (at least on my Fedora Rawhide box, it's all of
>>> 64K). There's also some interaction danger with MAP_POPULATE and memory
>>> fragmentation. But both POPULATE and HUGETLB have a high chance of
>>> not playing well with LOCKED.
>>>
>>
>> Good catch. Indeed, removing MAP_HUGETLB allows the program to succeed.
>> I'll try increasing the resource limit, but may have to have to sudo it, do
>> the mmap() and then chusr to my real ID.
>>
>> The fd of -1 that some folks thought might be a problem is suggested in
>> the man page for use with MAP_ANONYMOUS. That flag means I'm not using a
>> backing file, after all.
>>
>> --
>> Kevin O'Gorman
>>
>> programmer, n. an organism that transmutes caffeine into software.
>> Please consider the environment before printing this email.
>>
>
>
>
> --
> Kevin O'Gorman
>
> programmer, n. an organism that transmutes caffeine into software.
> Please consider the environment before printing this email.
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140818/40b2e652/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 441 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140818/40b2e652/attachment-0001.gif
More information about the Kernelnewbies
mailing list