cgroups: cannot write on file memory.oom_control
Hedi Boufaied
hedi.boufaied at amadeus.com
Fri Apr 25 09:50:47 EDT 2014
Hi Rami,
I have looked at the example code in your slides and wrote something
similar to handle the OoM condition and it seems to work fine. Thank you
for that.
Since it would be even better for me to avoid playing with OoM, I explored
another option for monitoring memory usage: keeping the OoM killer enabled
and using the "memory threshold" feature (see below) to get notified when
a group's memory usage exceeds the limit.
First, my C code for monitoring the memory usage looked like this (running
as a separate process):
// Assuming that initial memory usage is below the threshold
while(1)
{
if (read(efd, &counter, sizeof(uint64_t)) != sizeof(uint64_t))
die("read eventfd");
printf("Event received: memory usage ABOVE limit\n");
if (read(efd, &counter, sizeof(uint64_t)) != sizeof(uint64_t))
die("read eventfd");
printf("Event received: memory usage BELOW limit\n");
}
Then I thought: what happens if mem usage crosses the threshold more than
once between 2 consecutive calls to read() ? (a scenario that cannot be
formally excluded).
I realised that in this situation the "counter" value returned by read()
would contain the number of occurrences crossing events.
So for example if, between 2 calls to read(), the memory usage goes beyond
then below the threshold, the value of "counter" after the 2nd read will
be 2.
I ended up using this counter behaviour to determine if the current usage
is below or beyond the limit:
counter = 0;
above_mem_limit = 0;
while(1)
{
if (read(efd, &counter, sizeof(uint64_t)) != sizeof(uint64_t))
die("read eventfd");
if(counter%2) above_mem_limit = !above_mem_limit;
printf("Event received! (counter=%d), above_mem_limit=%d\n", counter,
above_mem_limit);
}
My question: I have never seen this "event counter" behaviour documented
anywhere. The cgroups docs and forums do not say anything about the 64bit
value returned by the read() call. Do you think one can safely rely on
this to count the number of crossing events ?
Thanks and Regards,
Hedi
----
From https://www.kernel.org/doc/Documentation/cgroups/memory.txt:
9. Memory thresholds
Memory cgroup implements memory thresholds using the cgroups notification
API (see cgroups.txt). It allows to register multiple memory and memsw
thresholds and gets notifications when it crosses.
To register a threshold, an application must:
- create an eventfd using eventfd(2);
- open memory.usage_in_bytes or memory.memsw.usage_in_bytes;
- write string like "<event_fd> <fd of memory.usage_in_bytes> <threshold>"
to
cgroup.event_control.
Application will be notified through eventfd when memory usage crosses
threshold in any direction.
It's applicable for root and non-root cgroup.
----
Best Regard,
Hedi
From: Hedi Boufaied/NCE/AMADEUS
To: Rami Rosen <roszenrami at gmail.com>@AMAEXTMAIL_MUC,
Cc: kernelnewbies <kernelnewbies at kernelnewbies.org>
Date: 22/04/2014 13:12
Subject: Re: cgroups: cannot write on file memory.oom_control
Hi Rami,
Thank you for the advice. Yes, I will try using the event handler.
Best Regards,
Hedi
From: Rami Rosen <roszenrami at gmail.com>
To: Hedi Boufaied <hedi.boufaied at amadeus.com>,
Cc: kernelnewbies <kernelnewbies at kernelnewbies.org>
Date: 18/04/2014 22:38
Subject: Re: cgroups: cannot write on file memory.oom_control
Sent by: kernelnewbies-bounces at kernelnewbies.org
Hi Hedi,
Well, you can use disabling oom in conjunction with event handler. Thus,
you can free memory by changing memory setting of other cgroups when an
oom occurs in a specific cgroup, and avoid having an oops in that cgroup.
You can see an example of using event handler in conjunction with
disabling the oom killer in
"Namespaces and cgroups in linux", in slides 101/102, see:
http://ramirose.wix.com/ramirosen
A shameless plug - I wrote this presentation of 121 slides about
namespaces and cgroups.
Regards,
Rami Rosen
בתאריך 17 באפר 2014 11:24, "Hedi Boufaied" <hedi.boufaied at amadeus.com>
כתב:
Hi Rami,
Thanks a lot for your sequence. I followed it and it does work.
I now see my process paused in sleep state instead of getting killed when
the cgroup reaches OoO.
This being said:
* I do not really understand why the use_hierarchy option needs to be
disabled in order to allow disabling of the oom killer...
* I'm not sure that disabling the OoO killer is safe/reliable enough: in
some of the trials I made, I saw other processes (xterms) not part of the
OoO cgroup getting completely stuck and I had to reboot my machine...
Best Regards,
Hedi
From: Rami Rosen <roszenrami at gmail.com>
To: Hedi Boufaied <hedi.boufaied at amadeus.com>,
Cc: kernelnewbies <kernelnewbies at kernelnewbies.org>
Date: 15/04/2014 15:39
Subject: Re: cgroups: cannot write on file memory.oom_control
Hi, Hedi,
This sequence works for me:
echo 0 > /sys/fs/cgroup/memory/memory.use_hierarchy
mkdir /sys/fs/cgroup/memory/0
Then, the following two commands changes the value of oom_kill_disable:
echo 1 > /sys/fs/cgroup/memory/0/memory.oom_control
echo 0 > /sys/fs/cgroup/memory/0/memory.oom_control
For more info, please look for the text about enabling/disabling
use_hierarchy in section 6,
http://lxr.free-electrons.com/source/Documentation/cgroups/memory.txt.
Regards,
Rami Rosen
http://ramirose.wix.com/ramirosen
On Mon, Apr 14, 2014 at 12:22 PM, Hedi Boufaied
<hedi.boufaied at amadeus.com> wrote:
> Hi everyone,
>
> I am working with cgroups on OpenSuse (over VirtualBox) and I cannot
disable
> the OOM killer by writing to the file memory.oom_control:
>
>> echo 1 > memory.oom_control
>> -bash: echo: write error: Invalid argument
>
> I can change the memory limit and several other settings by writing to
the
> appropriate files (like memory.limit_in_bytes) but I could never write
to
> file memory.oom_control although I am root.
>
> I saw someone posted a similar issue a few days ago but there was no
> reply...
>
> I have copied below the sequence of command I am using. Any idea what
could
> be the issue ?
>
> Thanks in advance for your help!
>
> Hedi
>
>
> ----
>
> /sys/fs> su - root
>
> /sys/fs> cd cgroup
>
> /sys/fs/cgroup> cgcreate -g memory:/mygroup
>
> /sys/fs/cgroup> cd memory/mygroup
>
> /sys/fs/cgroup/memory/mygroup> echo 32M > memory.limit_in_bytes
>
> /sys/fs/cgroup/memory/mygroup> cat memory.limit_in_bytes
> 33554432
>
> /sys/fs/cgroup/memory/mygroup> echo 1 > memory.oom_control
> -bash: echo: write error: Invalid argument
>
> /sys/fs/cgroup/memory/mygroup> cat memory.oom_control
> oom_kill_disable 0
> under_oom 0
>
> ---
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140425/253a4874/attachment-0001.html
More information about the Kernelnewbies
mailing list