The usual solution in low latency environments where the higher delays is an issue is to simply spin on a poll. If your polling multiple sockets then you need to cater for this in your code and avoid the kernel paths wherever possible. Depending on the hardware configuration consistent low latency performance is very possible, I've done it on a number of occasions, you need to avoid the kernel at all costs. Of course I know nothing of your specific requirements... Cheers On Thu, Oct 16, 2014 at 11:17 PM, Greg KH <greg@kroah.com> wrote:

On Thu, Oct 16, 2014 at 02:09:05PM +0200, Nev Ikte wrote:

...

Hi,

I've a server using epoll and I've noticed that when the traffic is low, the epoll_wait() latency goes up.

I've tried to reproduce it with a single client and basically, if ep_poll() is able to find an event or the timeout is 0, the latency is down to 5usec, otherwise if it enters the waitqueue the latency goes up to 10-25usec, which impact the application performance.

Looking at the code there is something like a todo ("Is it worth to try to dig for events ?") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/even...

Anyone knows how the situation can be improved? in theory having something like a 50usec spin loop checking for events before entering in the waitqueue should improve the situation.

suggestion? patches?

The epoll code is subtle, tricky, and very very difficult to modify. I suggest trying your suggestion and seeing if it helps any. But be prepared to go down a long rabbit-hole in this area of the kernel.

Enjoy the trip! :)

greg k-h

_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

-- -- Teach your kids Science, or somebody else will :/ jason@ball.net vk2vjb@google.com callsign: vk2vjb

Hi! On 13:02 Thu 16 Oct , Valdis.Kletnieks@vt.edu wrote:

On Thu, 16 Oct 2014 14:09:05 +0200, "Nev Ikte" said:

...

I've tried to reproduce it with a single client and basically, if ep_poll() is able to find an event or the timeout is 0, the latency is down to 5usec, otherwise if it enters the waitqueue the latency goes up to 10-25usec, which impact the application performance.

Is it possible that what you're actually measuring here is your CPU's latency coming out of a sleep state if every task is in a wait queue?

I have seen wakeup latencies of ~100usec on some systems. You may want to disable sleep states and see of performance gets better. Take a look at: Documentation/power/pm_qos_interface.txt Also, you probably want disable frequency scaling: /sys/devices/system/cpu/cpu.../cpufreq/scaling_governor should return "performance" for every cpu. You can change it with eche "performance" > /sys/... -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com