hrtimer + Kernel Networking - freeze

[Thomas Schmidt]

Hello,

i try to do some experiment generating high throughput network traffic and try to analyze the impact of system utilization on the network performance especially inter packet delay variation. One part of that is that i implement basically same functionality in user space and kernel space - where UDP packets are sent (not received) with high cadence and i compare the behavior. the UDP packets have always fixed content.

For the kernel implementation i have done several experiments with netpoll API and alternatively with socket API, where the netpoll_send_udp() respective sock_sendmsg() is triggered by hrtimer. Basically this works like expected, however especially if timer value is extremely low (e.g. < 10000  nsec - well knowing that this is VERY challenging for the kernel) the system freezes totally after few seconds. 

I tried as well to separate the hrtimer + network send into an own kernel thread - but no success - still the system freezes after a while.
This behavior is only observed in case network functions _and_ hrtimer are in kernel space - i made one experiment where only hrtimer is in kernel space and triggers over a signal a user space process to send packets - which is stable (even with extremely optimistic low nano second interval settings of hrtimer) and does not freeze the system.
Unfortunately kernel remote debugging gives not too much idea. Last entry is "hrtimer: interrupt took 11398 ns“...

I assume that the system is just overloaded and not able to process that many timer interrupts. When the network send process runs in user space there is (i guess) a protection but (naturally ?!) when it runs all in kernel space there is no protection against overload.

Questions:

- is there any way to detect that hrtimer interrupts (+ the callback process) taking too much processing time - so that i could prevent a freeze ?

- may i have not used an elegant way to implement my experiment. (I tried 2 alternative approaches A.: start module + basic network setup —>  start hrtimer —> callback sending udp packet and re-setup timer; approach B.: start module —> start thread —> thread function does basic network setup and starts timer —> callback sending udp packet and re-setup time ) is there anything else which would may works better ?

- with the netpoll API and kernel socket API I outsource IP+UDP packet assembly to kernel. How is it impacting if i do own thread ? because the IP/UDP assembly for sure is handled in other thread and there is as well also IO operation from network device module needed - which runs eventually in some other kernel thread  - it might not influence anything if i have my module doing very low amount of logic (assembling IP &UDP is surely more effort) in own thread.


Would be very nice if somebody gives some ideas.

Many Thanks
Thomas S.