I am writing some software that monitors a guest VM using virtual-machine introspection and "hijacks" system calls under certain conditions. For example, the program might inject an int3/breakpoint into the guest kernel at the entry point to sys_open. When the breakpoint is hit, the program might set the guest instruction pointer to the address to which sys_open would have itself returned and set register RAX to some desired error-code return value.
The problem I am encountering is that for some reason the process is triggering a "uprobe ... failed to handle uretprobe" message from the guest kernel. I do not yet know enough about uprobes to understand what might be causing this. Is there something in procedures such as sys_open which must execute to prevent the error which causes the kernel to print this message?
What vm hypervisor do you use?
We are using Xen + libvmi. I have continued to read the kernel sources, and as best as I can understand it the kernel installs uprobe instrumentation if it detects a software breakpoint. Our program does not reinject the software breakpoints it services back into the guest, so I am still trying to figure out why uprobes seems to get triggered. -- Mike :wq