locking a device for exclusive use by userspace process (or kernel thread)
Hi, I have a device and I have to write a driver that exposes the following three operations to kernel modules AND to userspace programs: - LOCK LOCK operation is used to get exclusive access to the device. If exclusive access is already granted, the caller will wait for it be released. - PROCESS PROCESS can be issued only after a successful LOCK operation. PROCESS operations can be issued multiple times between LOCK and UNLOCK. - UNLOCK UNLOCK operation release the exclusive access to the device. This operation can be issued only after a successful LOCK operation. In kernelspace, the lock would be implemented using a semaphore (with an initial count of 1). That's the easy part. In userspace, it's expected LOCK and UNLOCK can be issued in different threads within one process, and, if, while holding the lock, a process die, either being killed or by exit(), an implicit UNLOCK should be done. I've initially thought I could represent the lock by a file descriptor, map LOCK to open() and UNLOCK() to close() (with the added benefit that opened file descriptor will be closed on exit): operation userspace kernelspace LOCK open() ->open() PROCESS ioctl() ->ioctl() UNLOCK close() ->release() open() would block if the lock is already held by some other process (and would not block if O_NONBLOCK or O_NDELAY is used). Seems a great scheme. But that doesn't fit well with Unix / POSIX, as both fork() and dup{,2,3}() or fcntl(, F_DUPFD*,) creates new references to open()'ed device: - A child will inherit the file descriptor for the device after parent fork(). As the child has a file descriptor for the device, the parent couldn't UNLOCK by calling close() since the device won't be released until the child also close() its copy of the file descriptor. It should be noted that even if the child has the device opened by the vertue of inheritance, it does not own the lock, so it should not be allowed to issue PROCESS operation. But at the same time, it prevents the real owner from releasing the lock (... I'm seeing dead locks). So who really own the lock ? It seems awkward to allow an exclusive lock to be shared between two different process even if they are parents. - In the very same process, multiple references to the device could be created with dup(). So close()ing the file descriptor returned by open() won't UNLOCK the device, as the device will only be released once the last reference would be close()'d. It seems awkward to allow an exclusive lock to be "duplicated" within a process. I've tried to (ab)use ->flush() to implement UNLOCK operation so that a call to close() on the device's file descriptor would UNLOCK, even if a duplicated file descriptor exists in the same process or in a child. But this has the unfortunate consequence of allowing to UNLOCK when closing a duplicated file descriptor (and closing the file descriptor in the child could also mean UNLOCK). So in the end, I'm probably not going to use open() for LOCK and close() for UNLOCK. I think I have to use dedicated LOCK and UNLOCK ioctls. But I failed to find a way to be able to release the lock when the process owning the lock die / exit. Perhaps I need a mechanism to be notified of the death of the process owning the lock. Do you have such mean in mind ? Or any other reasonable solution to implement the three operations of such driver ? Regards. -- Yann Droneaud OPTEYA
On Wed, Mar 04, 2015 at 06:59:04PM +0100, Yann Droneaud wrote:
Hi,
I have a device and I have to write a driver that exposes the following three operations to kernel modules AND to userspace programs:
<snip> Why? What type of device is this? And who is asking you to do this homework assignment? good luck, greg k-h
Hi Greg, Le mercredi 04 mars 2015 à 10:11 -0800, Greg KH a écrit :
On Wed, Mar 04, 2015 at 06:59:04PM +0100, Yann Droneaud wrote:
Hi,
I have a device and I have to write a driver that exposes the following three operations to kernel modules AND to userspace programs:
<snip>
Why?
I'm trying to improve the current driver for this device to allow the lock to be released when userspace program owning the lock is killed.
What type of device is this?
It's a device with some kind of over complicated mailbox. A userspace program is supposed to feed the mailbox with multiple commands and no other userspace program should be allowed to play with the device during that time.
And who is asking you to do this homework assignment?
It's not a homework assignment. It's not even something a little penguin ask me to do. I was so sure I could map the semaphore sematics to open() and close() and use that file descriptor to represent the lock that I feel terribly sorry for not being able to do so. (At one point I felt like I was fighting against 40 years of Unix, and such a fight is not going to be won by myself :) So I'm back to step 0 and looking for a way to be able to release a lock in case of a userspace program is killed.
good luck,
Thanks :) Regards. -- Yann Droneaud OPTEYA
On Wed, Mar 04, 2015 at 07:38:52PM +0100, Yann Droneaud wrote:
Hi Greg,
Le mercredi 04 mars 2015 à 10:11 -0800, Greg KH a écrit :
On Wed, Mar 04, 2015 at 06:59:04PM +0100, Yann Droneaud wrote:
Hi,
I have a device and I have to write a driver that exposes the following three operations to kernel modules AND to userspace programs:
<snip>
Why?
I'm trying to improve the current driver for this device to allow the lock to be released when userspace program owning the lock is killed.
What type of device is this?
It's a device with some kind of over complicated mailbox.
That's not very descriptive, what _exact_ type of device is this? What existing driver supports it? Pointers to code would be helpful.
A userspace program is supposed to feed the mailbox with multiple commands and no other userspace program should be allowed to play with the device during that time.
What type of program? What type of user/kernel interface are you using? Why wouldn't a simple "only allow one userspace program to open the device node" work?
And who is asking you to do this homework assignment?
It's not a homework assignment. It's not even something a little penguin ask me to do.
Who asked you to do this in this specific manner? What root problem are you trying to solve? Why not just prevent userspace from talking to the device at the same time, you have control over this, right?
I was so sure I could map the semaphore sematics to open() and close() and use that file descriptor to represent the lock that I feel terribly sorry for not being able to do so.
(At one point I felt like I was fighting against 40 years of Unix, and such a fight is not going to be won by myself :)
So I'm back to step 0 and looking for a way to be able to release a lock in case of a userspace program is killed.
Should be easy to do on the close/release path, why isn't that working? Or a simple reference count would also work, if you lock it properly. Again, pointers to code would be best. thanks, greg k-h
Hi Greg, Hi Yann, I have been actually more naive by implementing it this way (led on by ldd3, page 176). I have the issue for an FPGA board and reconfiguration. Basically if a program has control of the FPGA it does want the FPGA do its computations, without another program changing the design loaded onto the FPGA. For the time being I will settle on the user opening the device to be smart enough to not duplicate the fd. But I think there are usecases for the requested feature. regards Malte On 04/03/15 18:38, Yann Droneaud wrote:
Hi Greg,
Le mercredi 04 mars 2015 à 10:11 -0800, Greg KH a écrit :
On Wed, Mar 04, 2015 at 06:59:04PM +0100, Yann Droneaud wrote:
Hi,
I have a device and I have to write a driver that exposes the following three operations to kernel modules AND to userspace programs: <snip>
Why? I'm trying to improve the current driver for this device to allow the lock to be released when userspace program owning the lock is killed.
What type of device is this? It's a device with some kind of over complicated mailbox.
A userspace program is supposed to feed the mailbox with multiple commands and no other userspace program should be allowed to play with the device during that time.
And who is asking you to do this homework assignment?
It's not a homework assignment. It's not even something a little penguin ask me to do.
I was so sure I could map the semaphore sematics to open() and close() and use that file descriptor to represent the lock that I feel terribly sorry for not being able to do so.
(At one point I felt like I was fighting against 40 years of Unix, and such a fight is not going to be won by myself :)
So I'm back to step 0 and looking for a way to be able to release a lock in case of a userspace program is killed.
good luck,
Thanks :)
Regards.
participants (3)
-
Greg KH -
Malte Vesper -
Yann Droneaud