Unexpected scheduling with mutexes

[Greg KH]

On Wed, Apr 03, 2019 at 11:33:56AM +0200, Martin Christian wrote:
> >> For certain values of `X` there is a significant difference in size
> >> between the two files, which I don't expect.
> >>
> >> A read call to the driver does the following:
> >>  1. `mutex_lock_interruptible(iolock)`
> >>  2. `usb_bulk_msg(dev, pipe, buf, X, timeout)`
> >>  3. `mutex_unlock(iolock)`
> >>  4. `copy_to_user(buf)`
> > 
> > What are these values of X that cause differences here?
> 
> Starting around 1k character device A gets more data until it turns over
> at around 4K. Request size from 10K yield the expected data rates.

Those are huge USB data stream sizes, what is the size of your USB
endpoints?

By doing large transfers like this, you are causing the USB core to do
all the work (which is fine), but while that happens, lots of other
things happen at the same time, making trying to measure things much
more difficult.

> Character device A is a "real" random source and returns data much
> slower than device B, which is a pseudo random source.

So those map to different USB device endpoints?

> > But if you are trying to somehow create a real api that you have to
> > enforce the passing off of writing data from two different character
> > devices in an interleaved format, you are doing this totally wrong, as
> > this is not going to work with a simple mutex, as you have found out.
> 
> As mentioned above, the USB device provides two different streams of
> random. But the device can process only one request at a time. Also I
> didn't want to have too much dynamic memory allocation, because I would
> need to allocate up to 64KB kernel memory on each open.

So your USB device can not handle data from different endpoints at the
same time?  Or is it multiplexing it on the same endpoint?  You need to
provide a bit more information about your device for us to be able to
help you out better.

> That's because the USB device is designed to provide up to 64K of random
> in a single "request". A request has a header and footer "protecting"
> the request as a whole from data confusion.

Who are you protecting the request from being confused from?  The
kernel?  Userspace?  Something else?

Why not just tie your device into the kernel's random number system like
other USB devices do that provide good entropy to the system?  That way
you don't have to do crazy things with character streams and blocking
requests :)

> To make things simpler I decided to just allow one user space process at
> a time for each source - which is enough for our application. But yes,
> that could probably also got to user space.

Again, why not just use the random services provided by the kernel, and
have your device feed that?  That way everyone benefits and you don't
have to do odd things and create a custom user api that no one else can
use.

thanks,

greg k-h