Inexplicable PROT_EXEC flag set on mmap callback

Kenneth Adam Miller kennethadammiller at
Thu Jan 14 12:28:43 EST 2016

On Thu, Jan 14, 2016 at 12:00 PM, Mike Krinkin <krinkin.m.u at>

> Hi, i have a couple of questions to clarify, if you don't mind
> On Thu, Jan 14, 2016 at 11:04:28AM -0500, Kenneth Adam Miller wrote:
> > I have a custom drive and userland program pair that I'm using for a very
> > special use case at my workplace where we are mapping specific physical
> > address ranges into userland memory with a mmap callback. Everything
> works
> > together well with a C userland program that calls into our driver's
> ioctl
> > and mmap definitions, but for our case we are using an alternative
> systems
> > language just for the userland program.
> So you have userland app written in C, and another not written in C?
> The former works well while the latter doesn't, am i right?

Yes, the former works in so much as mmap completes successfully. I've
verified that the
parameters are identical in the non-C program. The issue of just using the
C only program
is that the actual implementation of interest is in the non-C program, and
that's because
that language facilitates other features that are *required* on our end.

> > That mmap call is failing (properly
> > as we want) out from the driver's mmap implementation due to the fact
> that
> > the vm_flags have the VM_EXEC flag set. We do not want users to be able
> to
> > map the memory range as executable, so the driver should check for this
> as
> > it does. The issue is in the fact that somewhere between where mmap is
> > called and when the parameters are given to the driver, the vma->vm_flags
> > are being set to 255. I've manually checked the values being given to the
> > mmap call in our non-C binary, and they are *equivalent* in value to that
> > of the C program.
> By "manually" do you mean strace? Could you show strace output for
> both apps? And also could you show readelf -l output for both binaries?

By manually, I mean with a print call just before the mmap call in each of
programs. Right now, I'm working on getting a strace output, but I have to
run that in qemu.
To be able to run it in qemu in order to isolate the driver and all from my
host, I have to build
with buildroot. So I'll email that when I get it, but it'll be a while.

> >
> > My question is, is there anything that can cause the vma->vm_flags to be
> > changed in the trip between when the user land program calls mmap and
> when
> > control is delivered to the mmap callback?
> > _______________________________________________
> > Kernelnewbies mailing list
> > Kernelnewbies at
> >
-------------- next part --------------
An HTML attachment was scrubbed...

More information about the Kernelnewbies mailing list