Typecasting a void pointer to unsigned long in zsmalloc.c
While looking through the zsmalloc.c file I stumbled upon something. On the following line, in the zs_malloc() function: handle = cache_alloc_handle(pool, gfp); https://elixir.bootlin.com/linux/v6.13-rc3/source/mm/zsmalloc.c#L1356 the handle is a result of a typecast from a void pointer to an unsigned long. return (unsigned long)kmem_cache_alloc(...); https://elixir.bootlin.com/linux/v6.13-rc3/source/mm/zsmalloc.c#L333 What's the guarantee that an unsigned long can hold the data from the void pointer? Is this safe to do? Shouldn't there be a special type for doing this type of conversion? I understand the reason for not using the void pointer directly and why they need a "handle" instead, but I'm just a bit confused by the method that has been chosen here.
On Thu, Jan 23, 2025 at 04:52:24PM +0100, Sotir Danailov wrote:
What's the guarantee that an unsigned long can hold the data from the void pointer?
This is a requirement that Linux makes for the compiler to work with it for any platform that Linux runs on.
Is this safe to do?
Yes.
Shouldn't there be a special type for doing this type of conversion?
That type is "unsigned long" :) thanks, greg k-h
On Thu, Jan 23, 2025 at 05:27:03PM +0100, Sotir Danailov wrote:
This is a requirement that Linux makes for the compiler to work with it for any platform that Linux runs on.
Ah I see, thank you! Is this documented somewhere? I was searching for information on this, but nothing came up in the kernel docs.
It's just a "known thing" for Linux platforms. It's documented somewhere I think but I can't remember where, sorry. greg k-h
I see, well if I find some info, I'll link it in the thread if that's okay, in case someone stumbles over this in the future.
Would you mind posting it on the Kernel Newbies website? That would let people fing it for years to come. Thanks! Leam On Thu, Jan 23, 2025, 11:34 Sotir Danailov <sndanailov@gmail.com> wrote:
I see, well if I find some info, I'll link it in the thread if that's okay, in case someone stumbles over this in the future.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Thu, Jan 23, 2025 at 6:41 PM Leam Hall <leamhall@gmail.com> wrote:
Would you mind posting it on the Kernel Newbies website? That would let people fing it for years to come.
I don't mind, but I'm not allowed to edit the wiki and I'm not sure where I'm supposed to add it on the website anyway.
On 1/23/25 12:30, Sotir Danailov wrote:
On Thu, Jan 23, 2025 at 6:41 PM Leam Hall <leamhall@gmail.com> wrote:
Would you mind posting it on the Kernel Newbies website? That would let people fing it for years to come.
I don't mind, but I'm not allowed to edit the wiki and I'm not sure where I'm supposed to add it on the website anyway.
Sotir, I can edit the wiki and I don't mind taking care of it, if you can let me know what it should say. Your kernel skill is much higher than mine. If you want, we can discuss it here so others can help us get the wording and location right. Thanks! Leam Linux Software Engineer (reuel.net/career) Scribe: The Domici War (domiciwar.net) Coding Ne'er-do-well (github.com/LeamHall) Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh Practice allows options and foresight. -- lh
On Thu, Jan 23, 2025 at 8:55 PM Leam Hall <leamhall@gmail.com> wrote:
I can edit the wiki and I don't mind taking care of it, if you can let me know what it should say.
Okay sounds good, I just need to dig around and find out more about the topic and see if there's proper documentation for it on kernel.org. I hope someone, that knows something about it in more details, can just mention in the thread.
Your kernel skill is much higher than mine.
No, definitely not, I'm a complete newbie when it comes to kernel development.
If you want, we can discuss it here so others can help us get the wording and location right.
Yeah of course! I'd prefer this be a public discussion for sure.
Hi. I've found this: Pointers are __u64, cast from/to a uintptr_t on the userspace side and from/to a void __user * in the kernel. Try really hard not to delay this conversion or worse, fiddle the raw __u64 through your code since that diminishes the checking tools like sparse can provide. The macro u64_to_user_ptr can be used in the kernel to avoid warnings about integers and pointers of different sizes. https://origin.kernel.org/doc/html/latest/process/botching-up-ioctls.html#:~...
On Thu, Jan 23, 2025 at 11:09:47PM +0200, Costa Shulyupin wrote:
Hi. I've found this:
Pointers are __u64, cast from/to a uintptr_t on the userspace side and from/to a void __user * in the kernel. Try really hard not to delay this conversion or worse, fiddle the raw __u64 through your code since that diminishes the checking tools like sparse can provide. The macro u64_to_user_ptr can be used in the kernel to avoid warnings about integers and pointers of different sizes.
https://origin.kernel.org/doc/html/latest/process/botching-up-ioctls.html#:~...
That is talking about the ioctl user/kernel boundry and passing pointers across it. Not the use of pointers directly in the kernel tree. The goal of that paragraph is to tell people to always use a 64bit value for a pointer they are putting into an ioctl, which ensures that the structure remains the same size no matter if you are running on a system that has 32bit or 64bit pointers. So while this is great advice, and should be followed, it doesn't really discuss the implicit rule that "unsigned long can always hold a pointer" that Linux relies on. hope this helps, greg k-h
I found this page which looks a bit legacy and doesn't seem to be linked on kernel.org anywhere: https://www.kernel.org/doc/ In here, under "Standards documents applicable to the Linux kernel", there's an interesting line: "LP64 standard defining the size of char, short, int, and long on 32-bit and 64-bit platforms." and it links to this: https://unix.org/whitepapers/64bit.html In the link, under "64-bit Data Models", there's a table, which shows that LP64 means: char 8 bits, short 16 bits, int 32 bits, long 64 bits, pointer 64 bits This lead me to: https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-... which explains that for "System V AMD64 ABI": on page 12: "all pointer types are 64-bit objects (LP64)" on page 17: there's a table showing that "unsigned long (LP64)" is 8 bytes. Finally, in GCC's manual for x86: https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html#index-mabi-7 "The default is to use the Microsoft ABI when targeting Microsoft Windows and the SysV ABI on all other systems." I was wondering how it worked for other architectures, so I decided to grep the Makefiles in the Linux tree and apparently for some architectures it's explicitly stated: ./arch/arm64/Makefile:53:KBUILD_CFLAGS += $(call cc-option,-mabi=lp64) ./arch/riscv/Makefile:33: KBUILD_CFLAGS += -mabi=lp64 What I found in kernelnewbies: https://kernelnewbies.org/InternalKernelDataTypes "One thing that has caused a lot of problems, as 64-bit machines are getting more popular, is the fact that the size of a pointer is not the same as the size of an unsigned integer. The size of a pointer is equal to the size of an unsigned long." I think there should be an explanation both on kernel.org and kernelnewbies what determines the sizes, so that people can have a better idea of what's going on. It should be something that is easy to find and part of "introductions" to kernel development, because it's not obvious at all how all of these things are setup. To me it seems fundamental, to be able to quickly find easy to digest information regarding the data types you're working with.
On 1/24/25 15:45, Sotir Danailov wrote:
I found this page which looks a bit legacy and doesn't seem to be linked on kernel.org anywhere: https://www.kernel.org/doc/ In here, under "Standards documents applicable to the Linux kernel", there's an interesting line: "LP64 standard defining the size of char, short, int, and long on 32-bit and 64-bit platforms." and it links to this: https://unix.org/whitepapers/64bit.html In the link, under "64-bit Data Models", there's a table, which shows that LP64 means: char 8 bits, short 16 bits, int 32 bits, long 64 bits, pointer 64 bits
This lead me to: https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-... which explains that for "System V AMD64 ABI": on page 12: "all pointer types are 64-bit objects (LP64)" on page 17: there's a table showing that "unsigned long (LP64)" is 8 bytes.
Finally, in GCC's manual for x86: https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html#index-mabi-7 "The default is to use the Microsoft ABI when targeting Microsoft Windows and the SysV ABI on all other systems."
I was wondering how it worked for other architectures, so I decided to grep the Makefiles in the Linux tree and apparently for some architectures it's explicitly stated: ./arch/arm64/Makefile:53:KBUILD_CFLAGS += $(call cc-option,-mabi=lp64) ./arch/riscv/Makefile:33: KBUILD_CFLAGS += -mabi=lp64
What I found in kernelnewbies: https://kernelnewbies.org/InternalKernelDataTypes "One thing that has caused a lot of problems, as 64-bit machines are getting more popular, is the fact that the size of a pointer is not the same as the size of an unsigned integer. The size of a pointer is equal to the size of an unsigned long."
I think there should be an explanation both on kernel.org and kernelnewbies what determines the sizes, so that people can have a better idea of what's going on. It should be something that is easy to find and part of "introductions" to kernel development, because it's not obvious at all how all of these things are setup. To me it seems fundamental, to be able to quickly find easy to digest information regarding the data types you're working with.
Sotir, great research! Sadly, https://kernelnewbies.org is giving a "ConfigurationError". :( So I'm not sure if the reference to https://www.kernel.org/doc/ is already in place, or not. Something to check when KN is back up. I'd think pointing to good documents is better than replicating them and risk staleness, isn't it? Leam Linux Software Engineer (reuel.net/career) Scribe: The Domici War (domiciwar.net) Coding Ne'er-do-well (github.com/LeamHall) Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh Practice allows options and foresight. -- lh
On Mon, Jan 27, 2025 at 12:43 AM Leam Hall <leamhall@gmail.com> wrote:
I'd think pointing to good documents is better than replicating them and risk staleness, isn't it?
Yes, but then the issue becomes, what if the documentation you're referencing moves or is removed. People would lose the information completely. I think it's good to point to it as a "source", but still have a basic description. The LP32/LP64 data models seem like one of those things which shouldn't change anytime soon for the kernel anyway. I would like to gather info regarding uintptr_t and other similar types as well. Why they aren't used and have it all compiled into one place.
On 1/27/25 04:39, Sotir Danailov wrote:
On Mon, Jan 27, 2025 at 12:43 AM Leam Hall <leamhall@gmail.com> wrote:
I'd think pointing to good documents is better than replicating them and risk staleness, isn't it?
Yes, but then the issue becomes, what if the documentation you're referencing moves or is removed. People would lose the information completely. I think it's good to point to it as a "source", but still have a basic description. The LP32/LP64 data models seem like one of those things which shouldn't change anytime soon for the kernel anyway.
I would like to gather info regarding uintptr_t and other similar types as well. Why they aren't used and have it all compiled into one place.
My thinking is that a single authoritative source reduces time and change based risk. You really dug into this and found the information, the facts. Those facts can't actually be removed because they are part of the kernel code itself. Well, they can be removed but then they are no longer relevant facts. KernelNewbies is the gentle introduction to working on Linux, but much of its information is dated. Kernel 2.x is referenced often, and we're currently up to 6.x! That's one of the things I want to fix. If you want my opinion, I'll offer it. The research you did was great, and useful for kernel programmers. I feel it is a more advanced topic, something a newbie wouldn't have a significant mental frame of reference for. Here are the sites that seem most relevant: kernel.org - authoritative source of facts - encourage those who have access (which is not me) and responsibility to improve the documentation of those facts tldp.org - In-depth information on Linux facts - - Sadly, it seems to be aging as the latest Guides update is almost ten years old kernelnewbies.org - Help newbies find and use facts Would you consider writing up a guide for either tldp.org or kernel.org? Leam Linux Software Engineer (reuel.net/career) Scribe: The Domici War (domiciwar.net) Coding Ne'er-do-well (github.com/LeamHall) Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh Practice allows options and foresight. -- lh
Kernel 2.x is referenced often, and we're currently up to 6.x! That's one of the things I want to fix.
Yeah this definitely seems more high priority currently. Is there anything outside people can do, to help the effort?
something a newbie wouldn't have a significant mental frame of reference for.
You're probably right, come to think of it now maybe describing all of these details in kernelnewbies isn't appropriate. What my concern with documentation usually is, when it's hard to navigate or when fundamental information is missing or is spread outside of the documentation, it becomes really discouraging to engage with a project, especially for new people. I'll look into what's missing, if anything, at kernel.org regarding data types and attempt to send a patch for the docs there, when I have the time.
tldp.org - In-depth information on Linux facts - - Sadly, it seems to be aging as the latest Guides update is almost ten years old
I didn't even know about this one.
kernelnewbies.org - Help newbies find and use facts
This makes sense, I guess I interpreted it more as a small summary of kernel.org, but helping people find the facts is the more practical solution. There's definitely no way to keep track of everything happening and updating it constantly, now that I think of it.
Hi You are welcome to contribute to https://en.wikibooks.org/wiki/The_Linux_Kernel/Memory#Pointers_and_addresses Thanks Costa
On Tue, Jan 28, 2025 at 9:59 AM Costa Shulyupin <costa.shul@redhat.com> wrote:
You are welcome to contribute to https://en.wikibooks.org/wiki/The_Linux_Kernel/Memory#Pointers_and_addresses
Nice, I wasn't aware of this as well. Thank you!
I'm sorry, I didn't intend for this thread to go beyond the initial email and title! The rest of the thread should've been separate I feel like. Sorry if I filled the mailing list with noise, also wasn't my intention!
On 1/28/25 03:24, Sotir Danailov wrote:
I'm sorry, I didn't intend for this thread to go beyond the initial email and title! The rest of the thread should've been separate I feel like.
Sorry if I filled the mailing list with noise, also wasn't my intention!
I have enjoyed it, and I think it has been educational. Which is a KernelNewbies thing. Leam Linux Software Engineer (reuel.net/career) Scribe: The Domici War (domiciwar.net) Coding Ne'er-do-well (github.com/LeamHall) Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh Practice allows options and foresight. -- lh
On Thu, Jan 23, 2025 at 9:27 AM Sotir Danailov <sndanailov@gmail.com> wrote:
This is a requirement that Linux makes for the compiler to work with it for any platform that Linux runs on.
Ah I see, thank you! Is this documented somewhere? I was searching for information on this, but nothing came up in the kernel docs.
theres a chance that it is implemented as a BUILD_BUG, which would then be kinda self-documenting. These are compiled in assertions, and break the build early. Look at container_of() and marvel at its mechanics.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
participants (5)
-
Costa Shulyupin -
Greg KH -
jim.cromie＠gmail.com -
Leam Hall -
Sotir Danailov