About File systems magic numbers
Hi All, i have been reading Unix filesystem - evolution, design and implementation by Steve D. Pate and it provides a simple file system source code for learning and playing with. On the header file, it defines the fs magic number as #define UX_MAGIC 0x58494e55 1. which means, will it assign this magic number to ALL files created on this sample fs? That can't be so. Otherwise, why we need to define a magic number? 2. I assume, magic number will be assigned to files by the tools which we use to create the files(like vi, cat, etc). Is this correct? 3. It seems, windows files also have the magic numbers. let's say I copy some files from windows to Linux. We should have the common magic numbers between different Operating systems, so there won't be confusions. I mean, is there any standards common for all os's files magic numbers?!? Best regards Sekar
On Wed, Jan 17, 2018 at 8:05 AM, inventsekar <inventsekar@gmail.com> wrote:
Hi All, i have been reading Unix filesystem - evolution, design and implementation by Steve D. Pate
and it provides a simple file system source code for learning and playing with. On the header file, it defines the fs magic number as #define UX_MAGIC 0x58494e55
You need to understand the reason for having the magic number first. Why is it needed and then most of the below questions can be answered.
1. which means, will it assign this magic number to ALL files created on this sample fs? That can't be so. Otherwise, why we need to define a magic number?
There is no rule for it and it totally depends on the will of the filesystem developer. If he wants, he can put a magic number in every file/block etc. even though it might be stupid to do so. Magic numbers are just a way to identify something you already know. They are used during filesystem recovery, to verify that things are ok before mounting etc. etc. If you can't find your known pattern at the place you wrote or where you expect then something is wrong.
2. I assume, magic number will be assigned to files by the tools which we use to create the files(like vi, cat, etc). Is this correct?
Do not assume :-). Look at the code. Tools don't decide what or where to put the magic number. It is the FS code which decides irrespective of tool.
3. It seems, windows files also have the magic numbers. let's say I copy some files from windows to Linux. We should have the common magic numbers between different Operating systems, so there won't be confusions. I mean, is there any standards common for all os's files magic numbers?!?
Nope. Magic numbers are a personal (somewhat) choice. They aren't preserved across filesystems. It would actually be wrong if you can read the magic number of the FS while doing 'cat' on a file. Thanks - Manish
Best regards Sekar
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
n Wed, Jan 17, 2018 at 9:05 AM, inventsekar <inventsekar@gmail.com> wrote:
Hi All, i have been reading Unix filesystem - evolution, design and implementation by Steve D. Pate
and it provides a simple file system source code for learning and playing with. On the header file, it defines the fs magic number as #define UX_MAGIC 0x58494e55
1. which means, will it assign this magic number to ALL files created on this sample fs? That can't be so. Otherwise, why we need to define a magic number?
There can be many different types of filesystems on a Linux computer; the magic number can be used to distinguish which type a particular filesystem is. If you look at usr/include/linux/magic.h you'll see, for example, #define EXT2_SUPER_MAGIC 0xEF53 and as an example of using that particular magic number, check out arch/blackfin/kernel/setup.c or fs/ext2/super.c. -- Jim
On Wed, 17 Jan 2018 21:35:13 +0530, inventsekar said:
On the header file, it defines the fs magic number as #define UX_MAGIC 0x58494e55
1. which means, will it assign this magic number to ALL files created on this sample fs? That can't be so. Otherwise, why we need to define a magic number?
That's not a file magic number, that's a filesystem superblock magic number. That appears just once in the filesystem, usually in/near the superblock. It's only reason for existence is so things like fsck can read in the superblock, and do a: if (superblock.magic != UX_MAGIC) { printf("I see no UX filesystem here!\n); exit(1); }
2. I assume, magic number will be assigned to files by the tools which we use to create the files(like vi, cat, etc). Is this correct?
No, it's scribbled into the superblock by mkfs
3. It seems, windows files also have the magic numbers. let's say I copy some files from windows to Linux. We should have the common magic numbers between different Operating systems, so there won't be confusions. I mean, is there any standards common for all os's files magic numbers?!?
Well, since file magic numbers are different from filesystem magic numbers, and a Windows box is probably never going to be able to mount an ext4 filesystem(footnote *), it's probably mostly irrelevant. And things like tar's magic number, only matter to /usr/bin/tar and /usr/bin/file... See /usr/share/misc/magic (or wherever your system's /bin/file command stashes the file - that's the list of magic numbers that 'file' uses to do stuff like this: % file /bin/bash ./bmap-1.0.20.tar.gz /bin/bash: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=44318cacd823951e3359ae4b5dba06b5383df1f4, stripped ./bmap-1.0.20.tar.gz: gzip compressed data, last modified: Tue Jan 23 17:40:23 2007, from Unix (blast from the past entry in there: ##------------------------------------------------------------------------------ ## $File: alliant,v 1.7 2009/09/19 16:28:07 christos Exp $ ## alliant: file(1) magic for Alliant FX series a.out files ## ## If the FX series is the one that had a processor with a 68K-derived ## instruction set, the "short" should probably become "beshort" and the ## "long" should probably become "belong". ## If it's the i860-based one, they should probably become either the ## big-endian or little-endian versions, depending on the mode they ran ## the 860 in.... ## 0 short 0420 0420 Alliant virtual executable
2 short &0x0020 common library 16 long >0 not stripped 0 short 0421 0421 Alliant compact executable 2 short &0x0020 common library 16 long >0 not stripped
The Alliant was a machine that I worked on at my previous job. I'm coming up on 29 years in my current job, it's unclear whether there are any Alliant a.out files to be found anyplace.. Though the fact that it also knows about gzip files created on TOPS-20 is pretty trippy too.... (footnote *) It's been attempted, and the filesystem gods did heavily smite the programmer for their arrogance. :)
Thanks Manish, Jim.. Thanks Valdis, .. learnt good deal. I will check further regarding the file&tar example and revert back if further queries. Thanks again. Best regards Sekar On Jan 18, 2018 2:38 AM, <valdis.kletnieks@vt.edu> wrote:
On Wed, 17 Jan 2018 21:35:13 +0530, inventsekar said:
On the header file, it defines the fs magic number as #define UX_MAGIC 0x58494e55
1. which means, will it assign this magic number to ALL files created on this sample fs? That can't be so. Otherwise, why we need to define a magic number?
That's not a file magic number, that's a filesystem superblock magic number.
That appears just once in the filesystem, usually in/near the superblock. It's only reason for existence is so things like fsck can read in the superblock, and do a:
if (superblock.magic != UX_MAGIC) { printf("I see no UX filesystem here!\n); exit(1); }
2. I assume, magic number will be assigned to files by the tools which we use to create the files(like vi, cat, etc). Is this correct?
No, it's scribbled into the superblock by mkfs
3. It seems, windows files also have the magic numbers. let's say I copy some files from windows to Linux. We should have the common magic numbers between different Operating systems, so there won't be confusions. I mean, is there any standards common for all os's files magic numbers?!?
Well, since file magic numbers are different from filesystem magic numbers, and a Windows box is probably never going to be able to mount an ext4 filesystem(footnote *), it's probably mostly irrelevant. And things like tar's magic number, only matter to /usr/bin/tar and /usr/bin/file...
See /usr/share/misc/magic (or wherever your system's /bin/file command stashes the file - that's the list of magic numbers that 'file' uses to do stuff like this:
% file /bin/bash ./bmap-1.0.20.tar.gz /bin/bash: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=44318cacd823951e3359ae4b5dba06b5383df1f4, stripped ./bmap-1.0.20.tar.gz: gzip compressed data, last modified: Tue Jan 23 17:40:23 2007, from Unix
(blast from the past entry in there:
##---------------------------------------------------------- -------------------- ## $File: alliant,v 1.7 2009/09/19 16:28:07 christos Exp $ ## alliant: file(1) magic for Alliant FX series a.out files ## ## If the FX series is the one that had a processor with a 68K-derived ## instruction set, the "short" should probably become "beshort" and the ## "long" should probably become "belong". ## If it's the i860-based one, they should probably become either the ## big-endian or little-endian versions, depending on the mode they ran ## the 860 in.... ## 0 short 0420 0420 Alliant virtual executable
2 short &0x0020 common library 16 long >0 not stripped 0 short 0421 0421 Alliant compact executable 2 short &0x0020 common library 16 long >0 not stripped
The Alliant was a machine that I worked on at my previous job. I'm coming up on 29 years in my current job, it's unclear whether there are any Alliant a.out files to be found anyplace..
Though the fact that it also knows about gzip files created on TOPS-20 is pretty trippy too....
(footnote *) It's been attempted, and the filesystem gods did heavily smite the programmer for their arrogance. :)
participants (4)
-
inventsekar -
Jim Davis -
Manish Katiyar -
valdis.kletnieks＠vt.edu