How vmlinux is recognized?
Hi, Sorry if this question is stupid. How the vmlinux (an ELF executable) is recognized by the processor? What are the files that are responsible for this? Thanks, Vikram
Hi Vikram, On Wed, May 11, 2011 at 11:06 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
Hi,
Sorry if this question is stupid. How the vmlinux (an ELF executable) is recognized by the processor? What are the files that are responsible for this?
Well the short answer is that it isn't. The ELF file is normally just one stage of the process. You still need to extract a binary from the ELF, and the binary contains the raw executable code that the processor uses. Normally the boot loader will extract a binary (perhaps from an ELF, or perhaps from a raw binary image) and this is what the processor sees. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com
On Thu, May 12, 2011 at 12:47 AM, Dave Hylands <dhylands@gmail.com> wrote:
Hi Vikram,
On Wed, May 11, 2011 at 11:06 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
Hi,
Sorry if this question is stupid. How the vmlinux (an ELF executable) is recognized by the processor? What are the files that are responsible for this?
Well the short answer is that it isn't.
The ELF file is normally just one stage of the process. You still need to extract a binary from the ELF, and the binary contains the raw executable code that the processor uses.
Normally the boot loader will extract a binary (perhaps from an ELF, or perhaps from a raw binary image) and this is what the processor sees. So in case of x86, say Grub will be taking care of this extraction. Right? If, so the grub code will have the mechanisms for extracting the raw binary from ELF. Am i right?
- Thanks, Vikram
On Thu, May 12, 2011 at 02:31, Vikram Narayanan <vikram186@gmail.com> wrote:
So in case of x86, say Grub will be taking care of this extraction. Right? If, so the grub code will have the mechanisms for extracting the raw binary from ELF. Am i right?
you mean, vmlinuz right? the bzImage right? not the vmlinux.... because that's the one GRUB handles...not vmlinux one... well, in that case, see this first: $ file -k -z /boot/vmlinuz-2.6.32-31-generic /boot/vmlinuz-2.6.32-31-generic: Linux kernel x86 boot executable bzImage, version 2.6.32-31-generic (buildd@rothe, RO-rootFS, root_dev 0x801, swap_dev 0x3, Normal VGA\012- x86 boot sector, code offset 0x5 i am sure you will get idea based upon the above file identification, on what vmlinuz is and how is it supposed to be treated by boot loader.... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
On Thu, May 12, 2011 at 1:15 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Thu, May 12, 2011 at 02:31, Vikram Narayanan <vikram186@gmail.com> wrote:
So in case of x86, say Grub will be taking care of this extraction. Right? If, so the grub code will have the mechanisms for extracting the raw binary from ELF. Am i right?
you mean, vmlinuz right? the bzImage right? not the vmlinux.... because that's the one GRUB handles...not vmlinux one...
The vmlinux is an ELF binary. right? If so, Who does the unpacking of raw binary image from that ELF?
well, in that case, see this first: $ file -k -z /boot/vmlinuz-2.6.32-31-generic
This is the compressed one. The uncompression is done by the kernel and not by the grub, If I am not wrong.
/boot/vmlinuz-2.6.32-31-generic: Linux kernel x86 boot executable bzImage, version 2.6.32-31-generic (buildd@rothe, RO-rootFS, root_dev 0x801, swap_dev 0x3, Normal VGA\012- x86 boot sector, code offset 0x5
i am sure you will get idea based upon the above file identification, on what vmlinuz is and how is it supposed to be treated by boot loader.... I am still confused :(
Thanks, Vikram
On Thu, May 12, 2011 at 03:04, Vikram Narayanan <vikram186@gmail.com> wrote:
The vmlinux is an ELF binary. right? If so, Who does the unpacking of raw binary image from that ELF?
why do you put concern on vmlinux anyway? boot loader loads vmlinuz, not vmlinux.... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
On Thu, May 12, 2011 at 1:39 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Thu, May 12, 2011 at 03:04, Vikram Narayanan <vikram186@gmail.com> wrote:
The vmlinux is an ELF binary. right? If so, Who does the unpacking of raw binary image from that ELF?
why do you put concern on vmlinux anyway? boot loader loads vmlinuz, not vmlinux.... Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done?
Thanks, Vikram
On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done?
OK I try my best to understand your question :) i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both. After final phase of final kernel image creation, it will go into making bootable image first. in order to do that, first it will be compressed 1st. These days, gz is the choice. So, it is gzipped..and the boot loading code is appended in front of it... there, you get vmlinuz. And vmlinux? developers usually use vmlinux as symbol file... and the way it is created, back to the above phase, is by linking it according to the accompanying elf linker script. Finally, ELF that contains kernel is there. Another guess, maybe you wanna know how to extract the kernel code from ELF image? then why so? that is indeed the kernel image itself...it is just appended ELF headers, sections and so on just to represent ELF construction. But it is not behaving like standart ELF binary i.e the entry point is not main() but IIRC start_kernel or something like that. that helps you? -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done?
OK I try my best to understand your question :)
i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both.
I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) 7) Coming back to the vmlinux image, Who takes care of the loading activity.? 8) Who recognizes that the image is ELF format and do the necessary things accordingly.? Hope I have my question clear now. - Thanks, Vikram
Hi Vikram, On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done?
OK I try my best to understand your question :)
i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both.
I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) 7) Coming back to the vmlinux image, Who takes care of the loading activity.? 8) Who recognizes that the image is ELF format and do the necessary things accordingly.?
Hope I have my question clear now.
If understand your question correctly, you believe that the uncompressed kernel is in elf format. correct?. it is in binary format, so elf interpretation is not required, #5 is wrong. You can see this by building the kernel using 'make V=1' and note the following line in the output, "arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage" means bzImage is made out of two binary files extracted from the elf images. -- Thanks Sudheer
On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran <inbox1.sudheer@gmail.com> wrote:
Hi Vikram,
On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done?
OK I try my best to understand your question :)
i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both.
I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) 7) Coming back to the vmlinux image, Who takes care of the loading activity.? 8) Who recognizes that the image is ELF format and do the necessary things accordingly.?
Hope I have my question clear now.
If understand your question correctly, you believe that the uncompressed kernel is in elf format. correct?. it is in binary format, so elf interpretation is not required, #5 is wrong.
You can see this by building the kernel using 'make V=1' and note the following line in the output,
"arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage"
means bzImage is made out of two binary files extracted from the elf images.
One more info I want to clarify is, vmlinux.bin mentioned in the above snippet contains the compressed binary image and some other routines. Just go through the 'make V=1' output, you can see that the build process is actually compressing binary file extracted from the vmlinux elf image, which is again combined with some object files, creates another elf and again extracts the binary and finally combined with the setup.bin to create the final bzImage. So, elf interpretation doesn't happen on the uncompressed code. -- Thanks Sudheer
On 12/05/2011 8.21, Sudheer Divakaran wrote:
On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran <inbox1.sudheer@gmail.com> wrote:
Hi Vikram,
On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan<vikram186@gmail.com> wrote:
On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Thu, May 12, 2011 at 03:11, Vikram Narayanan<vikram186@gmail.com> wrote:
Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done?
OK I try my best to understand your question :)
i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both.
I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) 7) Coming back to the vmlinux image, Who takes care of the loading activity.? 8) Who recognizes that the image is ELF format and do the necessary things accordingly.?
Hope I have my question clear now.
If understand your question correctly, you believe that the uncompressed kernel is in elf format. correct?. it is in binary format, so elf interpretation is not required, #5 is wrong.
You can see this by building the kernel using 'make V=1' and note the following line in the output,
"arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT> arch/x86/boot/bzImage"
means bzImage is made out of two binary files extracted from the elf images.
One more info I want to clarify is, vmlinux.bin mentioned in the above snippet contains the compressed binary image and some other routines. Just go through the 'make V=1' output, you can see that the build process is actually compressing binary file extracted from the vmlinux elf image, which is again combined with some object files, creates another elf and again extracts the binary and finally combined with the setup.bin to create the final bzImage. So, elf interpretation doesn't happen on the uncompressed code.
Let's put some order here. The image that almost all bootloaders use is arch/x86/boot/bzImage which is made of a setup binary file (executable) joined with some compressed code (the real kernel) which is uncompressed in memory by the setup binary. The big suggestion I can give is to check the hidden files which end with .cmd. There is one of these for every object created by the compilation process. For example there is a file called .bzImage.cmd which tell you how bzImage was made: arch/x86/boot/tools/build -b arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage NOTE: I refer to a quite old kernel here, it's likely that the compilation process has changed somehow. You can proceed now in reverse order to find how bzImage was made (if I understand correctly that is the one you are interested in). Here is how is made on my kernel tree: bzImage: arch/x86/boot/tools/build -b arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage arch/x86/boot/vmlinux.bin: objcopy -O binary -R .note -R .comment -S arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin arch/x86/boot/compressed/vmlinux: ld -m elf_i386 -T arch/x86/boot/compressed/vmlinux_32.lds arch/x86/boot/compressed/head_32.o arch/x86/boot/compressed/misc.o arch/x86/boot/compressed/piggy.o -o arch/x86/boot/compressed/vmlinux arch/x86/boot/compressed/piggy.o: ld -m elf_i386 -r --format binary --oformat elf32-i386 -T arch/x86/boot/compressed/vmlinux.scr arch/x86/boot/compressed/vmlinux.bin.gz -o arch/x86/boot/compressed/piggy.o arch/x86/boot/compressed/vmlinux.bin.gz: gzip -f -9 < arch/x86/boot/compressed/vmlinux.bin > arch/x86/boot/compressed/vmlinux.bin.gz arch/x86/boot/compressed/vmlinux.bin: objcopy -O binary -R .note -R .comment -S vmlinux arch/x86/boot/compressed/vmlinux.bin vmlinux: ld -m elf_i386 --build-id -o vmlinux -T arch/x86/kernel/vmlinux.lds arch/x86/kernel/head_32.o arch/x86/kernel/init_task.o init/built-in.o --start-group usr/built-in.o arch/x86/kernel/built-in.o arch/x86/mm/built-in.o arch/x86/mach-default/built-in.o arch/x86/crypto/built-in.o arch/x86/vdso/built-in.o kernel/built-in.o mm/built-in.o fs/built-in.o ipc/built-in.o security/built-in.o crypto/built-in.o block/built-in.o lib/lib.a arch/x86/lib/lib.a lib/built-in.o arch/x86/lib/built-in.o drivers/built-in.o sound/built-in.o arch/x86/math-emu/built-in.o arch/x86/pci/built-in.o arch/x86/power/built-in.o net/built-in.o --end-group .tmp_kallsyms2.o NOTE: there are 2 vmlinux (vmlinux and arch\x86\boot\compressed\vmlinux) and 2 vmlinux.bin (arch\x86\boot\compressed\vmlinux.bin and arch\x86\boot\vmlinux.bin) which are not the same Hope this helps Regards Luca Ellero
Hi... On Thu, May 12, 2011 at 10:32, Vikram Narayanan <vikram186@gmail.com> wrote:
I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right?
nope... it's a binary....but not ELF...and that's not even named vmlinux or similar to vmlinux...
5) The vmlinux is in ELF format. Correct? yes.... but see above...
-- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
On Thu, May 12, 2011 at 1:47 PM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
Hi...
On Thu, May 12, 2011 at 10:32, Vikram Narayanan <vikram186@gmail.com> wrote:
I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right?
nope... it's a binary....but not ELF...and that's not even named vmlinux or similar to vmlinux...
5) The vmlinux is in ELF format. Correct? yes.... but see above...
Thanks for all your explanations. So the uncompressed one is _NOT_ an ELF file, but a raw binary. So it doesn't need any interpretation. Hope this is right. So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped Thanks, Vikram
Hi Vikram, ...snip...
So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped
Take luca's email and start at the bottom working towards the top. linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together to produce arch/x86/boot/compressed/vmlinux (an ELF file). objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage. So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel. GRUB loads bzImage into memory and decompresses it and then executes the resulting binary. This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com
On Fri, May 13, 2011 at 7:12 AM, Dave Hylands <dhylands@gmail.com> wrote:
Hi Vikram,
...snip...
So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped
Take luca's email and start at the bottom working towards the top.
linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together to produce arch/x86/boot/compressed/vmlinux (an ELF file). objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage.
So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel.
GRUB loads bzImage into memory and decompresses it and then executes the resulting binary. This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running.
Thanks for the detailed explanation. Clarified. :) - Thanks Vikram
I loved this reply.......can I annotate it with references to the linux kernel sources? On Fri, May 13, 2011 at 9:42 AM, Dave Hylands <dhylands@gmail.com> wrote:
Hi Vikram,
...snip...
So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped
Take luca's email and start at the bottom working towards the top.
linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin
yes: See ./arch/x86/boot/Makefile
This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz
See ./arch/x86/boot/compressed/Makefile
This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file
./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary - mkpiggy which will generate the piggy.o.
called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together
Yes, the routine is called "decompress_kernel", residing inside ./arch/x86/boot/compressed/misc.c. And this routine is called from ./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the gzipped data is decompressed and immediately jumped into (perhaps after some relocation if needed): /* * Do the decompression, and jump to the new kernel.. */ leal z_extract_offset_negative(%ebx), %ebp /* push arguments for decompress_kernel: */ pushl %ebp /* output address */ pushl $z_input_len /* input_len */ leal input_data(%ebx), %eax pushl %eax /* input_data */ leal boot_heap(%ebx), %eax pushl %eax /* heap area */ pushl %esi /* real mode pointer */ call decompress_kernel addl $20, %esp to produce arch/x86/boot/compressed/vmlinux (an ELF file).
objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage.
Inside arch/x86/boot/Makefile: Creating the vmlinux.bin from vmlinux via objcopy (note that this operation will throw all relocation information): $(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE $(call if_changed,objcopy) And then packing together linearly to form the "bzImage" (output from make): make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage make -f scripts/Makefile.build obj=arch/x86/boot/compressed arch/x86/boot/compressed/vmlinux arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage
So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel.
GRUB loads bzImage into memory and decompresses it and then executes the resulting binary.
To be more precise, grub will load bzImage and jump into the startup_32 function located in arch/x86/boot/compressed/head_32.S at the following fixed address (from source code): /* * head.S contains the 32-bit startup code. * * NOTE!!! Startup happens at absolute address 0x00001000, which is also where * the page directory will exist. The startup code will be overwritten by * the page directory. [According to comments etc elsewhere on a compressed * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC] * * Page 0 is deliberately kept safe, since System Management Mode code in * laptops may need to access the BIOS data stored there. This is also * useful for future device drivers that either access the BIOS via VM86 * mode. */ More info: http://books.google.com/books?id=e8BbHxVhzFAC&pg=PA1224&lpg=PA1224&dq=grub+h...
This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running.
-- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com
_______________________________________________ K
-- Regards, Peter Teoh
On Mon, May 16, 2011 at 9:14 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
I loved this reply.......can I annotate it with references to the linux kernel sources?
On Fri, May 13, 2011 at 9:42 AM, Dave Hylands <dhylands@gmail.com> wrote:
Hi Vikram,
...snip...
So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped
Take luca's email and start at the bottom working towards the top.
linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin
yes: See ./arch/x86/boot/Makefile
This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz
See ./arch/x86/boot/compressed/Makefile
This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file
./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary - mkpiggy which will generate the piggy.o.
called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together
Yes, the routine is called "decompress_kernel", residing inside ./arch/x86/boot/compressed/misc.c. And this routine is called from ./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the gzipped data is decompressed and immediately jumped into (perhaps after some relocation if needed): /* * Do the decompression, and jump to the new kernel.. */ leal z_extract_offset_negative(%ebx), %ebp /* push arguments for decompress_kernel: */ pushl %ebp /* output address */ pushl $z_input_len /* input_len */ leal input_data(%ebx), %eax pushl %eax /* input_data */ leal boot_heap(%ebx), %eax pushl %eax /* heap area */ pushl %esi /* real mode pointer */ call decompress_kernel addl $20, %esp
to produce arch/x86/boot/compressed/vmlinux (an ELF file). objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage.
Inside arch/x86/boot/Makefile: Creating the vmlinux.bin from vmlinux via objcopy (note that this operation will throw all relocation information): $(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE $(call if_changed,objcopy) And then packing together linearly to form the "bzImage" (output from make): make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage make -f scripts/Makefile.build obj=arch/x86/boot/compressed arch/x86/boot/compressed/vmlinux arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage
So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel.
GRUB loads bzImage into memory and decompresses it and then executes the resulting binary.
To be more precise, grub will load bzImage and jump into the startup_32 function located in arch/x86/boot/compressed/head_32.S at the following fixed address (from source code): /* * head.S contains the 32-bit startup code. * * NOTE!!! Startup happens at absolute address 0x00001000, which is also where * the page directory will exist. The startup code will be overwritten by * the page directory. [According to comments etc elsewhere on a compressed * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC] * * Page 0 is deliberately kept safe, since System Management Mode code in * laptops may need to access the BIOS data stored there. This is also * useful for future device drivers that either access the BIOS via VM86 * mode. */ More info: http://books.google.com/books?id=e8BbHxVhzFAC&pg=PA1224&lpg=PA1224&dq=grub+h...
This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running.
-- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com
_______________________________________________ K
-- Regards, Peter Teoh
That was a great explanation. Thanks a lot. I think this will be very much useful for people who want to know how things work in the background. - Thanks, Vikram
Hi All On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan <vikram186@gmail.com>wrote:
On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done?
OK I try my best to understand your question :)
i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both.
I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct?
I Guess Yes. 6) If the OS boots and if u try to run an ELF file, the loader knows
how to load that in the RAM. (I mean it knows how to interpret the ELF format)
See the multi-boot specification. GRUB is a multi-boot compliant boot loader
7) Coming back to the vmlinux image, Who takes care of the loading activity.?
GRUB 8) Who recognizes that the image is ELF format and do the necessary
things accordingly.?
GRUB
Hope I have my question clear now.
- Thanks, Vikram
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Anuj Aggarwal .''`. : :Ⓐ : # apt-get install hakuna-matata `. `'` `-
Hi Vikram, How the vmlinux (an ELF executable) is recognized by the processor?
ELF is just a file format. That is, the machine instructions and data are stored in a specific format. The _processor_ simply recognizes machine instructions and this needs to be taken from the ELF file and loaded into memory (the instruction pointer is then pointed to the place the instructions were loaded). The format is simply a set of rules defined in the specification (a pretty nice introduction is available at www.skyfree.org/linux/references/*ELF* _Format.pdf <http://www.skyfree.org/linux/references/ELF_Format.pdf>). For example, when you ask a Linux kernel to execute an ELF file, it has code to know how to decode the information and place it into memory (see fs/binfmt_elf.c). As for the vmlinux file specifically, the Wikipedia page on vmlinux ( http://en.wikipedia.org/wiki/Vmlinux) seems like a good start. As shown above with Linux, GRUB needs to have a way to decode whatever format is passed to it (bzImage). The kernel however places the unzipping code into the bzImage itself so that it is loaded into memory by the bootloader and is then run. This code then unzips the kernel. This way, GRUB doesn't need to know how to decode ELF files and the job is left to the kernel code. You can see arch/x86/boot/Makefile and look for the bzImage target to see what files constitute the bzImage. I may be wrong about this with regard to newer kernels so I hope others correct me in this case. Another great explanation is by Alessandro Rubini at: http://www.ibiblio.org/oswg/oswg-nightly/oswg/en_US.ISO_8859-1/articles/ales... Hope this helped! :-) -- /manohar
On Thu, May 12, 2011 at 2:03 AM, Manohar Vanga <manohar.vanga@gmail.com> wrote:
Hi Vikram,
How the vmlinux (an ELF executable) is recognized by the processor?
ELF is just a file format. That is, the machine instructions and data are stored in a specific format. The _processor_ simply recognizes machine instructions and this needs to be taken from the ELF file and loaded into memory (the instruction pointer is then pointed to the place the instructions were loaded).
Hope everyone here got my question wrong. I am aware that ELF is a format and there will be specific loader for loading ELF files. Please refer to the previous reply. Thanks, Vikram
This way, GRUB doesn't need to know how to decode ELF files and the job is left to the kernel code.
GRUB has a elf decoder, but it should have multiboot header. http://osdev.berlios.de/grub.html#multiboot Hope this answers your doubt.
participants (9)
-
Dave Hylands -
luca ellero -
Manohar Vanga -
mindentropy -
Mulyadi Santosa -
Peter Teoh -
Sudheer Divakaran -
Vikram Narayanan -
अनुज