MTD: How to get actual image size from MTD partition
Hi, Our ARM32 Linux embedded system consists of these: * Linux Kernel: 4.14 * Processor: Qualcomm Arm32 Cortex-A7 * Storage: NAND 512MB * Platform: Simple busybox * Filesystem: UBIFS, Squashfs * Consists of nand raw partitions, squashfs ubi volumes. My requirement: To find the checksum of a real image in runtime which is flashed in an MTD partition. Problem: Currently, to find the checksum, we are using: $ md5sum /dev/mtd14 This returns the proper checksum of the entire partition. But we wanted to find the checksum only for the actual image data which will be used by our C utility to validate the image. Here, we don't know the actual image size. We only know the "partition-size" and "erasesize". So, is there a mechanism to somehow find the image size at runtime? Regards, Pintu
Pintu, ----- Ursprüngliche Mail -----
Von: "Pintu Agarwal" <pintu.ping@gmail.com> My requirement: To find the checksum of a real image in runtime which is flashed in an MTD partition.
Problem: Currently, to find the checksum, we are using: $ md5sum /dev/mtd14 This returns the proper checksum of the entire partition. But we wanted to find the checksum only for the actual image data which will be used by our C utility to validate the image. Here, we don't know the actual image size. We only know the "partition-size" and "erasesize".
So, is there a mechanism to somehow find the image size at runtime?
not really, UBI manages the MTD and does wearleveling, auto growing of volumes, etc... So as soon you attach the image once, it is changed and the checksum won't match. It may work if you don't attach UBI and your flash program tool keeps track of what pages it wrote. Thanks, //richard
On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
Hi,
Our ARM32 Linux embedded system consists of these: * Linux Kernel: 4.14 * Processor: Qualcomm Arm32 Cortex-A7 * Storage: NAND 512MB * Platform: Simple busybox * Filesystem: UBIFS, Squashfs * Consists of nand raw partitions, squashfs ubi volumes.
My requirement: To find the checksum of a real image in runtime which is flashed in an MTD partition.
Try using the dm-verity module for ensuring that a block device really is properly signed before mounting it. That's what it was designed for and is independent of the block device type. good luck! greg k-h
----- Ursprüngliche Mail -----
Von: "Greg KH" <greg@kroah.com> An: "Pintu Agarwal" <pintu.ping@gmail.com> CC: "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher" <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org>, "richard" <richard@nod.at> Gesendet: Freitag, 16. Juli 2021 17:41:01 Betreff: Re: MTD: How to get actual image size from MTD partition
On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
Hi,
Our ARM32 Linux embedded system consists of these: * Linux Kernel: 4.14 * Processor: Qualcomm Arm32 Cortex-A7 * Storage: NAND 512MB * Platform: Simple busybox * Filesystem: UBIFS, Squashfs * Consists of nand raw partitions, squashfs ubi volumes.
My requirement: To find the checksum of a real image in runtime which is flashed in an MTD partition.
Try using the dm-verity module for ensuring that a block device really is properly signed before mounting it. That's what it was designed for and is independent of the block device type.
MTDs are not block devices. :-) Thanks, //richard
On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
My requirement: To find the checksum of a real image in runtime which is flashed in an MTD partition.
Try using the dm-verity module for ensuring that a block device really is properly signed before mounting it. That's what it was designed for and is independent of the block device type.
MTDs are not block devices. :-)
Is it possible to use dm-verity with squashfs ? We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44
----- Ursprüngliche Mail -----
Von: "Pintu Agarwal" <pintu.ping@gmail.com> An: "richard" <richard@nod.at> CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher" <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org> Gesendet: Montag, 19. Juli 2021 11:09:46 Betreff: Re: MTD: How to get actual image size from MTD partition
On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
My requirement: To find the checksum of a real image in runtime which is flashed in an MTD partition.
Try using the dm-verity module for ensuring that a block device really is properly signed before mounting it. That's what it was designed for and is independent of the block device type.
MTDs are not block devices. :-)
Is it possible to use dm-verity with squashfs ? We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44
Well, if you emulate a block device using mtdblock, you can use dm-verity and friends. Also consider using ubiblock. It offers better performance and wear leveling support. Thanks, //richard
On Mon, 19 Jul 2021 at 14:58, Richard Weinberger <richard@nod.at> wrote:
----- Ursprüngliche Mail -----
Von: "Pintu Agarwal" <pintu.ping@gmail.com> An: "richard" <richard@nod.at> CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher" <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org> Gesendet: Montag, 19. Juli 2021 11:09:46 Betreff: Re: MTD: How to get actual image size from MTD partition
On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
My requirement: To find the checksum of a real image in runtime which is flashed in an MTD partition.
Try using the dm-verity module for ensuring that a block device really is properly signed before mounting it. That's what it was designed for and is independent of the block device type.
MTDs are not block devices. :-)
Is it possible to use dm-verity with squashfs ? We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44
Well, if you emulate a block device using mtdblock, you can use dm-verity and friends. Also consider using ubiblock. It offers better performance and wear leveling support.
Okay thank you. We have tried dm-verity with squashfs (for our rootfs) but we are facing some mounting issues. [...] [ 4.697757] device-mapper: init: adding target '0 96160 verity 1 /dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256 d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681 aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7' [ 4.704771] device-mapper: verity: sha256 using implementation "sha256-generic" [...] [ 4.727366] device-mapper: init: dm-0 is ready [ 4.912558] VFS: Cannot open root device "dm-0" or unknown-block(253,0): error -5 The same works with ext4 emulation. So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ? Anyways, I will create a separate thread for dm-verity issue and keep this thread still open for UBI image size issue. We may use dm-verify for rootfs during booting, but still we need to perform integrity check for other nand partitions and UBI volumes. So, instead of calculating the checksum for the entire partition, is it possible to perform checksum only based on the image size ? Right now, we are still exploring what are the best possible mechanisms available for this. Thanks, Pintu
----- Ursprüngliche Mail -----
Von: "Pintu Agarwal" <pintu.ping@gmail.com> Okay thank you. We have tried dm-verity with squashfs (for our rootfs) but we are facing some mounting issues. [...] [ 4.697757] device-mapper: init: adding target '0 96160 verity 1 /dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256 d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681 aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7' [ 4.704771] device-mapper: verity: sha256 using implementation "sha256-generic" [...] [ 4.727366] device-mapper: init: dm-0 is ready [ 4.912558] VFS: Cannot open root device "dm-0" or unknown-block(253,0): error -5
The same works with ext4 emulation. So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ?
I don't know.
Anyways, I will create a separate thread for dm-verity issue and keep this thread still open for UBI image size issue. We may use dm-verify for rootfs during booting, but still we need to perform integrity check for other nand partitions and UBI volumes.
So, instead of calculating the checksum for the entire partition, is it possible to perform checksum only based on the image size ? Right now, we are still exploring what are the best possible mechanisms available for this.
I still don't fully understand what you are trying to achieve. Is it about cryptographic integrity of your storage or detecting errors after the flashing process? But let me advertise ubiblock a second time. If you place your squashfs on a UBI static volume, UBI knows the exact length and you can checksum it more easily. Thanks, //richard
On Tue, 20 Jul 2021 at 12:10, Richard Weinberger <richard@nod.at> wrote:
Anyways, I will create a separate thread for dm-verity issue and keep this thread still open for UBI image size issue. We may use dm-verify for rootfs during booting, but still we need to perform integrity check for other nand partitions and UBI volumes.
So, instead of calculating the checksum for the entire partition, is it possible to perform checksum only based on the image size ? Right now, we are still exploring what are the best possible mechanisms available for this.
I still don't fully understand what you are trying to achieve. Is it about cryptographic integrity of your storage or detecting errors after the flashing process?
Yes, it is about md5 checksum verification for every partition to check its integrity before updates.
But let me advertise ubiblock a second time. Sorry, I could not understand about the ubiblock request. Is it possible to elaborate little more ? We are already using squashfs on top of our UBI volumes (including rootfs mounting). This is the kernel command line we pass: rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30 And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel. Do we need to do something different for ubiblock ?
If you place your squashfs on a UBI static volume, UBI knows the exact length and you can checksum it more easily. Yes, we use squashfs on UBI volumes, but our volume type is still dynamic. Also, you said, UBI knows the exact length, you mean the whole image length ? How can we get this length at runtime ? Also, how can we get the checksum of the entire UBI volume content (ignoring the erased/empty/bad block content) ?
Or, you mean to say, the whole checksum logic is in-built inside the UBI layer and users don't need to worry about the integrity at all ? Thanks, Pintu
----- Ursprüngliche Mail -----
But let me advertise ubiblock a second time. Sorry, I could not understand about the ubiblock request. Is it possible to elaborate little more ? We are already using squashfs on top of our UBI volumes (including rootfs mounting). This is the kernel command line we pass: rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30 And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel. Do we need to do something different for ubiblock ?
From that command line I understand that you are *not* using squashfs on top of UBI. You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only block device. See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
If you place your squashfs on a UBI static volume, UBI knows the exact length and you can checksum it more easily. Yes, we use squashfs on UBI volumes, but our volume type is still dynamic. Also, you said, UBI knows the exact length, you mean the whole image length ? How can we get this length at runtime ?
You need a static volume for that. If you update a static volume the length is known by UBI.
Also, how can we get the checksum of the entire UBI volume content (ignoring the erased/empty/bad block content) ?
Just read from the volume. /dev/ubiX_Y.
Or, you mean to say, the whole checksum logic is in-built inside the UBI layer and users don't need to worry about the integrity at all ?
Static volumes have a crc32 checksum over the whole content. Of course this offers no cryptographic integrity. See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_overview Thanks, //richard
On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
----- Ursprüngliche Mail -----
But let me advertise ubiblock a second time. Sorry, I could not understand about the ubiblock request. Is it possible to elaborate little more ? We are already using squashfs on top of our UBI volumes (including rootfs mounting). This is the kernel command line we pass: rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30 And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel. Do we need to do something different for ubiblock ?
From that command line I understand that you are *not* using squashfs on top of UBI. You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only block device. See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
Okay, you mean to say, we should use this ? ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0 Instead of this: root=/dev/mtdblock44 ubi.mtd=40,0,30 Okay I will discuss this internally and check..
If you place your squashfs on a UBI static volume, UBI knows the exact length and you can checksum it more easily. Yes, we use squashfs on UBI volumes, but our volume type is still dynamic. Also, you said, UBI knows the exact length, you mean the whole image length ? How can we get this length at runtime ?
You need a static volume for that. If you update a static volume the length is known by UBI.
Thank you so much for your reply! Sorry, I could not get this part. How static volume can give image len ? You mean there is some interface available in kernel to get actual image len ?
Also, how can we get the checksum of the entire UBI volume content (ignoring the erased/empty/bad block content) ?
Just read from the volume. /dev/ubiX_Y.
I think this also will give the entire volume size, but we still don't know how many pages have real data ? For example: Suppose, my raw partition/volume is of size 10MB But my actual data inside it is of size ~3MB (may be split across?) Then, how can we get the actual size of the data content ? You mean to say: /dev/ubiX_Y should contain only data blocks ?
Or, you mean to say, the whole checksum logic is in-built inside the UBI layer and users don't need to worry about the integrity at all ?
On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
----- Ursprüngliche Mail -----
But let me advertise ubiblock a second time. Sorry, I could not understand about the ubiblock request. Is it possible to elaborate little more ? We are already using squashfs on top of our UBI volumes (including rootfs mounting). This is the kernel command line we pass: rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30 And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel. Do we need to do something different for ubiblock ?
From that command line I understand that you are *not* using squashfs on top of UBI. You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only block device. See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
Okay, you mean to say, we should use this ? ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0 Instead of this: root=/dev/mtdblock44 ubi.mtd=40,0,30
Yes. But it is not only about a different command line. It is a different concept. You use a emulated block device on top of UBI, and not directly on top of an MTD part.
Sorry, I could not get this part. How static volume can give image len ? You mean there is some interface available in kernel to get actual image len ?
use the ubinfo tool. Static volumes know exactly how much they are filled.
Also, how can we get the checksum of the entire UBI volume content (ignoring the erased/empty/bad block content) ?
Just read from the volume. /dev/ubiX_Y.
I think this also will give the entire volume size, but we still don't know how many pages have real data ?
"ubiinfo /dev/ubiX_Y" will tell you if the volume is of type static.
For example: Suppose, my raw partition/volume is of size 10MB But my actual data inside it is of size ~3MB (may be split across?) Then, how can we get the actual size of the data content ?
See above.
You mean to say: /dev/ubiX_Y should contain only data blocks ?
Yes. An UBI volume contains only "user data". -- Thanks, //richard
Hi Richard, On Tue, 27 Jul 2021 at 18:16, Richard Weinberger <richard.weinberger@gmail.com> wrote:
On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
----- Ursprüngliche Mail -----
But let me advertise ubiblock a second time. Sorry, I could not understand about the ubiblock request. Is it possible to elaborate little more ? We are already using squashfs on top of our UBI volumes (including rootfs mounting). This is the kernel command line we pass: rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30 And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel. Do we need to do something different for ubiblock ?
From that command line I understand that you are *not* using squashfs on top of UBI. You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only block device. See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
[snip] Ouch, so surprised that after all these years someone is doing squashfs/mtdblock instead of using ubiblock :-) Can we patch either Kconfig or add some warn_once on mtdblock usage, suggesting to use ubiblock instead? I remember there was still some use case(s) for mtdblock but I can't remember now what was it, perhaps we should document the expectations? (Is that for JFFS2 to mount?) Thanks, Ezequiel
participants (5)
-
Ezequiel Garcia -
Greg KH -
Pintu Agarwal -
Richard Weinberger -
Richard Weinberger