Re: MTD: How to get actual image size from MTD partition
On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
Ezequiel,
----- Ursprüngliche Mail -----
[snip]
Ouch, so surprised that after all these years someone is doing squashfs/mtdblock instead of using ubiblock :-)
Can we patch either Kconfig or add some warn_once on mtdblock usage, suggesting to use ubiblock instead?
a hint in Kconfig makes IMHO sense. Do you want to send a patch? A warning is too much since on some tiny embedded system with NOR flash mtdblock is still a good choice. ubiblock is mostly useful for NAND flash.
I remember there was still some use case(s) for mtdblock but I can't remember now what was it, perhaps we should document the expectations? (Is that for JFFS2 to mount?)
a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount JFFS2. This limitation is gone.
OK, let me try to cook a patch for you.
Dear Eze and Richard, First of all, thank you so much for all your replies so far. Sorry, I have limited knowledge about NAND, MTD, UBI layers, but my current work involves all these so I am here. But I will surely share this information to our internal team about using ubiblock instead of mtdblock. However, I still fail to understand the problem and consequences of using mtdblock for rootfs instead of ubiblock. Last time, for my squashfs test, when I tried to replace the command line with ubiblock, I could not see any difference. How to visibly see the difference so that I can easily understand and explain the difference internally? Or, is there a document available somewhere to highlight the difference between the two? BTW, we have few raw nand partitions and few ubi volumes [including rootfs(squashfs), data(ubifs, rw)] So, I guess we should use ubiblock for all ubi volumes? Regarding Kconfig, I have few opinions. Yes, adding more description in Kconfig is a good choice. But I see that most of the time these kernel config options remains (as default) and platform developers never cares about the description. So, how to better create awareness among them, not to make these mistakes? One option is to capture these details as part of Kernel documentation. Right now when I search I could not find anything. Another option is to add a few (critical) warnings in kernel bootup logs to give some hint to the developer that there is probably something wrong. This will directly be visible to all and some developers will not like to ignore it. Or, maybe adding both options is also good. Thanks, Pintu
On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
Ezequiel,
----- Ursprüngliche Mail -----
[snip]
Ouch, so surprised that after all these years someone is doing squashfs/mtdblock instead of using ubiblock :-)
Can we patch either Kconfig or add some warn_once on mtdblock usage, suggesting to use ubiblock instead?
a hint in Kconfig makes IMHO sense. Do you want to send a patch? A warning is too much since on some tiny embedded system with NOR flash mtdblock is still a good choice. ubiblock is mostly useful for NAND flash.
I remember there was still some use case(s) for mtdblock but I can't remember now what was it, perhaps we should document the expectations? (Is that for JFFS2 to mount?)
a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount JFFS2. This limitation is gone.
Hi, Just a further follow-up on this discussion. Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs) mounting during boot. As suggested here: Instead of using this in kernel command line: [ 0.000000] Kernel command line: ... rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30 ... I used this: [ 0.000000] Kernel command line: ... rootfstype=squashfs ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ... The device is booting fine with ubiblock as well. But, per say, I could not find any visible difference. I just observed a slight improvement in boot time, but I need to double-check on this, with few more reboot cycles. Apart from this what are the other visible benefits of using ubiblock which can be explained to be management or internal team ? I could not find any documentation explaining the difference, except this one: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock Can someone also point me to the respective driver code in case of using /dev/mtdblock and /dev/ubiblock ? Apart from theory I also want to check the impact at the code level.. Thanks, Pintu
Hi Pintu, On Fri, 20 Aug 2021 at 15:25, Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
Ezequiel,
----- Ursprüngliche Mail -----
[snip]
Ouch, so surprised that after all these years someone is doing squashfs/mtdblock instead of using ubiblock :-)
Can we patch either Kconfig or add some warn_once on mtdblock usage, suggesting to use ubiblock instead?
a hint in Kconfig makes IMHO sense. Do you want to send a patch? A warning is too much since on some tiny embedded system with NOR flash mtdblock is still a good choice. ubiblock is mostly useful for NAND flash.
I remember there was still some use case(s) for mtdblock but I can't remember now what was it, perhaps we should document the expectations? (Is that for JFFS2 to mount?)
a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount JFFS2. This limitation is gone.
Hi,
Just a further follow-up on this discussion. Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs) mounting during boot.
As suggested here: Instead of using this in kernel command line: [ 0.000000] Kernel command line: ... rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30 ...
I used this: [ 0.000000] Kernel command line: ... rootfstype=squashfs ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ...
The device is booting fine with ubiblock as well. But, per say, I could not find any visible difference. I just observed a slight improvement in boot time, but I need to double-check on this, with few more reboot cycles.
That's a very good thing, it means we offered you a smooth transition :-)
Apart from this what are the other visible benefits of using ubiblock which can be explained to be management or internal team ? I could not find any documentation explaining the difference, except this one: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
I'm not a flash expert here. In any case, you are expected to do your own research (just like we all did), design your own setup matching your use-case, design tests based on your workload and access patterns, etc. There are presentations on YouTube which discuss UBI, UBIFS and NAND-based designs on Linux, as well as white papers discussing NAND flashes challenges. Having said that... When you use UBI block, you are accessing the flash via the UBI layer. This is IMO the best way to design your system, since UBI addresses wear leveling and bad blocks, and offers atomic updates. In other words, IMO it's best to expose the NAND through UBI for both read-only and read-write access, using a single UBI device, and then creating UBI volumes as needed. This will allow UBI to spread wear leveling across the whole device, which is expected to increase the flash lifetime. For instance, just as some silly example, you could have something like this: | RootFS SquashFS | | UBI block | UBIFS User R-W area ------------------------------------------------------------------------ Kernel A | Kernel B | RootFS A | RootFS B | User ------------------------------------------------------------------------ UBIX ------------------------------------------------------------------------ /dev/mtdX This setup allows safe kernel and rootfs upgrading. The RootFS is read-only via SquashFS and there's a read-write user area. UBI is supporting all the volumes, handling bad blocks and wear leveling.
Can someone also point me to the respective driver code in case of using /dev/mtdblock and /dev/ubiblock ? Apart from theory I also want to check the impact at the code level..
You can find all the UBI code in drivers/mtd/ubi of course. The differences between mtdblock and ubiblock are huge: one goes directly to the flash, and the other uses UBI. Good luck! Ezequiel
On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
In other words, IMO it's best to expose the NAND through UBI for both read-only and read-write access, using a single UBI device, and then creating UBI volumes as needed. This will allow UBI to spread wear leveling across the whole device, which is expected to increase the flash lifetime.
For instance, just as some silly example, you could have something like this:
| RootFS SquashFS | | UBI block | UBIFS User R-W area ------------------------------------------------------------------------ Kernel A | Kernel B | RootFS A | RootFS B | User ------------------------------------------------------------------------ UBIX ------------------------------------------------------------------------ /dev/mtdX
This setup allows safe kernel and rootfs upgrading. The RootFS is read-only via SquashFS and there's a read-write user area. UBI is supporting all the volumes, handling bad blocks and wear leveling.
Dear Ezequiel, Thank you so much for your reply. This is exactly what we are also doing :) In our system we have a mix of raw and ubi partitions. The ubi partitioning is done almost exactly the same way. Only for the rootfs (squashfs) I see we were using /mtd/block<id> to mount the rootfs. Now, I understood we should change it to use /dev/ubiblock<id> This might have several benefits, but one most important could be, using ubiblock can handle bad-blocks/wear-leveling automatically, whereas mtdblocks access the flash directly ? I found some references for these.. So, this seems good for my proposal. Another thing that is still open for us is: How do we calculate the exact image size from a raw mtd partition ? For example, support for one of the raw nand partitions, the size is defined as 15MB but we flash the actual image of size only 2.5MB. So, in the runtime how to determine the image size as ~2.5MB (at least roughly) ? Is it still possible ? Thanks, Pintu
Hi All, On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
In other words, IMO it's best to expose the NAND through UBI for both read-only and read-write access, using a single UBI device, and then creating UBI volumes as needed. This will allow UBI to spread wear leveling across the whole device, which is expected to increase the flash lifetime.
For instance, just as some silly example, you could have something like this:
| RootFS SquashFS | | UBI block | UBIFS User R-W area ------------------------------------------------------------------------ Kernel A | Kernel B | RootFS A | RootFS B | User ------------------------------------------------------------------------ UBIX ------------------------------------------------------------------------ /dev/mtdX
This setup allows safe kernel and rootfs upgrading. The RootFS is read-only via SquashFS and there's a read-write user area. UBI is supporting all the volumes, handling bad blocks and wear leveling.
Dear Ezequiel, Thank you so much for your reply.
This is exactly what we are also doing :) In our system we have a mix of raw and ubi partitions. The ubi partitioning is done almost exactly the same way. Only for the rootfs (squashfs) I see we were using /mtd/block<id> to mount the rootfs. Now, I understood we should change it to use /dev/ubiblock<id> This might have several benefits, but one most important could be, using ubiblock can handle bad-blocks/wear-leveling automatically, whereas mtdblocks access the flash directly ? I found some references for these.. So, this seems good for my proposal.
Another thing that is still open for us is: How do we calculate the exact image size from a raw mtd partition ? For example, support for one of the raw nand partitions, the size is defined as 15MB but we flash the actual image of size only 2.5MB. So, in the runtime how to determine the image size as ~2.5MB (at least roughly) ? Is it still possible ?
I am happy to inform you that using "ubiblock" for squashfs mounting seems very helpful for us. We have seen almost the double performance boost when using ubiblock for rootfs as well as other read-only volume mounting. However, we have found few issues while defining the read only volume as STATIC. With static volume we see that OTA update is failing during "fsync". That is ota_fsync is failing from here: https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df... int status = fsync(fd); if (status == -1 && errno == EIO) * { have_eio_error = true; } * return status; } Is this the known issue with static volume? For now we are using dynamic volume itself but the problem is that with dynamic volume we cannot get the exact image size from: $ cat /sys/class/ubi/ubi0_0/data_bytes ==> In case of dynamic volume this will return the total volume size. ==> Thus our md5 integrity check does not match exactly with the flashed image size. Is there an alternate way to handle this issue ? Thanks, Pintu
On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
Hi All,
On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
In other words, IMO it's best to expose the NAND through UBI for both read-only and read-write access, using a single UBI device, and then creating UBI volumes as needed. This will allow UBI to spread wear leveling across the whole device, which is expected to increase the flash lifetime.
For instance, just as some silly example, you could have something like this:
| RootFS SquashFS | | UBI block | UBIFS User R-W area ------------------------------------------------------------------------ Kernel A | Kernel B | RootFS A | RootFS B | User ------------------------------------------------------------------------ UBIX ------------------------------------------------------------------------ /dev/mtdX
This setup allows safe kernel and rootfs upgrading. The RootFS is read-only via SquashFS and there's a read-write user area. UBI is supporting all the volumes, handling bad blocks and wear leveling.
Dear Ezequiel, Thank you so much for your reply.
This is exactly what we are also doing :) In our system we have a mix of raw and ubi partitions. The ubi partitioning is done almost exactly the same way. Only for the rootfs (squashfs) I see we were using /mtd/block<id> to mount the rootfs. Now, I understood we should change it to use /dev/ubiblock<id> This might have several benefits, but one most important could be, using ubiblock can handle bad-blocks/wear-leveling automatically, whereas mtdblocks access the flash directly ? I found some references for these.. So, this seems good for my proposal.
Another thing that is still open for us is: How do we calculate the exact image size from a raw mtd partition ? For example, support for one of the raw nand partitions, the size is defined as 15MB but we flash the actual image of size only 2.5MB. So, in the runtime how to determine the image size as ~2.5MB (at least roughly) ? Is it still possible ?
I am happy to inform you that using "ubiblock" for squashfs mounting seems very helpful for us. We have seen almost the double performance boost when using ubiblock for rootfs as well as other read-only volume mounting.
However, we have found few issues while defining the read only volume as STATIC. With static volume we see that OTA update is failing during "fsync". That is ota_fsync is failing from here: https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df... int status = fsync(fd); if (status == -1 && errno == EIO) * { have_eio_error = true; } * return status; }
Is this the known issue with static volume?
I don't know exactly how you are updating your volume, the right way is using UBI_IOCVOLUP. See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate If you google around I'm sure you'll find some articles about this, but I'm not sure if they'll go into details and subtleties. There are probably a few different ways to do firmware upgrade when you are on top of static volumes (and you want to be on top of static volumes if it's read-only, because AFAIK they give you an extra data-integrity guarantee). One way, would be to have two static volumes A/B. The system uses normally the A volume, and then you doUBI_IOCVOLUP (or ubiupdatevol) to update the B volume. After the update is succesful you run the atomic volume rename and flip A->B, B->A. (If you don't have enough space to hold two A/B volumes.... ... you'll have to find some other solution, I have no idea about that.) Hope it helps, Eze
Hi, On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
Hi All,
On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
In other words, IMO it's best to expose the NAND through UBI for both read-only and read-write access, using a single UBI device, and then creating UBI volumes as needed. This will allow UBI to spread wear leveling across the whole device, which is expected to increase the flash lifetime.
For instance, just as some silly example, you could have something like this:
| RootFS SquashFS | | UBI block | UBIFS User R-W area ------------------------------------------------------------------------ Kernel A | Kernel B | RootFS A | RootFS B | User ------------------------------------------------------------------------ UBIX ------------------------------------------------------------------------ /dev/mtdX
This setup allows safe kernel and rootfs upgrading. The RootFS is read-only via SquashFS and there's a read-write user area. UBI is supporting all the volumes, handling bad blocks and wear leveling.
Dear Ezequiel, Thank you so much for your reply.
This is exactly what we are also doing :) In our system we have a mix of raw and ubi partitions. The ubi partitioning is done almost exactly the same way. Only for the rootfs (squashfs) I see we were using /mtd/block<id> to mount the rootfs. Now, I understood we should change it to use /dev/ubiblock<id> This might have several benefits, but one most important could be, using ubiblock can handle bad-blocks/wear-leveling automatically, whereas mtdblocks access the flash directly ? I found some references for these.. So, this seems good for my proposal.
Another thing that is still open for us is: How do we calculate the exact image size from a raw mtd partition ? For example, support for one of the raw nand partitions, the size is defined as 15MB but we flash the actual image of size only 2.5MB. So, in the runtime how to determine the image size as ~2.5MB (at least roughly) ? Is it still possible ?
I am happy to inform you that using "ubiblock" for squashfs mounting seems very helpful for us. We have seen almost the double performance boost when using ubiblock for rootfs as well as other read-only volume mounting.
However, we have found few issues while defining the read only volume as STATIC. With static volume we see that OTA update is failing during "fsync". That is ota_fsync is failing from here: https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df... int status = fsync(fd); if (status == -1 && errno == EIO) * { have_eio_error = true; } * return status; }
Is this the known issue with static volume?
I don't know exactly how you are updating your volume, the right way is using UBI_IOCVOLUP.
See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
If you google around I'm sure you'll find some articles about this, but I'm not sure if they'll go into details and subtleties.
There are probably a few different ways to do firmware upgrade when you are on top of static volumes (and you want to be on top of static volumes if it's read-only, because AFAIK they give you an extra data-integrity guarantee).
One way, would be to have two static volumes A/B. The system uses normally the A volume, and then you doUBI_IOCVOLUP (or ubiupdatevol) to update the B volume. After the update is succesful you run the atomic volume rename and flip A->B, B->A.
(If you don't have enough space to hold two A/B volumes.... ... you'll have to find some other solution, I have no idea about that.)
Yes, this is what we are also doing exactly. But, currently we are running into this issue right now: 1) The FOTA update is failing if we use static volume (building and flashing the static image is fine) It works fine, if we use the volume as dynamic (even for RO type with squashfs) But with dynamic type we end up using the entire volume as image size. So, we wanted to know if there is any other way to get volume image size at runtime ? Thanks, Pintu
On Mon, 8 Nov 2021 at 10:51, Pintu Agarwal <pintu.ping@gmail.com> wrote:
Hi,
On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
Hi All,
On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> wrote:
In other words, IMO it's best to expose the NAND through UBI for both read-only and read-write access, using a single UBI device, and then creating UBI volumes as needed. This will allow UBI to spread wear leveling across the whole device, which is expected to increase the flash lifetime.
For instance, just as some silly example, you could have something like this:
| RootFS SquashFS | | UBI block | UBIFS User R-W area ------------------------------------------------------------------------ Kernel A | Kernel B | RootFS A | RootFS B | User ------------------------------------------------------------------------ UBIX ------------------------------------------------------------------------ /dev/mtdX
This setup allows safe kernel and rootfs upgrading. The RootFS is read-only via SquashFS and there's a read-write user area. UBI is supporting all the volumes, handling bad blocks and wear leveling.
Dear Ezequiel, Thank you so much for your reply.
This is exactly what we are also doing :) In our system we have a mix of raw and ubi partitions. The ubi partitioning is done almost exactly the same way. Only for the rootfs (squashfs) I see we were using /mtd/block<id> to mount the rootfs. Now, I understood we should change it to use /dev/ubiblock<id> This might have several benefits, but one most important could be, using ubiblock can handle bad-blocks/wear-leveling automatically, whereas mtdblocks access the flash directly ? I found some references for these.. So, this seems good for my proposal.
Another thing that is still open for us is: How do we calculate the exact image size from a raw mtd partition ? For example, support for one of the raw nand partitions, the size is defined as 15MB but we flash the actual image of size only 2.5MB. So, in the runtime how to determine the image size as ~2.5MB (at least roughly) ? Is it still possible ?
I am happy to inform you that using "ubiblock" for squashfs mounting seems very helpful for us. We have seen almost the double performance boost when using ubiblock for rootfs as well as other read-only volume mounting.
However, we have found few issues while defining the read only volume as STATIC. With static volume we see that OTA update is failing during "fsync". That is ota_fsync is failing from here: https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df... int status = fsync(fd); if (status == -1 && errno == EIO) * { have_eio_error = true; } * return status; }
Is this the known issue with static volume?
I don't know exactly how you are updating your volume, the right way is using UBI_IOCVOLUP.
See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
If you google around I'm sure you'll find some articles about this, but I'm not sure if they'll go into details and subtleties.
There are probably a few different ways to do firmware upgrade when you are on top of static volumes (and you want to be on top of static volumes if it's read-only, because AFAIK they give you an extra data-integrity guarantee).
One way, would be to have two static volumes A/B. The system uses normally the A volume, and then you doUBI_IOCVOLUP (or ubiupdatevol) to update the B volume. After the update is succesful you run the atomic volume rename and flip A->B, B->A.
(If you don't have enough space to hold two A/B volumes.... ... you'll have to find some other solution, I have no idea about that.)
Yes, this is what we are also doing exactly. But, currently we are running into this issue right now: 1) The FOTA update is failing if we use static volume (building and flashing the static image is fine)
Please add traces to find what syscall is failing, and provide more details about it. Thanks, Ezequiel
participants (2)
-
Ezequiel Garcia -
Pintu Agarwal