Re: [kernel]:overlayfs: invalid origin (root/bmcweb_persistent_data.json, ftype=8000, origin ftype=4000)
On Thu, Apr 15, 2021 at 2:03 PM www <ouyangxuan10@163.com> wrote:
Dear Amir, kevin, miklos,
An open source project uses overlay filesystem and systemd. when the system first time boot, there is an error in the file system: kernel: overlayfs: invalid origin (root/bmcweb_persistent_data.json, ftype=8000, origin ftype=4000)
What's the reason for this? Can you give some help and suggestions to solve this problem?
The details are as follows: 1. linux kernel version: root@N/A:~# uname -a Linux N/A 5.1.5-yocto-s-dirty-1605363 #1 Tue Apr 13 01:13:48 UTC 2021 armv6l GNU/Linux
2. mount command mount -t overlay -o lowerdir=$rodir,upperdir=$upper,workdir=$work cow /root (rodir=run/initramfs/ro, upper=run/initramfs/rw/cow, work=rwdir=run/initramfs/rw/work)
3. mountinfo root@N/A:~# cat /proc/self/mountinfo 13 21 0:6 / /dev rw,relatime shared:8 - devtmpfs dev rw,size=220800k,nr_inodes=55200,mode=755 14 21 0:13 / /sys rw,relatime shared:3 - sysfs sys rw 15 21 0:4 / /proc rw,relatime shared:2 - proc proc rw 16 21 0:14 / /run rw,nosuid,nodev shared:11 - tmpfs tmpfs rw,mode=755 17 16 7:0 / /run/initramfs/ro ro,relatime shared:12 - squashfs /dev/loop0 ro 18 16 31:5 / /run/initramfs/rw rw,relatime shared:13 - jffs2 /dev/mtdblock5 rw 21 0 0:15 / / rw,relatime shared:1 - overlay cow rw,lowerdir=run/initramfs/ro,upperdir=run/initramfs/rw/cow,workdir=run/initramfs/rw/work 22 13 0:17 / /dev/shm rw,nosuid,nodev shared:9 - tmpfs tmpfs rw 23 13 0:18 / /dev/pts rw,relatime shared:10 - devpts devpts rw,gid=5,mode=620,ptmxmode=000 24 14 0:19 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,mode=755 25 24 0:20 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,nsdelegate 26 24 0:21 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,xattr,name=systemd 27 14 0:22 / /sys/fs/bpf rw,nosuid,nodev,noexec,relatime shared:7 - bpf bpf rw,mode=700 28 14 0:7 / /sys/kernel/debug rw,nosuid,nodev,noexec,relatime shared:14 - debugfs debugfs rw 29 14 0:12 / /sys/kernel/config rw,nosuid,nodev,noexec,relatime shared:15 - configfs configfs rw 30 21 0:23 / /tmp rw,nosuid,nodev shared:16 - tmpfs tmpfs rw 31 21 0:24 / /var/volatile rw,relatime shared:17 - tmpfs tmpfs rw
4. xino_auto
5. file in ro is correct 6. file in rw is correct 7. but the file in overlayfs is incorrect
What's the reason for this?
I suppose lower squashfs was recreated and mounted with an existing upper dir that already contains upper files with origin references to the old squashfs lower fs.
Can you give some help and suggestions to solve this problem?
Cherry-pick this fix to your kernel: a888db310195 ovl: fix regression with re-formatted lower squashfs OR as a workaround, build the lower squashfs with the "-no-exports" mksquashfs option. Thanks, Amir.
Hi Amir, Thank you very much for your help. I will import this solution and test it. thanks, Byron At 2021-04-16 01:09:23, "Amir Goldstein" <amir73il@gmail.com> wrote:
On Thu, Apr 15, 2021 at 2:03 PM www <ouyangxuan10@163.com> wrote:
Dear Amir, kevin, miklos,
An open source project uses overlay filesystem and systemd. when the system first time boot, there is an error in the file system: kernel: overlayfs: invalid origin (root/bmcweb_persistent_data.json, ftype=8000, origin ftype=4000)
What's the reason for this? Can you give some help and suggestions to solve this problem?
The details are as follows: 1. linux kernel version: root@N/A:~# uname -a Linux N/A 5.1.5-yocto-s-dirty-1605363 #1 Tue Apr 13 01:13:48 UTC 2021 armv6l GNU/Linux
2. mount command mount -t overlay -o lowerdir=$rodir,upperdir=$upper,workdir=$work cow /root (rodir=run/initramfs/ro, upper=run/initramfs/rw/cow, work=rwdir=run/initramfs/rw/work)
3. mountinfo root@N/A:~# cat /proc/self/mountinfo 13 21 0:6 / /dev rw,relatime shared:8 - devtmpfs dev rw,size=220800k,nr_inodes=55200,mode=755 14 21 0:13 / /sys rw,relatime shared:3 - sysfs sys rw 15 21 0:4 / /proc rw,relatime shared:2 - proc proc rw 16 21 0:14 / /run rw,nosuid,nodev shared:11 - tmpfs tmpfs rw,mode=755 17 16 7:0 / /run/initramfs/ro ro,relatime shared:12 - squashfs /dev/loop0 ro 18 16 31:5 / /run/initramfs/rw rw,relatime shared:13 - jffs2 /dev/mtdblock5 rw 21 0 0:15 / / rw,relatime shared:1 - overlay cow rw,lowerdir=run/initramfs/ro,upperdir=run/initramfs/rw/cow,workdir=run/initramfs/rw/work 22 13 0:17 / /dev/shm rw,nosuid,nodev shared:9 - tmpfs tmpfs rw 23 13 0:18 / /dev/pts rw,relatime shared:10 - devpts devpts rw,gid=5,mode=620,ptmxmode=000 24 14 0:19 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,mode=755 25 24 0:20 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,nsdelegate 26 24 0:21 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,xattr,name=systemd 27 14 0:22 / /sys/fs/bpf rw,nosuid,nodev,noexec,relatime shared:7 - bpf bpf rw,mode=700 28 14 0:7 / /sys/kernel/debug rw,nosuid,nodev,noexec,relatime shared:14 - debugfs debugfs rw 29 14 0:12 / /sys/kernel/config rw,nosuid,nodev,noexec,relatime shared:15 - configfs configfs rw 30 21 0:23 / /tmp rw,nosuid,nodev shared:16 - tmpfs tmpfs rw 31 21 0:24 / /var/volatile rw,relatime shared:17 - tmpfs tmpfs rw
4. xino_auto
5. file in ro is correct 6. file in rw is correct 7. but the file in overlayfs is incorrect
What's the reason for this?
I suppose lower squashfs was recreated and mounted with an existing upper dir that already contains upper files with origin references to the old squashfs lower fs.
Can you give some help and suggestions to solve this problem?
Cherry-pick this fix to your kernel: a888db310195 ovl: fix regression with re-formatted lower squashfs
OR as a workaround, build the lower squashfs with the "-no-exports" mksquashfs option.
Thanks, Amir.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi Amir, To solve this problem, we are trying to upgrade the kernel version or upgrade the file system separately, but because the kernel version we are using is quite different from the one that solves the problem. So I would like to ask you a few questions: 1. What is the root cause of this problem? 2. Can we just upgrade overlay fs to solve this problem without upgrading the kernel or all file systems? 3. If we upgrade overlayfs separately, we are not very good at verifying that we have solved this problem, because the recurrence probability of this problem is very low. So I want to ask, how can we quickly reproduce this problem? 4. Do you have any good suggestions? I am looking forward to your reply. Byron At 2021-04-16 09:34:18, "www" <ouyangxuan10@163.com> wrote: Hi Amir, Thank you very much for your help. I will import this solution and test it. thanks, Byron At 2021-04-16 01:09:23, "Amir Goldstein" <amir73il@gmail.com> wrote:
On Thu, Apr 15, 2021 at 2:03 PM www <ouyangxuan10@163.com> wrote:
Dear Amir, kevin, miklos,
An open source project uses overlay filesystem and systemd. when the system first time boot, there is an error in the file system: kernel: overlayfs: invalid origin (root/bmcweb_persistent_data.json, ftype=8000, origin ftype=4000)
What's the reason for this? Can you give some help and suggestions to solve this problem?
The details are as follows: 1. linux kernel version: root@N/A:~# uname -a Linux N/A 5.1.5-yocto-s-dirty-1605363 #1 Tue Apr 13 01:13:48 UTC 2021 armv6l GNU/Linux
2. mount command mount -t overlay -o lowerdir=$rodir,upperdir=$upper,workdir=$work cow /root (rodir=run/initramfs/ro, upper=run/initramfs/rw/cow, work=rwdir=run/initramfs/rw/work)
3. mountinfo root@N/A:~# cat /proc/self/mountinfo 13 21 0:6 / /dev rw,relatime shared:8 - devtmpfs dev rw,size=220800k,nr_inodes=55200,mode=755 14 21 0:13 / /sys rw,relatime shared:3 - sysfs sys rw 15 21 0:4 / /proc rw,relatime shared:2 - proc proc rw 16 21 0:14 / /run rw,nosuid,nodev shared:11 - tmpfs tmpfs rw,mode=755 17 16 7:0 / /run/initramfs/ro ro,relatime shared:12 - squashfs /dev/loop0 ro 18 16 31:5 / /run/initramfs/rw rw,relatime shared:13 - jffs2 /dev/mtdblock5 rw 21 0 0:15 / / rw,relatime shared:1 - overlay cow rw,lowerdir=run/initramfs/ro,upperdir=run/initramfs/rw/cow,workdir=run/initramfs/rw/work 22 13 0:17 / /dev/shm rw,nosuid,nodev shared:9 - tmpfs tmpfs rw 23 13 0:18 / /dev/pts rw,relatime shared:10 - devpts devpts rw,gid=5,mode=620,ptmxmode=000 24 14 0:19 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,mode=755 25 24 0:20 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,nsdelegate 26 24 0:21 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,xattr,name=systemd 27 14 0:22 / /sys/fs/bpf rw,nosuid,nodev,noexec,relatime shared:7 - bpf bpf rw,mode=700 28 14 0:7 / /sys/kernel/debug rw,nosuid,nodev,noexec,relatime shared:14 - debugfs debugfs rw 29 14 0:12 / /sys/kernel/config rw,nosuid,nodev,noexec,relatime shared:15 - configfs configfs rw 30 21 0:23 / /tmp rw,nosuid,nodev shared:16 - tmpfs tmpfs rw 31 21 0:24 / /var/volatile rw,relatime shared:17 - tmpfs tmpfs rw
4. xino_auto
5. file in ro is correct 6. file in rw is correct 7. but the file in overlayfs is incorrect
What's the reason for this?
I suppose lower squashfs was recreated and mounted with an existing upper dir that already contains upper files with origin references to the old squashfs lower fs.
Can you give some help and suggestions to solve this problem?
Cherry-pick this fix to your kernel: a888db310195 ovl: fix regression with re-formatted lower squashfs
OR as a workaround, build the lower squashfs with the "-no-exports" mksquashfs option.
Thanks, Amir.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Tue, Apr 27, 2021 at 11:40 AM www <ouyangxuan10@163.com> wrote:
Hi Amir,
To solve this problem, we are trying to upgrade the kernel version or upgrade the file system separately, but because the kernel version we are using is quite different from the one that solves the problem. So I would like to ask you a few questions:
1. What is the root cause of this problem?
As I wrote: I suppose lower squashfs was recreated and mounted with an existing upper dir that already contains upper files with origin references to the old squashfs lower fs.
2. Can we just upgrade overlay fs to solve this problem without upgrading the kernel or all file systems?
No.
3. If we upgrade overlayfs separately, we are not very good at verifying that we have solved this problem, because the recurrence probability of this problem is very low. So I want to ask, how can we quickly reproduce this problem?
Re-creating a lower squashfs after files have been copied to upper should reproduce the problem quite often.
4. Do you have any good suggestions?
Is the creation of the lower image under your control? Did you try the workaround I suggested to create lower squashfs with the "-no-exports" mksquashfs option? Try this patch: diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c index 71e264e2f16b..850c0a37f1f0 100644 --- a/fs/overlayfs/namei.c +++ b/fs/overlayfs/namei.c @@ -392,7 +392,7 @@ int ovl_check_origin_fh(struct ovl_fs *ofs, struct ovl_fh *fh, bool connected, upperdentry, d_inode(upperdentry)->i_mode & S_IFMT, d_inode(origin)->i_mode & S_IFMT); dput(origin); - return -EIO; + return -EINVAL; } static int ovl_check_origin(struct ovl_fs *ofs, struct dentry *upperdentry, @@ -408,7 +408,7 @@ static int ovl_check_origin(struct ovl_fs *ofs, struct dentry *upperdentry, kfree(fh); if (err) { - if (err == -ESTALE) + if (err == -ESTALE || err == -EINVAL) return 0; return err; } It should apply to any kernel and if not it's quite easy to manually apply it. This fix will not get rid of the warning printed to kmsg, but it will hide the error from userspace. If it fixes your problem I think we should apply it to upstream and stable kernels as well. Thanks, Amir.
participants (2)
-
Amir Goldstein -
www