About bio_endio

Tue Jun 24 12:23:50 EDT 2014

Hello Pranay!

Thanks for your reply.  I apologize for my very late reply, I was very
preoccupied earlier at work.

On Tue, Jun 24, 2014 at 1:07 PM, Pranay Srivastava <pranjas at gmail.com>
wrote:

> Hello Alvin,
>
> On Mon, Jun 23, 2014 at 10:39 PM, Alvin Abitria <abitria.alvin at gmail.com>
> wrote:
> > Hello,
> >
> > I'm developing a block driver using the make_request method, effectively
> > bypassing existing scsi or request stack in block layer.  So that means
> im
> > directly working with bios.  As prescribed in linux documentation and
> from
> > referring to similar drivers in kernel, you close a session with a bio
> with
> > the bio_endio function.
>
> So it means you are just passing on the bios without the request
> structure if I'm correct?
> I don't know how you are handling blk_finish_plug without having
> request or request queue,
> I maybe wrong in understanding how you are handling it.
>
> Yes, I'm working on bio's level.  No struct requests, and I haven't used
blk_finish_plug yet.
The block driver method I'm implementing is somewhat along the same line
with nvme and mtip2xxx
drivers in drivers/block directory (but differing in hardware specific
level of course).

> >
> > I usually invoke bio_endio during successful I/O completion, meaning
> with an
> > error code of zero.  But there are cases that this is not fulfilled or
> there
> > are error cases.  My question is, what are the valid error codes that
> can be
> > used with it?  My initial impression is that other than zero as error
> code,
> -EIO is the one that you should use I think,
> > bio_endio will fail.  I've read somewhere that -EBUSY is not recognized,
> and
> > I tried -EIO but my driver crashed.  I got a panic in some dio_xxx
> function
> > leading from bio_endio(bio,-EIO). I would like to block subsequent bios
> sent
>
> If it's okay for you to post the error then can you do that? I was
> seeing the code for
> dio_end_io but it would be good if you can post the exact crash
> backtrace if you've got that.
>

Here you go:

BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<ffffffff811b9a80>] bio_check_pages_dirty+0x50/0xe0
PGD 42e5e4067 PUD 42e6e7067 PMD 0
Oops: 0000 [#1] SMP
last sysfs file:
/sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size
CPU 7
Modules linked in: block_module(U) fuse ip6table_filter ip6_tables
ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4
nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle
iptable_filter ip_tables bridge autofs4 sunrpc 8021q garp stp llc
cpufreq_ondemand freq_table pcc_cpufreq ipv6 vhost_net macvtap macvlan tun
kvm_intel kvm uinput power_meter hpilo hpwdt sg tg3 microcode serio_raw
iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod
crc_t10dif hpsa pata_acpi ata_generic ata_piix dm_mirror dm_region_hash
dm_log dm_mod [last unloaded: scsi_wait_scan]

Pid: 3740, comm: fio Not tainted 2.6.32-358.el6.x86_64 #1 HP ProLiant
DL380p Gen8
RIP: 0010:[<ffffffff811b9a80>]  [<ffffffff811b9a80>]
bio_check_pages_dirty+0x50/0xe0
RSP: 0018:ffff8804191618c8  EFLAGS: 00010046
RAX: 2000000000000000 RBX: ffff88041909f0c0 RCX: 00000000000011ae
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff8804191618f8 R08: ffffffff81c07728 R09: 0000000000000040
R10: 0000000000000002 R11: 0000000000000002 R12: 0000000000000000
R13: ffff8804191b9b80 R14: ffff8804191b9b80 R15: 0000000000000000
FS:  00007fcd43e2d720(0000) GS:ffff8800366e0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000041e69f000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process fio (pid: 3740, threadinfo ffff880419160000, task ffff88043341f500)
Stack:
 0000000000000000 ffff88043433f400 ffff88043433f520 ffff88041909f0c0
<d> ffff88041909f0c0 ffff8804191b9b80 ffff880419161948 ffffffff811bdc38
<d> ffff880419161968 0000000034236400 00000000fffffffb ffff88043433f400
Call Trace:
 [<ffffffff811bdc38>] dio_bio_complete+0xc8/0xd0
 [<ffffffff811bef4f>] dio_bio_end_aio+0x2f/0xd0
 [<ffffffff811b920d>] bio_endio+0x1d/0x40
 [<ffffffffa02c15a1>] block_make_request+0xe1/0x150 [block_module]
 [<ffffffff8125ccce>] generic_make_request+0x25e/0x530
 [<ffffffff811bae72>] ? bvec_alloc_bs+0x62/0x110
 [<ffffffff8125d02d>] submit_bio+0x8d/0x120
 [<ffffffff811bdf6c>] dio_bio_submit+0xbc/0xc0
 [<ffffffff811be951>] __blockdev_direct_IO_newtrunc+0x631/0xb30
 [<ffffffff8111afe3>] ? filemap_fault+0xd3/0x500
 [<ffffffff811beeae>] __blockdev_direct_IO+0x5e/0xd0
 [<ffffffff811bb280>] ? blkdev_get_blocks+0x0/0xc0
 [<ffffffff811bc347>] blkdev_direct_IO+0x57/0x60
 [<ffffffff811bb280>] ? blkdev_get_blocks+0x0/0xc0
 [<ffffffff8111bb8b>] generic_file_aio_read+0x6bb/0x700
 [<ffffffff81166a2a>] ? kmem_getpages+0xba/0x170
 [<ffffffff81166f87>] ? cache_grow+0x217/0x320
 [<ffffffff811bb893>] blkdev_aio_read+0x53/0xc0
 [<ffffffff8111c633>] ? mempool_alloc+0x63/0x140
 [<ffffffff811bb840>] ? blkdev_aio_read+0x0/0xc0
 [<ffffffff811cadc4>] aio_rw_vect_retry+0x84/0x200
 [<ffffffff811cc784>] aio_run_iocb+0x64/0x170
 [<ffffffff811cdbb1>] do_io_submit+0x291/0x920
 [<ffffffff811ce250>] sys_io_submit+0x10/0x20
 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
Code: 31 e4 45 31 ff eb 15 0f 1f 40 00 0f b7 43 28 41 83 c4 01 41 83 c7 01
44 39 e0 7e 36 4d 63 ec 49 c1 e5 04 4f 8d 2c 2e 49 8b 7d 00 <48> 8b 07 a8
10 75 06 66 a9 00 c0 74 d3 e8 5e 59 f7 ff 49 c7 45
RIP  [<ffffffff811b9a80>] bio_check_pages_dirty+0x50/0xe0
 RSP <ffff8804191618c8>
CR2: 0000000000000000

Basically, after receiving the bio from generic_make_request, I checked and
found that
I already have the maximum outstanding pending I/O (bios) and couldn't
accomodate
this bio for now.  Rather than just exiting the make_request() routine, I
decided to close
this bio or return it to kernel via bio_endio(bio, -EIO).  No processing or
modification
 was done to the bio, I just used the callback fn above.

> > to me after reaching my queue depth and with no tags left, and so I want
> to
> > use bio_endio with an error code.
>
> If you have a request queue then you could call blk_stop_queue and
> blk_start_queue but I don't know if this is
> relevant to your case.
>

I do have a request queue allocated using blk_alloc_queue, but this is more
of a dummy queue
in my case - I don't use it as much as the struct requests would use it.
 Its function that I used is
for me to register my make_request function, register the maximum I/O size
and max number of
buffer segments that can be sent to my driver.

Thanks for the suggestion, I haven't tried blk_stop_queue and
blk_start_queue but I hope it will
work.  I also like the possibility that you can prevent the upper layers
from sending me further bio
when my queue depth is full, then letting them know once my driver can
accomodate new bios -
more of telling them I'm busy right now, don't send me further bios, etc.
 Is that the general idea behind
the two functions you mentioned?

I also notice that the two kernel APIs you gave either set/clear a certain
request queue flag.  I'd like to
think that upper layers (generic_make_request etc) check that flag first to
decide if it can dispatch bios
to my driver, is that right?

>
> >
> > What are those error codes, and will they work for my intended function?
> > Thanks!
>
> -EIO should work, but first let's find out why you got the crash.
>
> Thanks.  I hope we get to the bottom why it failed and crashed.  Let me
know if you have questions.

> > _______________________________________________
> > Kernelnewbies mailing list
> > Kernelnewbies at kernelnewbies.org
> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >
>
>
>
> --
>         ---P.K.S
>
Alvin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140625/74cc1a74/attachment.html 