Block device driver question
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request. 1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it. Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me. I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA). 2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b'). Then in that case should i be using a queue in my layer - put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below. Regards, Neha
Hi Neha, On Fri, Nov 1, 2013 at 10:26 AM, neha naik <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it. Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me. I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
If you are not using oflag=direct with dd, then you are getting 'page' in bvec that belongs to buffer cache (in 2.6 it is implemented as page-cache of block_device->bd_inode->i_mapping). You get user buffer only with direct IO, but then you need to take care to issue aligned IO requests yourself (if your block device wants only aligned buffers its your implementation though).
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b'). Then in that case should i be using a queue in my layer - put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
If your application does not quarantee the ordering of writes, then you don't have to worry either. Most likely block layer will do the merges in page-cache if it is not a direct IO. As a driver developer, you don't need to worry about out of order writes from application.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi Rajat, Thanks for the information. One more question : Say my block device driver doesn't support reads and the application always does aligned io in 512 chunks (but it is not direct io). In that case, will i get a read because the page size is 4096 and yet we are writing 512. Because i am not getting any read which is why i am confused.I have been doing the io after syncing the page cache so it is not like i get a pagecache hit every time. I am doing a normal dd without any special flags, just 'bs=512'. Regards, Neha On Fri, Nov 1, 2013 at 12:16 PM, Rajat Sharma <fs.rajat@gmail.com> wrote:
Hi Neha,
On Fri, Nov 1, 2013 at 10:26 AM, neha naik <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it. Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me. I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
If you are not using oflag=direct with dd, then you are getting 'page' in bvec that belongs to buffer cache (in 2.6 it is implemented as page-cache of block_device->bd_inode->i_mapping). You get user buffer only with direct IO, but then you need to take care to issue aligned IO requests yourself (if your block device wants only aligned buffers its your implementation though).
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b'). Then in that case should i be using a queue in my layer - put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
If your application does not quarantee the ordering of writes, then you don't have to worry either. Most likely block layer will do the merges in page-cache if it is not a direct IO. As a driver developer, you don't need to worry about out of order writes from application.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Fri, Nov 1, 2013 at 12:17 PM, neha naik <nehanaik27@gmail.com> wrote:
Hi Rajat, Thanks for the information. One more question : Say my block device driver doesn't support reads and the application always does aligned io in 512 chunks (but it is not direct io). In that case, will i get a read because the page size is 4096 and yet we are writing 512. Because i am not getting any read which is why i am confused.I have been doing the io after syncing the page cache so it is not like i get a pagecache hit every time.
sync does not evict page cache. And is your block device sector size declared as 512 ?
I am doing a normal dd without any special flags, just 'bs=512'.
Regards, Neha
On Fri, Nov 1, 2013 at 12:16 PM, Rajat Sharma <fs.rajat@gmail.com> wrote:
Hi Neha,
On Fri, Nov 1, 2013 at 10:26 AM, neha naik <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it. Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me. I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
If you are not using oflag=direct with dd, then you are getting 'page' in bvec that belongs to buffer cache (in 2.6 it is implemented as page-cache of block_device->bd_inode->i_mapping). You get user buffer only with direct IO, but then you need to take care to issue aligned IO requests yourself (if your block device wants only aligned buffers its your implementation though).
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b'). Then in that case should i be using a queue in my layer - put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
If your application does not quarantee the ordering of writes, then you don't have to worry either. Most likely block layer will do the merges in page-cache if it is not a direct IO. As a driver developer, you don't need to worry about out of order writes from application.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi, I am executing the command sync && echo 3 | tee /proc/sys/vm/drop_cache for dropping all the caches. And yes my block device sector size is declared as 512. Should this ensure that if i write only 512 i will not get a read. Regards, Neha On Fri, Nov 1, 2013 at 2:25 PM, Rajat Sharma <fs.rajat@gmail.com> wrote:
On Fri, Nov 1, 2013 at 12:17 PM, neha naik <nehanaik27@gmail.com> wrote:
Hi Rajat, Thanks for the information. One more question : Say my block device driver doesn't support reads and the application always does aligned io in 512 chunks (but it is not direct io). In that case, will i get a read because the page size is 4096 and yet we are writing 512. Because i am not getting any read which is why i am confused.I have been doing the io after syncing the page cache so it is not like i get a pagecache hit every time.
sync does not evict page cache. And is your block device sector size declared as 512 ?
I am doing a normal dd without any special flags, just 'bs=512'.
Regards, Neha
On Fri, Nov 1, 2013 at 12:16 PM, Rajat Sharma <fs.rajat@gmail.com> wrote:
Hi Neha,
On Fri, Nov 1, 2013 at 10:26 AM, neha naik <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it. Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me. I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
If you are not using oflag=direct with dd, then you are getting 'page' in bvec that belongs to buffer cache (in 2.6 it is implemented as page-cache of block_device->bd_inode->i_mapping). You get user buffer only with direct IO, but then you need to take care to issue aligned IO requests yourself (if your block device wants only aligned buffers its your implementation though).
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b'). Then in that case should i be using a queue in my layer - put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
If your application does not quarantee the ordering of writes, then you don't have to worry either. Most likely block layer will do the merges in page-cache if it is not a direct IO. As a driver developer, you don't need to worry about out of order writes from application.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On 01-Nov-2013 10:57 PM, "neha naik" <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it.
Page cache use is for file system. Block driver has got nothing to do with it. So lets keep these separate. Bios don't care which page you give them all it needs is a page in bvec. The file system would wait on that page to be uptodate which might be done in bio_end_io if i/o was good. In case of buffer heads the same thing. Submit_bh would create a bio for that bh so really same stuff.
Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me.
If you are writing why it would read the page? Reads would initiate write outs i think. Take a look at generic_file_aio_write.
I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
Whatever filesystem gave it. If it uses the generic functions that should come from page cache but again it depends on how filesystem created bio. So for block driver you need to know if the page you are given in bvec is something you can use or you need to check and take measures to successfully do i/o.
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b').
Then in that case should i be using a queue in my layer -
put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
You mean layer above right? that is the filesystem correct? But if thats the case then wouldn't your second request be blocked until the page was unlocked by file system which would happen i think after your driver was done with i/o. Thats because you won't mark the request as complete so i guess threads would wait_on_page to be unlocked. If however your driver "lies" about completing requests then yeah you need to take appropriate measure.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--P.K.S
Hi Pranay, Your answers assume that there is always a filesystem above the block device driver which is not necessarily condition. Regards, Neha On Nov 2, 2013 8:58 AM, "Pranay Srivastava" <pranjas@gmail.com> wrote:
On 01-Nov-2013 10:57 PM, "neha naik" <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it.
Page cache use is for file system. Block driver has got nothing to do with it. So lets keep these separate.
Bios don't care which page you give them all it needs is a page in bvec. The file system would wait on that page to be uptodate which might be done in bio_end_io if i/o was good.
In case of buffer heads the same thing. Submit_bh would create a bio for that bh so really same stuff.
Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me.
If you are writing why it would read the page? Reads would initiate write outs i think. Take a look at generic_file_aio_write.
I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
Whatever filesystem gave it. If it uses the generic functions that should come from page cache but again it depends on how filesystem created bio.
So for block driver you need to know if the page you are given in bvec is something you can use or you need to check and take measures to successfully do i/o.
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b').
Then in that case should i be using a queue in my layer -
put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
You mean layer above right? that is the filesystem correct? But if thats the case then wouldn't your second request be blocked until the page was unlocked by file system which would happen i think after your driver was done with i/o. Thats because you won't mark the request as complete so i guess threads would wait_on_page to be unlocked.
If however your driver "lies" about completing requests then yeah you need to take appropriate measure.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--P.K.S
On 03-Nov-2013 9:10 AM, "neha naik" <nehanaik27@gmail.com> wrote:
Hi Pranay, Your answers assume that there is always a filesystem above the block
device driver which is not necessarily condition. Yes. But isn't your case the same one? You are doing i/o directly on device right? Does your case differ? ---P.K.S
Regards, Neha
On Nov 2, 2013 8:58 AM, "Pranay Srivastava" <pranjas@gmail.com> wrote:
On 01-Nov-2013 10:57 PM, "neha naik" <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it.
Page cache use is for file system. Block driver has got nothing to do
with it. So lets keep these separate.
Bios don't care which page you give them all it needs is a page in bvec.
The file system would wait on that page to be uptodate which might be done in bio_end_io if i/o was good.
In case of buffer heads the same thing. Submit_bh would create a bio for
that bh so really same stuff.
Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me.
If you are writing why it would read the page? Reads would initiate
write outs i think. Take a look at generic_file_aio_write.
I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
Whatever filesystem gave it. If it uses the generic functions that
should come from page cache but again it depends on how filesystem created bio.
So for block driver you need to know if the page you are given in bvec
is something you can use or you need to check and take measures to successfully do i/o.
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b').
Then in that case should i be using a queue in my layer -
put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
You mean layer above right? that is the filesystem correct? But if thats
the case then wouldn't your second request be blocked until the page was unlocked by file system which would happen i think after your driver was done with i/o. Thats because you won't mark the request as complete so i guess threads would wait_on_page to be unlocked.
If however your driver "lies" about completing requests then yeah you
need to take appropriate measure.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--P.K.S
Hi Pranay, Basically user can do io directly on device without involving the filesystem. Regards, Neha On Sat, Nov 2, 2013 at 11:18 PM, Pranay Srivastava <pranjas@gmail.com> wrote:
On 03-Nov-2013 9:10 AM, "neha naik" <nehanaik27@gmail.com> wrote:
Hi Pranay, Your answers assume that there is always a filesystem above the block device driver which is not necessarily condition.
Yes. But isn't your case the same one? You are doing i/o directly on device right? Does your case differ?
---P.K.S
Regards, Neha
On Nov 2, 2013 8:58 AM, "Pranay Srivastava" <pranjas@gmail.com> wrote:
On 01-Nov-2013 10:57 PM, "neha naik" <nehanaik27@gmail.com> wrote:
Hi, I am writing a block device driver and i am using the 'blq_queue_make_request' call while registering my block device driver. Now as far as i understand this will bypass the linux kernel queue for each block device driver (bypassing the elevator algorithm etc). However, i am still not very clear about exactly how i get a request.
1. Consider i am doing a dd on the block device directly : Will it bypass the buffer cache(/page cache) or will it use it.
Page cache use is for file system. Block driver has got nothing to do with it. So lets keep these separate.
Bios don't care which page you give them all it needs is a page in bvec. The file system would wait on that page to be uptodate which might be done in bio_end_io if i/o was good.
In case of buffer heads the same thing. Submit_bh would create a bio for that bh so really same stuff.
Example if i register my block device with set_blocksize() as 512. And i do a dd of 512 bytes will i get a read because it passes through the buffer cache and since the minimum page size is 4096 it has to read the page first and then pass it to me.
If you are writing why it would read the page? Reads would initiate write outs i think. Take a look at generic_file_aio_write.
I am still unclear about the 'page' in the bvec. What does that refer to? Is it a page from the page cache or a user buffer (DMA).
Whatever filesystem gave it. If it uses the generic functions that should come from page cache but again it depends on how filesystem created bio.
So for block driver you need to know if the page you are given in bvec is something you can use or you need to check and take measures to successfully do i/o.
2. Another thing i am not clear about is a queue. When i register my driver, the 'make_request' function gets called whenever there is an io. Now in my device driver, i have some more logic about writing this io i.e some time may be spent in the device driver for each io. In such a case, if i get two ios on the same block one after the other (say one is writing 'a' and the other is writing 'b') then isn't it possible that i may end up passing 'b' followed by 'a' to the layer below me (changing the order because thread 'a' took more time than thread 'b').
Then in that case should i be using a queue in my layer -
put the ios in the queue whenever i get a call to 'make_request'. Another thread keeps pulling the ios from the queue and processing them and passing it to the layer below.
You mean layer above right? that is the filesystem correct? But if thats the case then wouldn't your second request be blocked until the page was unlocked by file system which would happen i think after your driver was done with i/o. Thats because you won't mark the request as complete so i guess threads would wait_on_page to be unlocked.
If however your driver "lies" about completing requests then yeah you need to take appropriate measure.
Regards, Neha
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--P.K.S
participants (3)
-
neha naik -
Pranay Srivastava -
Rajat Sharma