Sorry, dropped kernelnewbies somehow. Please keep them in cc. Your message and my reply below. On Wed, Oct 17, 2012 at 6:46 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:

On Wed, Oct 17, 2012 at 12:06 PM, Dmitry Filippov <filippovd20@gmail.com> wrote:

...

Greg,

thank you for reply! Yeah there is a kind of analogy to network stack in scsi stack.

the are 3 layers in scsi stack: upper layer (sg, sd, sr, st drivers) that implements high level abstraction middle layer which routes scsi commnads/requests to low level drivers low layer - actually host adapters. low level drivers which registers within scsi stack, gets scsi commands from mid layer and perform actual actions on device.

so from file system any syscall reaches vfs layer first, and then to upper scsi layer.

thanks, Dmitry

vfs actually talks to the block i/o layer which may or may not include multiple device mapper layers. Device mapper (DM) can of course implement logical volumes and/or some raid levels by combining multiple block devices. It can also do block translations etc.

(There is also the MD layer, but I don't really know how it fits in. DM is a newer subsystem and has a cleaner architecture I believe, but it doesn't offer as many raid levels as MD does. You can have both DM and MD in use together if appropriate.)

At the bottom of the stack there is also a scsi-physical layer: much like IP can travel over multiple physical media (Ethernet, T1/E1, etc.) scsi can traverse lots of different physical media.

And then in some cases there is a translation layer below that. SAT (SCSI / ATA Translation) is the best known and typically implemented in external USB carriers.

So it's more like:

syscall -> vfs -> (block layer <-> device mapper <-> MD) -> scsi upper layer -> scsi mid layer -> scsi low layer -> scsi physical (eg. Parallel, SAS, USB) -> translation (eg. SAT)

Notice how I showed the block layer, device mapper, and MD on one line. That's because they really interact a lot and potentially have multiple layers of DM / MD translations.

Also, I don't show libata in the above stack. If you want to talk to a SATA drive, then you'll want to understand how it fits in.

(If you don't know, libata is NOT a full fledged block driver. It was shoe horned into the above SCSI stack. That's why it's /dev/sdx now and not /dev/hda like it was 10 years ago. The /dev/hda (legacy IDE) block driver stack is still in kernel, but it no longer accepts major updates. (deprecated? I think that's right.))

But all that's the linux kernel structure.

There are also scsi layers defined in the SCSI specs. If you're going to work with SCSI you will want to be somewhat familiar with the specs. t10 is the SCSI standards body. Take a look a this architectural chart:

http://www.t10.org/scsi-3.htm

Note that it is the architecture of the specs, and not necessarily the architecture of kernel.

I think libata fits into that architecture just below the SAM level, so it is a sibling of parallel SCSI, SAS, FC, iSCSI, USB, etc.

Again, that is not really the optimal way for libata to work, but it prevents having to write unique upper layers for libata. libata has been around as long as the 2.6 kernel at least (it was in 2.4 too, but was less important).

It has been a goal of the libata developers to become a full fledged block driver for all these years, but I don't think they've made much progress on that. I suspect it's too much work for little gain.

== Now with that basic info in place, back to your original desire:

"I want to write module that implements SCSI-disk logic, keeping its data on block device."

To me, that sounds like libata. Libata is a kernel module that sits under the SCSI upper/mid layers and translates scsi commands to ATA commands and then sends those commands to a ATA controller (both SATA and PATA controllers are supported).

But, it sounds like you want to be able to issue your writes as SCSI requests? Is that right? If so, that almost sounds like iSCSI with a built-in loopback network.

Maybe you can elaborate using more specific terminology?

FYI: If I were you, I would want to create a new DM (device mapper) module that could tie in directly at the block layer of the linux kernel. It would be able to accept block i/o commands from the block layer, manipulate those commands, then re-issue them to the block layer so they could be pushed down into the actual block stack (ie. SCSI).

All the infrastructure is already in place to do that.

Thus, while you say you want to write a SCSI module, I think a Device Mapper module makes far more sense to do what you want. And you're not forcing your concept on to the kernel. Instead your taking the kernel's design and leveraging it.

Hope that helps Greg