Simple question on

Hopefully a very quick question for anyone familiar with IOUserSCSIParallelInterfaceController - what is the right initialization to implement:

init () override;

This is basically "I exist". You can do things like create your core data structure, but you aren't attached to a specific driver (well, you know that you know about) and generally won't do any "active" configuration.

Start (IOService * provider) override;

Congratulations, you're a driver now! You now have your provider, which means you can talk to the kernel and start exploring the wider world. In virtually all drivers, this is where most of the actual "work" starts.

...and then the SCSI stack starts.

UserInitializeController () override;

This is your chance to get yourself "ready" to actually do something useful. The most critical thing is that you MUST get your queues set up as the documentation describes. Note this comment:

"For best results, use a model with three dispatch queues. DriverKit creates a default queue for you, and you’ll also need to create interrupt auxiliary queues. Because DriverKit dispatch queues are serial, this arrangement prevents calls from DriverKit, interrupts, and I/O work from competing with one another on the same thread."

The term "For best results..." is another way of saying "we don't expect any other way to work". More specifically, there are multiple places in SCSIControllerDriverKit where the kernel will reenter your DEXT while your DEXT is executing a different callback. If your DispatchQueue configuration or usage is wrong, you'll either deadlock your driver or panic the kernel.

One final note on DispatchQueue. Mostly because of the name and usage pattern, it's tempting to think of DispatchQueue in the same way you would user space dispatch queues and, following that line of thinking, assume that you could improve performance by using more queues to increase parallelism. While I'm sure there are cases where that might be true, I haven't seen one yet and I'm not expecting to anytime soon. A storage DEXT operates as one component within a much larger system and that larger system is what primarily defines your threading behavior, NOT your DEXT. More importantly, a storage driver is the definition of "I/O bound", which means the real performance gains are around how you process I/O request (sizes/counts/etc), not overhead of individual requests.

something else

...and then things get more complicated. I'm not going to try and outline that full flow, but the references I'd recommend are:

• The WWDC session introducing SCSIControllerDriverKit.

• The SCSIControllerDriverKit header doc.

• The open source release of the SCSIParallelFamily, particularly IOSCSIParallelInterfaceController.cpp.

While the open-source release does not include IOUserSCSIParallelInterfaceController (and please, do file a bug asking for us to open source it), IOSCSIParallelInterfaceController is the base class for IOUserSCSIParallelInterfaceController, and its implementation defines the general "flow" the driver uses. For example, "UserInitializeController" is simply the DEXT method the kernel calls inside... InitializeController(). Seeing how that flow works can be really helpful in understanding how your DEXT works.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Kevin, thank you again for your help (which both clarified this proper sequence here and in the other thread). The key for any others who may come to this thread later:

(1) the sequence for IOUserSCSIParallelInterfaceController is different than for other DriverKit drivers, because it implements a lot of functionality on top of what other device drivers (eg raw PCI) would implement

(2) the kernel uses Info.plist properties of the DEXT along with what's in the binary to decide what methods to call in what sequence - if these are not exactly right, you may get strange calls during the initialization sequence, and MacOS itself may get confused and panic or otherwise malfunction

Yes, we need to file the bug re: "the open-source release does not include IOUserSCSIParallelInterfaceController (and please, do file a bug asking for us to open source it), IOSCSIParallelInterfaceController is the base class for IOUserSCSIParallelInterfaceController" and have others ask for the same, as it would save a lot of time.

(1) The sequence for IOUserSCSIParallelInterfaceController is different than for other DriverKit drivers, because it implements a lot of functionality on top of what other device drivers (eg raw PCI) would implement.

Very true, plus parts of its architecture are inherently reentrant. For example, when your controller reports a new device, what's basically happening is:

1. Your DEXT calls into the kernel saying "hey, there's a new device".

2. The kernel creates and attaches a new storage device onto your in-kernel driver.

3. That new driver starts its own initialization, eventually calling back into its provider (your in-kernel driver).

4. The in-kernel driver calls BACK out to your DEXT to handle #3.

5. A DEXT method gets called in your driver.

6. 3->5 repeat multiple times as the new device moves through its process.

7. Device initialization completes and #1 returns.

This kind of issue is what the dispatch architecture described in UserCreateTargetForID is trying to manage.

And, yes, this issue is fairly unique to SCSIController DriverKit. Most drivers are simply responding to the commands they receive from "above" them, not actively creating/managing the same layer that's sending them commands.

Yes, we need to file the bug

Bug number?

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware