STUC on You

(Originally published at Mac Edition)

(2024 update: this is mainly here to describe some of the most fun I’ve had with computers, digging into more hardware-level stuff and exercising my C skillZ. Also probably the first long-form blog posts I’ve written. I’ve done little editing outside of what Porruka has already done)

Early in the summer of 2003, I was contacted for a small contracting gig involving pulling data off of SCSI DAT tapes that were made from some other unix systems. It turns out that Mac OS X does not have a SCSI tape driver, so the standard unix commands like mt and tar don’t exist or can’t get the data off of the tape. The company was porting an application to Mac OS X and needed to be able to retrieve the saved data from the older version of the product, so it was important that their installed base can get their existing data into the new system.

Since I had never done anything like this before, I asked some of my friends at Apple the very sophisticated question of “uh, how would you do this?”. Turns out there were a couple of options. The first option was to write a SCSI tape driver from scratch or to port the FreeBSD driver over. This would require kernel programming (Kernel programming is hard! Let’s go shopping!), a kernel extension, plus wrapping my mind around the IOKit C++ frameworks on short notice. That sounded painful.

The other option was to use STUC, the SCSI Toolkit User Client. This is a toolkit that can be used in user-land application (as opposed to kernel-land). Why is that important? If you have a pointer error in a user program, the program will crash. Big deal. If you have a pointer error in kernel code, you could (at best) panic the machine, and (at worst) corrupt some kernel data structure and really mess things up.

STUC is an API that does the talking to the kernel drivers to actually communicate with the SCSI device, but the program manipulating the device can live entirely in user-land and not be concerned with the perils of kernel programming. Granted, you can still hose your machine and require a reboot to clear things up, but that happens a lot less frequently than when you’re mucking around with the kernel.

Naturally I took the STUC route, which is what I’ll be babbling about here. Apple isn’t unique in having a user-land SCSI library, but it’s still very nice not having to mess around in the kernel.

The actual job I was tasked to do was pretty simple: read data off of the tape and send it to a file for processing by another program, which someone else was responsible for. That’s it. I didn’t have to parse tar headers, or deal with anything regarding writing to the tape. I just had to read files, and also skip to the next file on the tape. And since I’ve been spending a lot of time lately in Cloud-Cocoa Land, I figured a a change of pace would be fun.

Preliminaries To Use STUC, you’ll want to download the SDK from Apple and install it. You can find it ftp://ftp.apple.com/developer/Development_Kits/SCSITaskUserClientSDK.pkg.sit.bin (well, you used to be able to find it there) The SDK adds some examples to /Developer/Examples/IOKit/scsi-commands/; a Carbon sample and two Cocoa samples. I relied heavily on the UserClientTester sample to get me up and running. To follow along at home, you’ll also need a DAT tape drive from somewhere, but the general techniques shown here apply to any SCSI device. DAT drives are kind of expensive and archaic, especially in today’s world of CD and DVD burners (2024 update: laughing here. Kids These Days “what’s a DVD?”). They’re also pretty easy to deal with. I was lucky to have one that we used doing backups of our OS 9 machines for the last couple of years.

You’ll need to hook the drive up to your computer. I’ve used both the Orange Micro SCSI/Firewire bridge to plug the DAT drive into my laptop, and also used an Adaptec card in one of the wind-tunnel G4s. Even though some of the Apple documentation implies that STUC only works across firewire or USB, the toolkit work fine when dealing with SCSI cards.

When attaching the drive, I usually had to totally power down the computer, hook up the cable, power on the drive, and then start the computer to have it be recognized by the system when using the SCSI/Firewire bridge. After restarting your machine, run Apple’s UserClientTester sample, or use the Apple System Profiler to make sure the computer is seeing the device. Otherwise you could burn some time bashing your head wondering why nothing is working, not that I actually did that.

Getting the Device Before you can command the SCSI device to do your bidding, your program has to actually find the device. First off, open a connection to the kernel so you can use IOKit calls that poke around its guts.

kern_return_t result;
mach_port_t masterPort;
result = IOMasterPort (MACH_PORT_NULL, &masterPort);

If the result isn’t KERN_SUCCESS, something horrible happened and you can’t continue. For doing low(ish)-level programming like this, be sure to check the return codes from all of your calls. You never know when a piece of hardware will flake out on you and start returning errors. You’d like to catch this condition as early as possible. Next make two CFDictionaries. One is a “matching” dictionary which the IOKit will use to match against existing hardware services it knows about. Inside of the Matching dictionary is a Property dictionary that contains properties to use when looking for specific OS services.

Into the Matching dictionary, add a key for the device category with the value for STUC:

CFDictionarySetValue (propertyDict,
                      CFSTR (kIOPropertySCSITaskDeviceCategory),
                      CFSTR (kIOPropertySCSITaskUserClientDevice));

This means to only match against STUC devices, so it won’t pick up any modems or mice or anything like that.

Stick the sequential access peripheral device type (fancy name for “tape”) into the Property dictionary. For those curious, this is SCSI device type 0x01. Other devices are things like 0x02 for printers, 0x05 for CD-ROM, and 0x06 for scanners.

    UInt8 peripheralDeviceType;
    peripheralDeviceType 
        = kINQUIRY_PERIPHERAL_TYPE_SequentialAccessSSCDevice;

    // make an CFNumber to wrap the integer
    CFNumberRef     peripheralDeviceTypeRef = NULL;
    peripheralDeviceTypeRef =
        CFNumberCreate (kCFAllocatorDefault,
                        kCFNumberCharType,
                        &peripheralDeviceType);

    // specify what device type we want
    CFDictionarySetValue (propertyDict,
                          CFSTR (kIOPropertySCSIPeripheralDeviceType),
                          peripheralDeviceTypeRef);

    CFRelease (peripheralDeviceTypeRef);

Since this is a number being put into a dictionary it gets wrapped in a CFNumber first. Notice that SSC in the name of the constant? That’s for “SCSI Stream Commands”, one of the SCSI standards documents that describes the commands and behaviors specific to streaming devices like tapes and printers. The rather odd combination of tapes and printers is that they are considered to be streaming devices, accepting a stream of bytes, as opposed to a true random access device, and so they both share the same standards document.

Finally add the Property dictionary to the Matching dictionary.

CFDictionarySetValue (matchingDict,
                      CFSTR (kIOPropertyMatchKey),
                      propertyDict);

Now take the Matching dictionary and ask IOServiceGetMatchingServices to look for matching devices. This function examines the dictionary to see what it is you want, and then returns (via a pointer argument) an iterator you can use to walk through the available devices. Or you can just pick off the first one, which I do here.

kern_return_t result;
result = IOServiceGetMatchingServices (masterPort,
                                       matchingDict,
                                       &serviceIterator);
if (result != KERN_SUCCESS) {
    // handle any errors
}
io_service_t service;
service = IOIteratorNext (serviceIterator)

So now you have a service which you can then use to interact with the device, like getting class name of the service.

io_name_t className;  // char[128]
result = IOObjectGetClass (service, className);

You can use the service to get a plug-in interface for the device.

Plug it in, Plug it in

First a digression about STUC and plug-ins. A good deal of the objects we’ll be dealing with are based on CFPlugInCOM, which is a struct-and-function-pointer way of doing object-oriented programming. There are a couple of places where you ask for a plug-in interface and the underlying system gives you back a structure, much like a dispatch table, with a bunch of function pointers filled in for you. You can then jump through these function pointers to get your work done. As implied by the name, the plug-ins use some of Microsoft’s COM technology, so you may occasionally see an un-Mac-like data type sneak in here and there. For the terminally curious, check out the header file for the gory details.

The “base class” is CFPlugInCOM which has three function pointers: QueryInterface (which we’ll use later), AddRef and Release for doing reference counting. Another class we’ll use is IOCFPlugInInterface, which has everything CFPlugInCOM has, plus function pointers for Probe, Start, and Stop (which we won’t be using) There’s also a SCSITaskDeviceInterface and a SCSITaskInterface in the mix which add yet more features.

So, we have a service that represents a device, and we want the IOCFPlugInInterface for that service so we can actually use it.

IOCFPlugInInterface **plugInInterface;
SInt32 score;

result = IOCreatePlugInInterfaceForService 
    (service,
     kIOSCSITaskDeviceUserClientTypeID, // plugin type
     kIOCFPlugInInterfaceID,            // interface type
     &plugInInterface,                  // the interface
     &score);                           // the score

We tell the plug-in interface creator that we want a STUC plug-in, and we want a CFPlugIn flavor (I don’t know if STUC is available in different plug-in flavors, but ya gotta pass it). I’m also unsure of the score, what that really means. We now have an IOCFPlugInInterface, and the only thing we’re going to use this for is to use its QueryInterface to dig one more layer deeper and get a SCSITaskDeviceInterface.

SCSITaskDeviceInterface **scsiInterface;
result = (*plugInInterface)->
    QueryInterface (plugInInterface,
                    CFUUIDGetUUIDBytes (kIOSCSITaskDeviceInterfaceID),
                    (LPVOID *) &scsiInterface); // LPVOID is a void *

This latest plug-in interface adds function pointers to obtain and release exclusive access (to make sure we’re the only ones controlling the device), as well as a way to create a specific SCSI task, which is an individual operation using the device.

The syntax for vectoring through the plug-in is a little odd. These plug-ins are actually a pointer to a pointer to a structure in memory that holds the instance variables and the dispatch table of function pointers (think that three times fast). So to actually get to the dispatch table, you have to dereference twice. I like the look of (*plug-in)->FunctionPointer(args) better than the look of (**plug-in).FunctionPointer(args), which are equivalent operations.

OK, so now we have a handle to the device. Obtain exclusive access before doing anything.

IOReturn scsiResult;
scsiResult = (*scsiInterface)->ObtainExclusiveAccess (scsiInterface);

This will return kIOReturnSuccess if we have exclusive access, kIOReturnBusy if someone else has exclusive access, or some other return value. Now we can actually do stuff with the hardware.

My First Command

Now it’s time to command the device to do something. The first one will be the Rewind command since it’s one of the simplest. We’ll also need to be able to tell the drive to rewind once we start playing with reading and skipping. But even in the simple case, there are a couple of moving pieces involved with dealing with SCSI devices. The first is the CDB (not CDB, the children’s book), but CDB as in Command Descriptor Block)

The Command Descriptor Block is a chunk of bytes that is the command you’re shipping off to the device. The CDB is given to the device exactly as you construct it. The various standards documents have diagrams showing the bit-by-bit layout of the CDB for each individual SCSI command, and what the various parts mean. Here’s the diagram for Rewind:

The first byte is the opcode, which is the general command you want the device to do (rewind, read, skip a file, eject). For Rewind it’s a value of one. The “Verify” command uses an opcode of 0x13. Subsequent bytes are dependent on what the specific command is, like the number of bytes or blocks to read, or special one-bit flags that change a command. You need to be(come) comfortable with C bitwise operators to pack and unpack data. For Rewind, the only extra tweakage you can do is set the low-order bit of the second byte. (if IMMED is zero, the device won’t return any status until the operation completes. If it’s set to one, the device will return its status before the rewind finishes)

So, declare a chunk of memory for the CDB and zero it out

SCSICommandDescriptorBlock cdb;
memset (&cdb, 0, sizeof(cdb));

It’s always a good idea to zero out your memory when dealing with devices. With the CDB, any stray bits can radically change the meaning of the command, like erase the tape or generate an EMP pulse. It’s also a good idea to clear any read or write buffers before using them, in case there’s sensitive information left over from a previous I/O operation. As shown in the diagram, the opcode for rewind is 0x01 (which is only coincidentally the same as the 0x01 value for streaming devices). In Apple’s header files, there are a number of symbolic constants defined for many of the SCSI opcodes (like kSCSICmd_REWIND), but many of them are in #if 0 blocks, which basically comments them out, so in our code we have to use the naked 0x01. Set the opcode in the CDB:

cdb[0] = 0x01; // kSCSICmd_REWIND

We don’t have any additional flags to set, so the CDB is complete. Note this makes the IMMED flag zero, which implies a synchronous operation as far as rewinding is concerned

Now create a scsiTask, which is a single command/response session with the device:

SCSITaskInterface **task = NULL;
task = (*interface)->CreateSCSITask (interface);

if (task == NULL) {
    // give up
}

Here we asked the SCSITaskDeviceInterface for Yet Another Plugin, this time a SCSITaskInterface, which, you guessed it, has a number of function pointers that we’ll be using. First off, tell the task to use our CDB:

IOReturn ioresult;
ioresult = (*task)->SetCommandDescriptorBlock (task, cdb, kSCSICDBSize_6Byte);

The return value is kIOReturnSuccess if the call succeeded, other error codes if the call failed. The size parameter tells the task how many bytes of the CDB to use as actual command bytes. The SCSI spec tells you how many bytes of CDB each command takes (they’re usually 6, 10, 12, or 16 bytes large). Rewind takes a 6 byte CDB.

Now set the timeout if you want. The timeout is measured in milliseconds, with zero being forever. Be careful when using the infinite timeout, especially during development. It’s easy to make your SCSI device totally unavailable unless you reboot. Not that I ever did that either. I haven’t figured out also how to tell if a timeout actually happens short of looking at the system times, but there may be a mechanism for doing that. For the work I was doing, it wasn’t an interesting question.

ioresult = (*task)->SetTimeoutDuration (task, 10000);

Apple’s sample code sets an attribute on the task to move it to the head of the command queue. The sample code says that it “is optional”, but I’m willing to use any amount of magic to make sure things work correctly. This sends the command to the head of the queue:

ioresult = (*task)->SetTaskAttribute (task, kSCSITask_HEAD_OF_QUEUE);

Now get and clear out the task status and sense data. These two are channels that return success and diagnostic information.

The task status represents the completion of the task. The two most interesting values are kSCSITaskStatus_GOOD, which means the command completed successfully with no complaints. The other common one is kSCSITaskStatus_CHECK_CONDITION, which can have different meanings based on the command. A check condition can sometimes mean “the command completed successfully, but not exactly how you told me to do it”, which we’ll visit in a bit regarding tape block sizes. It could mean, “something exceptional happened”, like you’ve run out of data on the tape, or you’ve hit the end of a logical file. Or it could mean “aiiieeeee! Something horrible happened! RUN!”. The sense data has elaborations on these additional meanings.

The sense data is a block of 18 bytes that contains result information. It has things like a response code, a chunk of information from the command, and command specific information. Of particular interest is the SENSE_KEY field. The top three bits are flags that tell you if you’ve hit a file mark (a marker on the tape saying you’ve gotten a complete file), end of medium, or the ILI bit is set. (ILI stands for Incorrect Length Indicator, which is discussed regarding tape block sizes). The bottom 4 bits have a result code for things like the device not being ready, a medium error, illegal request, unit attention (the tape is on fire), and so on. You can pull apart the sense data to see what’s lurking inside. Like the CDB, make sure these guys are cleared to zero.

SCSITaskStatus taskStatus = 0;
SCSI_Sense_Data senseData;
memset (&senseData, 0x00, sizeof(senseData));

Now finally, we can send out the Marines and have the device run the command. Use the ExecuteTaskSync function pointer of our ScsiTask to execute the command. The function will return after the command completes, or if it times out. There exists an asynchronous version of this, but I didn’t have to use them since my tape-reading code is called from a shell script.

UInt64 transferCount = 0;
ioresult = (*task)->ExecuteTaskSync (task, 
                                     &senseData, 
                                     &taskStatus, 
                                     &transferCount);

The transferCount is a 64 bit integer that is the amount of data transferred, if applicable. There are two result codes involved in the call. The ioresult is a value like kIOReturnSuccess if the task of shipping a command off to the device and getting any reply worked. and other values on errors. The second result is the task status code, which is the success or failure inside of the SCSI device. Here just check it for kSCSITaskStatus_GOOD for the rewind.

Finally clean up any messes made by the command.

(*task)->Release (task);

And that is a complete SCSI command, which should result in your tape being rewound.

Baby got data back

Rewinding tapes are all well and dandy, but sometimes you want to get something from your device. The next command on our tour is the INQUIRY command, which asks the device for some information. Apple provides a SCSICmd_INQUIRY_StandardData structure with the various pieces of inquiry data. There are things like the peripheral device type, whether the media is removable, the SCSI versions it supports, and, the part of interest now are three identification strings: VENDOR_IDENTIFICATION, PRODUCT_INDENTIFICATION (yes, it’s misspelled indentification), and PRODUCT_REVISION_LEVEL.

Here are all the steps again for issuing the command, including the read-back of the data.

Make the cdb and zero it out

SCSICommandDescriptorBlock cdb;
memset (&convenienceCDB, 0, sizeof(cdb));

Set the opcode, and we actually have an Apple-supplied symbolic constant for this

cdb[0] = kSCSICmd_INQUIRY

Declare our incoming data buffer, clear it, and put into the CDB how much data we’re expecting to get back

SCSICmd_INQUIRY_StandardData inqBuffer;
memset (&inqBuffer, 0x0, sizeof(inqBuffer));
cdb[4] = sizeof (inqBuffer);

Only one byte is used for the size here, so at most 255 bytes can be returned by this command. Now create the task

SCSITaskInterface **task = NULL;
task = (*interface)->CreateSCSITask (interface);

Set the CDB in the task

IOReturn ioresult;
ioresult = (*task)->
        SetCommandDescriptorBlock (task, cdb, kSCSICDBSize_6Byte);

And now tell the command where to put the data. The task can take an array of “scatter/gather” entries, called IOVirtualRanges, which are just pairs of addresses and lengths for various buffers. You can supply several of these IOVirtualRanges if you want to read or write using several different chunklets of memory. Me, I go the simple route and just use one buffer:

IOVirtualRange range;
range.address = (IOVirtualAddress) &inqBuffer;
range.length = sizeof (inqBuffer);

ioresult = (*task)->
        SetScatterGatherEntries (task,
                                 &range,       // start of an array
                                 1,            // number of entries
                                 range.length, // transfer size
                                 kSCSIDataTransfer_FromTargetToInitiator);

The transfer size is the total size to transfer. If you have more than one IOVirtualRange, the sum of lengths needs to be at least the this total. The last argument is the direction of data transfer. Since my program is initiating the command, the data will flow from the target (the tape drive) to me.

Set the timeout, and go to the head of the class. Er, queue.

ioresult = (*task)->SetTimeoutDuration (task, 10000);
ioresult = (*task)->SetTaskAttribute (task, kSCSITask_HEAD_OF_QUEUE);

And send the commmand to the device

memset (&senseData, 0x00, sizeof(senseData));
taskStatus = 0;
transferCount = 0;
ioresult = (*task)->ExecuteTaskSync (task, 
                                     &senseData, 
                                     &taskStatus, 
                                     &transferCount);

So, we’ve now told the device to execute an INQUIRY command, and to put the results back into our inqBuffer variable. Now it’s time to look in there. The last three fields are the ones of interest, the space-delimited description strings. These strings have no trailing zero byte, which is why this code sticks some in just to keep printf() happy.

(2024 update: I’m not sure what the [-1] indexing is actually accomplishing. I’m guessing either I or an editor was thinking of python and indexing off the back of the array?)

(inqBuffer.VENDOR_IDENTIFICATION)[-1] = '\0';
printf ("vendor ID: %s\n",
        inqBuffer.VENDOR_IDENTIFICATION);

(inqBuffer.PRODUCT_INDENTIFICATION)[-1] = '\0';
printf ("product ID: %s\n",
        inqBuffer.PRODUCT_INDENTIFICATION); // [sic]

(inqBuffer.PRODUCT_REVISION_LEVEL)[-1] = '\0';
printf ("product rev level: %s\n",
        inqBuffer.PRODUCT_REVISION_LEVEL);

And lastly release the task since we’re done.

(*task)->Release (task);

Take it to the Limit

There is another command, 0x05, affectionately known as READ_BLOCK_LIMITS (there is an #ifdef’d out constant in Apple’s headers for this one too. sigh) This behaves a lot like the INQUIRY command in that it sends an opcode and a length value, and the command fills in a pre-supplied buffer with information.

In this case, it provides information on the maximum and minimum block sizes for the device. If they’re the same, the device only supports fixed block sizes (which is important when we start reading data from the tape), and if they’re different, the device supports variable sized blocks, meaning the first block on the tape could be 10K, the next block 5K, and so on.

The code for this command is just like INQUIRY, so I won’t talk about it here much, outside of showing you the structure of the returned data

struct BlockLimitData {
    UInt8       granularity;    // mask with 0x1F
    UInt8       max_block_length_msb;
    UInt8       max_block_length_middle;
    UInt8       max_block_length_lsb;
    UInt8       min_block_length_msb;
    UInt8       min_block_length_lsb;
} __attribute((packed));

The max length is a 24 bit number, and the min block length is a 16 bit number. You can use bit shifting to assemble these into integer variables for convenience:

    UInt32 maxLength;
    maxLength = 
        (inqBuffer.max_block_length_msb << 16)
        | (inqBuffer.max_block_length_middle << 8)
        | (inqBuffer.max_block_length_lsb);

    UInt32 minLength;
    minLength =
        (inqBuffer.min_block_length_msb << 8)
        | (inqBuffer.min_block_length_lsb);

Reading Is Fundamental

All this stuff I’ve been talking about are just preliminaries to the real action: reading data from the tape. This is where I spent a lot of time on trial and error, since the SCSI specs are rather dense, and I had two tape drives available to me which had different behaviors based on the same initial conditions.

The SCSI opcode is going to be 0x08, kSCSICmd_READ_6, which uses 6 bytes of the CDB. There are also kSCSICmd_READ_10 and kSCSICmd_READ_12 opcodes which have 10 and 12 byte CDBs, allowing the reading of more data from the drive in a single operation, plus some additional flags you can specify.

Inside of the CDB for the read there are two one-bit flags that can be set: the FIXED (fixed-block) flag, and the SILI (suppress ILI) flag. You’ll see those two a little later. Finally there are three bytes for the transfer length.

If we’re in fixed-block mode, which READ_BLOCK_LIMITS would tell us, the transfer length is the number of blocks to return. The actual (potential) return data is blocksize * transferLength. And if we are dealing with fixed-sized blocks, set the FIXED flag so the device can know that we know we’re dealing with fixed-sized blocks.

If we’re dealing with variable blocks, the transfer length is the number of bytes to bring in from the tape. From what I can tell, the tape hardware likes dealing with complete blocks, so it won’t let you read half a block now, and half a block later. Nor does it seem to support reading multiple blocks in one operation. It could be a limitation of the hardware I had available, or may just be The Way Things Are with DAT tape.

You be ILI

So if we’re dealing with variable block size, how do we know how much to ask for in a single block read? We don’t. Instead, ask for a block that should be larger than what’s expected on the tape. For my application, I chose a blocksize of 128K since the tapes were made under controlled circumstances with tar using a blocksize of 10K.

When the read returns and the block on tape was less than 128K, we’ll have a task status of CHECK_CONDITION, and then we look in the sense data for the ILI flag being set. ILI, again, stands for “Incorrect Length Indicator”, which in our case is a perfect description. We asked for an incorrect amount of data, so the command will read a block, stick it in our buffer, and return this CHECK_CONDITION with the ILI flag set.

If you Know What You Are Doing, you can set the SILI (Suppress Incorrect Length Indicator) flag in the CDB, which will cause the SCSI command to return a GOOD status if there’s enough room in our buffer, and tell us how much it read. Unfortunately, life isn’t so nice. With a Seagate drive the transfer length returned from ExecuteTaskSync was the size of the block actually returned, which is very nice behavior. The other mechanism, a Sony drive, with the SILI flag set, would report the transfer length to be the size of the buffer we asked for (128K) rather than the size of the tape block (10K), which caused the program to generate very incorrect results. So leave the suppress-ILI flag clear, and deal with the sense data. Use the READ_6 opcode:

cdb[0] = kSCSICmd_READ_6;

Unpack the block size, and spread it across 3 bytes

int blocksize = 128 * 1024;
cdb[2] = (blocksize >> 16) & 0xFF;
cdb[3] = (blocksize >> 8) & 0xFF;
cdb[4] = (blocksize) & 0xFF; 

Point the scatter/gather to our buffer

UInt8 readBuffer = malloc (blocksize);

IOVirtualRange range;
range.address = (IOVirtualAddress) readBuffer;
range.length = blocksize;

And dispatch the command as before and release the task.

Now assuming a good function result, look at the task status. We’ll probably have CHECK_CONDITION set. First look at the SENSE_KEY to see what’s set. For instance, if kSENSE_FILEMARK_Set is set, we’ve hit a file mark which tells us we’re done reading this file. (there very well could be more files on the tape). Another flag is kSENSE_EOM_Set, which means we’ve hit the end of media. The interesting flag is kSENSE_IsILI_Set, meaning the ILI flag is set, so we have some extra work.

if (taskStatus == kSCSITaskStatus_CHECK_CONDITION) {

    if (senseData.SENSE_KEY & kSENSE_FILEMARK_Set) {
        // handle the end-of-file set
    } else if (senseData.SENSE_KEY & kSENSE_EOM_Set) {
        // handle the end of media case
    } else if (senseData.SENSE_KEY & kSENSE_ILI_Set) {
        // handle the ILI case
    }
}

So what happens in the ILI case? There are four INFORMATION_[1-4] fields in the sense data. These form a 4-byte integer which contains the residual size of the read, which is a fancy name for the requested transfer length minus the actual block length. So subtract out the information value from the requested length to get the actual size of the data:

int information;
information = (  ((senseData->INFORMATION_1 << 24) & 0xFF000000)
                 | ((senseData->INFORMATION_2 << 16) & 0x00FF0000)
                 | ((senseData->INFORMATION_3 << 8) & 0x0000FF00)
                 | ((senseData->INFORMATION_4) & 0x000000FF) );
int actualLength = blocksize -information;

If actualLength is negative, meaning that we have too little space in the buffer, we’re out of luck. As far as I can tell, there’s not a way to re-read an already read block, even if that was a partial read.

Corner Cases

There are a couple of additional things that can go wrong which need to be handled. You’ve seen before the file mark and the end of medium indicator. There is also an End Of Data condition, which means we’ve run out of data, but not run out of tape. If you don’t handle it, you might end up reading to the end of the tape. To handle this, look at the SENSE_KEY, strip off the high bits, and see if it is the value 0x08, meaning “Blank Check”. Then look at the ADDITIONAL_SENSE_CODE to be 0x00 and ADDITIONAL_SENSE_CODE_QUALIFIER to be 0x05. These two fields are just a couple of bytes of additional data the device returns to tell you exactly what happened. These settings mean you’ve hit end of data.

int senseKey;
senseKey = senseData.SENSE_KEY & 0x0F;

int additionalSense;
additionalSense 
    = senseData.ADDITIONAL_SENSE_CODE << 8
      | senseData.ADDITIONAL_SENSE_CODE_QUALIFIER;

if (senseKey == 0x08 // Blank Check
    && additionalSense == 0x0005 // end of data
    ) {
    // yes, it is end of data
} 

Apple’s cocoa sample code has a function that turns these kinds of sense values into something readable. You can work backwards from this for the values of the sense code and sense code qualifier.

Another corner case is a condition that happens after you insert a tape. There is a window of time where the “unit is becoming ready” (presumably threading the tape), and a “not-ready to ready transition”, which is the device sniffing the tape. The unit becoming ready case is sense key of 0x02 (not ready) and additional sense of 0x04001, and the not-ready to ready transition has a sense key of 0x06 (unit attention) and additional sense of 0x2800. If you get these, wait a bit and try again.

Cleanup

After you’ve made all the SCSI requests you need, be sure to practice good programming hygiene and release any resources that got allocated, like freeing any allocated memory. You’ll also need to release the exclusive access you have on the device:

ioResult = (*scsiInterface)->ReleaseExclusiveAccess (scsiInterface);

And also dispose of the various plug-ins

(*scsiInterface)->Release (scsiInterface);
IODestroyPlugInInterface (plugInInterface);

And clean up the service

IOObjectRelease (service);

And finally tell the kernel we’re done with it for now.

mach_port_deallocate (mach_task_self(), masterPort);

You’ll notice I’m not totally freakish in checking return results for the clean-up code. If something happens during this “we’re totally done” kind of cleanup, there’s not a whole lot you can do to recover.

That’s All Folks

Doing SCSI programming wasn’t nearly as hard as I originally thought it would be. Yes, it was tedious slogging through the standards documents figuring out stuff, quite a bit of trial-and-error as I figured out how the moving pieces fit together, and sometimes I had to go through some reboot cycles when I did something dumb. But luckily since I was doing all my work as a regular application, I had many fewer reboots than if I was mucking around in the kernel. Now, I don’t think I’d want to do this kind of thing for a living (I prefer end-user application programming), but it was a fun diversion.

Resources

http://t10.org : Technical Committee T10’s web site has a ton of information on SCSI Storage Interfaces. Of particular interest are the Draft Standards and Techincal Reports link. These are draft version of the ANSI standards documents. You have to pay money for the final versions, but the drafts are free. The ones I used most often were “SSC”, for the SCSI-3 Stream Commands, and “SPC-2” the SCSI Primary Commands documents. “SAM-3”, the SCSI Architecture Model, has a lexicon, and pointers to other standards documents, depending on the kind of SCSI device you’re interested in. Be forewarned that these are specifications documents, so they’re pretty dry, dense reading.

ftp://ftp.apple.com/developer/Development_Kits/SCSITaskUserClientSDK.pkg.sit.bin : Apple’s STUC SDK, includes example code. (Currently MIA)

I found the source code to the Linux scsi tape driver to be helpful when figuring out how to approach some problems. You can get the Linux kernel source from http://kernel.org. After you download the Linux kernel source code, the tape driver lives in drivers/scsi/st.c. There is a Linux SCSI Programming HOWTO, which is pretty specific to Linux, but has some easier to digest information than the SCSI specs.

Apple hosted a mailing list, ata-scsi-dev, which is for Technical discussion for developers of devices based on ATA and SCSI.

Working with SAM, where SAM is the Scsi Architecture Model. This might be a replacement for STUC. I still haven’t found where the UserClientTester sample has escaped to.

• Programming