Supports processor-to-processor communications through the SCSI target-mode device driver.
#include </usr/include/sys/devinfo.h>
#include </usr/include/sys/tmscsi.h>
#include </usr/include/sys/scsi.h>The Small Computer Systems Interface (SCSI) target-mode device driver provides an interface to allow processor-to-processor data transfer using the SCSI send command. This single device driver handles both SCSI initiator and SCSI target mode roles.
The user accesses the data transfer functions through the special files /dev/tmscsi0.xx, /dev/tmscsi1.xx, ... . These are all character special files. The xx can be either im, initiator-mode interface, or tm, target-mode interface. The initiator-mode interface is used by the caller to transmit data, and the target-mode interface is used to receive data.
The least significant bit of the minor device number indicates to the device driver which mode interface is selected by the caller. When the least significant bit of the minor device number is set to a value of 1, the target-mode interface is selected. When the least significant bit is set to a value of 0, the initiator-mode interface is selected. For example, tmscsi0.im should be defined as an even-numbered minor device number to select the initiator-mode interface, and tmscsi0.tm should be defined as an odd-numbered minor device number to select the target-mode interface.
When the caller opens the initiator-mode special file a logical path is established, allowing data to be transmitted. The user-mode caller issues a write, writev, writex, or writevx system call to initiate data transmission. The kernel-mode user issues an fp_write or fp_rwuio service call to initiate data transmission. The SCSI target-mode device driver then builds a SCSI send command to describe the transfer, and the data is sent to the device. Once the write entry point returns, the calling program can access the transmit buffer.
When the caller opens the target-mode special file a logical path is established, allowing data to be received. The user-mode caller issues a read, readv, readx, or readvx system call to initiate data reception. The kernel-mode caller issues an fp_read or fp_rwuio service call to initiate data reception. The SCSI target-mode device driver then returns data received for the application.
The SCSI target mode device driver allows access as an initiator mode device through the write entry point. Target mode device access is made through the read entry point. Simultaneous access to the read and write entry points is possible by using two separate processes, one running read subroutines and the other running write subroutines.
The SCSI target mode device driver does not implement any protocol to manage the sending and receiving of data, with the exception of attempting to prevent an application from excessive received-data buffer usage. Any protocol required to maintain or otherwise manage the communications of data must be implemented in the calling program. The only delays in sending or receiving data through the target mode device driver are those inherent to the hardware and software driver environment.
When the tmscsi0 special file is configured, both the tmscsi0.im and tmscsi0.tm special files are created. An initiator-mode/target-mode pair for each device instance should exist, even if only one of the modes is being used. The target-mode SCSI ID for an attached device should be the same as the initiator-mode SCSI ID, but the logical unit number (LUN) is ignored in target mode, because the host SCSI adapter can only respond as LUN 0.
If multiple LUNs are supported on the attached initiator device, a pair of tmscsin special files (where n is the device instance) are generated for each SCSI ID/LUN combination. The initiator-mode special files allow simultaneous access to the associated SCSI ID/LUN combinations. However, only one of the target-mode special files for this SCSI ID can be opened at one time. This is because only one LUN 0 is supported on the host adapter and only one logical connection can be actively using this ID at one time. If a target-mode special file is open for a given SCSI ID, attempts to open other target-mode special files for the same ID will fail.
The target-mode device driver configuration entry point must be called only for the initiator-mode device number. The driver configuration routine automatically creates the configuration data for the target-mode device minor number based on the initiator-mode data.
The target-mode device driver supports the open, close, read, write, select, and ioctl subroutines.
open Subroutine
The open subroutine allocates and initializes target or initiator device-dependent structures. No SCSI commands are sent to the device as a result of running the open subroutine.
The SCSI initiator or target-mode device must be configured and not already opened for that mode for the open subroutine to work. For the initiator-mode device to be successfully opened, its special file must be opened for writing only. For the target-mode device to be successfully opened, its special file must be opened for reading only.
Possible return values for the errno global variable include:
| Value | Description |
|---|---|
| EAGAIN | Lock kernel service failed. |
| EBUSY | Attempted to execute an open for a device instance that is already open. |
| EINVAL | Attempted to execute an open for a device instance using an incorrect open flag, or device is not yet configured . |
| EIO | An I/O error occurred. |
| ENOMEM | The SCSI device is lacking memory resources. |
close Subroutine
The close subroutine deallocates resources local to the target device driver for the target or initiator device. No SCSI commands are sent to the device as a result of running the close subroutine. Possible return values for the errno global variable include:
| Value | Description |
|---|---|
| EINVAL | Attempted to execute a close for a device instance that is not configured. |
| EIO | An I/O error occurred. |
read Subroutine
The read subroutine is supported only for the target-mode device. Data scattering is supported through the user-mode readv or readvx subroutine, or the kernel-mode fp_rwuio service call. If the read subroutine is unsuccessful, the return value is set to a return value of -1, and the errno global variable is set to the return value from the device driver. If the return value is something other than -1, then the read was successful and the return code indicates the number of bytes read. This should be validated by the caller. File offsets are not applicable and are therefore ignored for target-mode reads.
SCSI send commands provide the boundary for satisfying read requests. If more data is received in the send command than is requested in the current read operation, the requested data is passed to the caller, and the remaining data is retained and returned for the next read operation for this target device. If less data is received in the send command than is requested, the received data is passed for the read request, and the return value indicates how many bytes were read.
If a send command has not been completely received when a read request is made, the request blocks and waits for data. However, if the target device is opened with the O_NDELAY flag set, then the read does not block; it returns immediately. If no data is available for the read request, the read is unsuccessful and the errno global variable is set to EAGAIN. If data is available, it is returned and the return value indicates the number of bytes received. This is true even if the send command for this data has not ended.
If the calling program wishes to break a blocked read subroutine, the program can generate a signal. The target-mode device driver receives the signal and ends the current read subroutine with failure. The errno global variable is then set to EINTR. The read returns with whatever data has been received, even if the send command has not completed. If and when the remaining data for the send command is received, it is queued, waiting for either another read request or a close. When the target receives the signal and the current read is returned, another read can be initiated or the target can be closed. If the read request that the calling program wishes to break completes before the signal is generated, the read completes normally and the signal is ignored.
The target-mode device driver attempts to queue received data ahead of requests from the application. A read-ahead buffer area (whose length is determined by the product of 4096 and the num_bufs attribute value in the configuration database) is used to store the queued data. As the application program executes read subroutines, the queued data is copied to the application data buffer and the read-ahead buffer space is again made available for received data. If an error occurs while copying the data to the caller's data buffer, the read fails and the errno global variable is set to EFAULT. If the read subroutines are not executed quickly enough, so that almost all the read-ahead buffers for the device are filled, data reception will be delayed until the application runs a read subroutine again. When enough area is freed, data reception is restored from the device. Data may be delayed, but it is not lost or ignored. If almost all the read-ahead buffers are filled, status information is saved indicating this condition. The application may optionally query this status through the TMIOEVNT operation. If the application uses the optional select/poll operation, it can receive asynchronous notification of this and other events affecting the target-mode instance.
The target-mode device driver handles only received data in its read entry point. All other initiator-sent SCSI commands are handled without intervention by the target-mode device driver. This also means the target-mode device driver does not directly generate any SCSI sense data or SCSI status.
The read entry point may optionally be used in conjunction with the select entry point to provide a means of asynchronous notification of received data on one or more target devices.
Possible return values for the errno global variable include:
| Value | Description |
|---|---|
| EAGAIN | Indicates a non-blocking read request would have blocked, because no data is available. |
| EFAULT | An error occurred while copying data to the caller's buffer. |
| EINTR | Interrupted by a signal. |
| EINVAL | Attempted to execute a read for a device instance that is not configured, not open, or is not a target-mode minor device number. |
| EIO | I/O error occurred. |
write Subroutine
The write entry point is supported only for the initiator-mode device driver. The write entry point generates a single SCSI send command in response to a calling program's write request. If the write request is for a length larger than the host SCSI adapter's maximum transfer length or if the request cannot be pinned as a single request, then the write request fails with the errno global variable set to EINVAL. The maximum transfer size for this device is discovered by issuing an IOCINFO ioctl call to the target-mode device driver.
Some target mode capable adapters support data gathering of writes through the user_mode writev or writevx subroutine or the kernel-mode fp_wruio service call. The write buffers are gathered so that they are transferred, in order, as a single send command. The target-mode device driver passes information to the SCSI adapter device driver to allow it to perform the gathered write. Since the SCSI adapter device driver can be performing the gather function in software (when the hardware does not directly support data gathering), it is possible for the function to be unsuccessful because of a lack of memory or a copy error. The returned errno global variable is set to ENOMEM or EFAULT. Due to how gathered writes are handled, it is not possible for the target-mode device driver to perform retries. When an error does occur, the caller must retry or otherwise recover the operation.
If the write operation is unsuccessful, the return value is set to -1 and the errno global variable is set to the value of the return value from the device driver. If the return value is a value other than -1, the write operation was successful and the return value indicates the number of bytes written. The caller should validate the number of bytes sent to check for any errors. Since the entire data transfer length is sent in a single send command, a return code not equal to the expected total length should be considered an error. File offsets are not applicable and are ignored for target-mode writes.
If the calling program needs to break a blocked write operation, a signal should be generated. The target-mode device driver receives the signal and ends the current write operation. A write operation in progress fails, and the errno global variable is set to EINTR. The calling program may then continue by issuing another write operation, an ioctl operation, or may close the device. If the write operation the caller attempts to break completes before the signal is generated, the write completes normally and the signal is ignored.
The target-mode device driver automatically retries (up to the number of attempts specified by the value TM_MAXRETRY defined in the /usr/include/sys/tmscsi.h file) the send command if either a SCSI Busy response or no device response status is received for the command. By default, the target mode device driver delays each retry attempt by approximately two seconds to allow the target device to respond successfully. The caller can change the amount of time delayed through the TMCHGIMPARM operation. If retries are exhausted and the command is still unsuccessful, the write fails. The calling program can retry the write operation or perform other appropriate error recovery. All other error conditions are not retried but are returned with the appropriate errno global variable.
The target-mode device driver, by default, generates a time-out value, which is the amount of time allowed for the send command to complete. If the send command does not complete before the time-out value expires, the write fails. The time-out value is based on the length of the requested transfer, in bytes, and calculated as follows:
timeout_value = ((transfer_length / 65536) +1) *
10In the calculation, 10 is the default scaling factor used to generate the time-out value. The caller can customize the time-out value through the TMCHGIMPARM operation.
One of the errors that can occur during a write is a SCSI status of check condition. A check-condition error requires a SCSI request sense command to be issued to the device. This returns the device's SCSI sense data, which must be examined to discover the exact cause of the check condition. To allow the target-mode device driver to work with a variety of target devices when in initiator mode, the device driver does not evaluate device sense data on check conditions. Therefore, the caller is responsible for evaluating the sense data to determine the appropriate error recovery. The TMGETSENS operation is provided to allow the caller to get the sense data. A unique errno global variable, ENXIO, is used to identify check conditions so that the caller knows when to issue the TMGETSENS operation. This error is not logged in the system error log by the SCSI device driver. The writer of the calling program must be aware that according to SCSI standards, the request sense command must be the next command received by the device following a check-condition error. If any other command is sent to the device by this initiator, the sense data is cleared and the error information lost.
After each write subroutine, the target-mode device driver generates the appropriate return value and errno global variable. The device driver also updates a status area that is kept for the last command to each device. On certain errors, as well as successful completions, the caller may optionally read this status area to get more detailed error status for the command. The TMIOSTAT operation can be used for this purpose. The errno global variables covered by this status include EIO, EBUSY, ENXIO, and ETIMEDOUT.
Other possible return values for the errno global variable include:
| Value | Description |
|---|---|
| EBUSY | SCSI reservation conflict detected. Try again later or make sure device reservation is ended before proceeding. |
| EFAULT | This is applicable only during data gathering. The write operation was unsuccessful due to a kernel service error. |
| EINTR | Interrupted by signal. |
| EINVAL | Attempted to execute a write operation for a device
instance that is not configured, not open, or is not an initiator-mode
minor device number. Transfer length too long, or could not pin entire transfer. Try command again with a smaller transfer length. |
| EIO | I/O error occurred. Either an unreproducible error occurred or retries were exhausted without success on an unreproducible error. Perform appropriate error recovery. |
| ENOCONNECT | Indicates a SCSI bus fault has occurred. The caller should respond by retrying with asynchronous data transfer allowed. This is accomplished by issuing a TMIOASYNC operation to this device prior to the retry. If more than one retry is attempted, the TMIOASYNC operation should be performed only before the last retry. |
| ENOMEM | This is applicable only during data gathering. The write operation was unsuccessful due to lack of system memory. |
| ENXIO | SCSI check condition occurred. Execute a TMGETSENS operation to get the device sense data and then perform required error recovery. |
| ETIMEDOUT | The command has timed out. Perform appropriate error recovery. |
ioctl Subroutine
The following ioctl operations are provided by the target-mode device driver. Some are specific to either the target-mode device or the initiator-mode device. All require the respective device instance be open for the operation run.
| Operation | Description |
|---|---|
| IOCINFO | Returns a structure defined in the /usr/include/sys/devinfo.h file. |
| TMCHGIMPARM | Allows the caller to change certain parameters used by the target mode device driver for a particular device instance. |
| TMGETSENS | Runs a SCSI request sense command and returns the sense data to the user. |
| TMIOASYNC | Allows succeeding initiator-mode commands to a particular target-mode device to use asynchronous data transfer. |
| TMIOCMD | Sends SCSI commands directly to the attached device. |
| TMIOEVNT | Allows the caller to query the device driver for status on certain events. |
| TMIORESET | Sends a Bus Device Reset message to an attached target-mode device. |
| TMIOSTAT | Allows the caller to get detailed status information about the previously-run write or TMGETSENS ioctl operation. |
select Entry Point
The select entry point allows the caller to know when a specified event has occurred on one or more target-mode devices. The events input parameter allows the caller to specify which of one or more conditions it wants to be notified of by a bitwise OR of one or more flags. The target-mode device driver supports the following select events:
| Event | Description |
|---|---|
| POLLIN | Check if received data is available. |
| POLLPRI | Check if status is available. |
| POLLSYNC | Return only events that are currently pending. No asynchronous notification occurs. |
An additional event, POLLOUT, is not applicable and therefore is not supported by the target-mode device driver.
The reventp output parameter points to the result of the conditional checks. A bitwise OR of the following flags can be returned by the device driver:
| Flag | Description |
|---|---|
| POLLIN | Received data is available. |
| POLLPRI | Status is available. |
The chan input parameter is used for specifying a channel number. This is not applicable for non-multiplexed device drivers and should be set to a value of 0 for the target-mode device driver.
The POLLIN event is indicated by the device driver when any data is received for this target instance. A non-blocking read subroutine, if subsequently issued by the caller, returns data. For a blocking read subroutine, the read does not return until either the requested length is received or the send command completes, whichever comes first.
The POLLPRI event is indicated by the device driver when an exceptional event occurs. To determine the cause of the exceptional event, the caller must issue a TMIOEVNT operation to the device reporting the POLLPRI event.
The possible return value for the errno global variable includes:
| Value | Description |
|---|---|
| EINVAL | A specified event is not supported, or the device instance is either not configured or not open. |
Error Logging
Errors detected by the target-mode device driver can be one of the following:
The target-mode device driver passes error-recovery responsibility for most detected errors to the caller. For these errors, the target-mode device driver does not know if this type of error is permanent or temporary. These types of errors are logged as temporary errors.
Only errors the target-mode device driver can itself recover through retries can be determined to be either temporary or permanent. The error is logged as temporary if it succeeds during retry (a recovered error) or as permanent if retries are unsuccessful (an unrecovered error). The return code to the caller indicates success if a recovered error occurs or failure if an unrecovered error occurs. The caller can elect to retry the command or operation, but the probability of retry success is low for unrecovered errors.