MIO_open Subroutine

Purpose

Opens a file for reading or writing through the MIO library.

Library

Modular I/O library (libmio.a)

Syntax

#include <libmio.h> 
int MIO_open (Path, OFlag, Mode, Extra)
char *Path;int OFlag;
int Mode;
struct mio_extra *Extra;

Description

The MIO_open subroutine is a redirection to the MIO_open64 subroutine and is an entry point of the MIO library. To use the MIO library, the files have to be opened with the O_LARGEFILE flag. For more details on the O_LARGEFILE flag, see the fcntl.h File.

Use the MIO_open subroutine to instrument your application with the MIO library. You can replace the open kernel I/O subroutine with this equivalent MIO subroutine. See the Modular I/O in AIX® Version 7.1 Performance management for the MIO library implementation.

Use this subroutine to open a file through the Modular I/O (MIO) library. This library creates the context for this open file, according to the configuration set in the MIO environment variables, or in the Extra parameter.

To analyze your application I/O and tune the I/O, use the MIO subroutines in the place of the standard I/O subroutines.

The MIO subroutines are: The standard I/O subroutines are:

Parameters

Item Description
Extra Specifies additional arguments for the MIO library. The simplest implementation is to pass a zero pointer as the fourth argument. The fourth argument is a pointer to the mio_extra structure, you can usually pass a zero pointer, but you can also pass an mio_extra pointer (use this technique only if you are very familiar with how to code this argument).
The mio_extra structure is defined as follows: 
struct mio_extra {
                  int   cookie ;
                  /* Default value:  MIO_EXTRA_COOKIE/

                  int   taskid ; 
                  /*  for later */

                  int64 bufsiz ;
                  /* if > 1 : force the prefetch for write pffw */

                  char *modules ; 
      /* explicit module name, 
       if any modules returns from MIO_FILES environment variable match */

                  char *logical_name ;
      /* logical file name to open 
       if file name don't match with MIO_FILES regexp */                        
                  
                  int   flags ; 
      /* if MIO_EXTRA_SKIP_MIO_FILES_FLAG : 
       don't use MIO_FILES env variable, but use extra->modules */
                   } ;
Mode Specifies the modes. For more information, see the Mode flag in the open64 subroutine.
Oflag Specifies the type of access, the special open processing, the type of update, and the initial state of the open file. For more information, see the open64 subroutine.
Path Specifies the file to be opened.
Note: For applications that would not use the environment variable interface to apply MIO modules to a file, the mio_extra hook provides an easy way to do that.

Environment variables

MIO is controlled through the following four environment variables. These environment variables, which define the MIO features, are processed by the MIO_open64 subroutine.

The MIO_STATS variable is used to indicate a file that will be used as a repository for diagnostic messages and for output requested from the MIO modules. It is interpreted as a file name with two special cases. If the file is either thestderr or stdout output, the output will be directed towards the appropriate file stream. If the first character of the MIO_STATS variable is a plus sign (+), the file name to be used is the string following the plus sign (+), and the file is opened for appending. Without the preceding plus sign (+), the file is overwritten.

The MIO_FILES variable is the key to determine which modules are to be invoked for a given file when the MIO_open64 subroutine is called. The format for the MIO_FILES variable is the following:

first_file_name_list [ module list ] second_file_name_list [ module list]

When the MIO_open64 subroutine is called, MIO checks for the existence of the MIO_FILES variable and parses it as follows:

The MIO_FILES variable is parsed left to right. All characters up to the next occurrence of the bracket ([) are taken as a file name list. A file name list is a colon-separated list of file name templates. A file name template is used to match the name of the file opened by MIO and can use the following wildcard characters:
*
Matches zero or more characters of a directory or file name.
?
Matches one character of a directory or file name.
**
Matches all remaining characters of a full path name.

If the file name template does not contain a forward slash (/) , then all of the path directory information in the file name passed to the MIO_open64 subroutine is ignored and matching is applied only to the file name of the file being opened.

If the name of the file being opened is matched by one of the file name templates in the file name list then the module list to be invoked is taken as the string between brackets ([ ]). If the name of the file match two or more file name templates, the first match is taken into account. If the name of the file being opened does not match any of the file name templates in any of the file name lists then the file is opened with a default invocation of the AIX module.

If a match has occurred, the modules to be invoked are taken from the associated module list in the MIO_FILES variable. The modules are invoked left to right, with the left-most being closest to the user program and the right-most being closest to the operating system. If the module list does not start with the MIO module, a default invocation of the MIO module is added as a prefix. If theAIX module is not specified, a default invocation of theAIX module is appended.

The following is an example of the MIO_FILES variable: 
setenv MIO_FILES " *.dat [ trace/stats ]"
Assume the MIO_FILES variable is set as follows: 
MIO_FILES= *.dat:*.scr [ trace ] *.f01:*.f02:*.f03 [ trace | pf | trace ]
If the test.dat file is opened by the MIO_open64 subroutine, the test.dat file name matches *.dat and the following modules are invoked: 
mio | trace | aix
If the test.f02 file is opened by the MIO_open64 subroutine, the test.f02 file name matches the second file name templates in the second file name list and the following modules are invoked: 
mio | trace | pf | trace | aix
Each module has its own hardcoded default options for a default invocation. You can override the default options by specifying them in the associated MIO_FILES module list. The following example turns on the stats option for the trace module and requests that the output be directed to the my.stats file: 
MIO_FILES= *.dat : *.scr [ trace/stats=my.stats ]

The options for a module are delimited with a forward slash (/). Some options require an associated string value and others might require an integer value. For those requiring a string value, if the string includes a forward slash (/), enclose the string in braces ({ }).

For those options requiring an integer value, append the integer value with a k, m, g, or t to represent kilo, mega, giga, or tera. You might also input integer values in base 10, 8, or 16. If you add a 0x prefix to the integer value, the integer is interpreted as base 16. If you add a 0 prefix to the integer value, the integer is interpreted as base 8. If you add neither a 0x prefix nor a 0 prefix to the integer value, the integer is interpreted as base 10.

The MIO_DEFAULTS variable is intended as a way to keep the MIO_FILES variable more readable. If the user is specifying several modules for multiple file name list and module list pairs, then the MIO_FILES variable might become quite long. To repeatedly override the hardcoded defaults in the same manner, you can specify new defaults for a module by specifying such defaults in the MIO_DEFAULTS variable. The MIO_DEFAULTS variable is a comma separated list of modules with their new defaults.

The following is an example of the MIO_DEFAULTS variable: 
setenv MIO_DEFAULTS " trace/kbytes "
Assume that MIO_DEFAULTS variable is set as follows: 
MIO_DEFAULTS = trace/events=prob.events , aix/debug

Any default invocation of the trace module will have binary event tracing enabled and directed towards the prob.events file and any default invocation of the AIX module will have debug enabled.

The MIO_DEBUG variable is intended as an aid in debugging the use of MIO. MIO searches the MIO_DEFAULTS variable for keywords and provides debugging output for the option. The available keywords are the following:

ALL
Turns on all of the MIO_DEBUG variable keywords.
ENV
Outputs environment variable matching requests.
OPEN
Outputs open requests made to the MIO_open64 subroutine.
MODULES
Outputs modules invoked for each call to the MIO_open64 subroutine.
TIMESTAMP
Places a timestamp preceding each entry into a stats file.
DEF
Outputs the definition table of each module. When the file opens, the outputs of all of the MIO library's definitions are processed for all the MIO library modules.

Return values

The return values are those of the corresponding standard POSIX system call open64.

Error codes

The error codes are those of the corresponding standard POSIX system call open64.

Standard output

There is no MIO library output for the MIO_open64 subroutine.

Note: MIO library output statistics are written in the MIO_close subroutine. This output filename is configurable with the MIO_STATS environment variable.

In the example.stats. MIO output file, the module trace is set and reported, and the open requests are output. All the values are in kilobytes.

Examples

The following example.c file issues 100 writes of 16 KB, seeks to the beginning of the file, issues 100 reads of 16 KB, and then seeks backward through the file reading 16 KB records. At the end the file is truncated to 0 bytes in length.

The filename argument to the following example is the file to be created, written to and read forwards and backwards:

--------------------------------------------------------------------------------
#define _LARGE_FILES
#include <fcntl.h>
#include <stdio.h>
#include <errno.h>

#include "libmio.h"

/* Define open64, lseek64 and ftruncate64,  not 
 * open, lseek, and ftruncate that are used in the code.  This is
 * because libmio.h defines _LARGE_FILES which forces <fcntl.h> to
 * redefine open, lseek, and ftruncate as open64, lseek64, and
 * ftruncate64
 */

#define open64(a,b,c) MIO_open64(a,b,c,0)
#define close         MIO_close
#define lseek64       MIO_lseek64
#define write         MIO_write
#define read          MIO_read
#define ftruncate64   MIO_ftruncate64

#define RECSIZE 16384
#define NREC    100

main(int argc, char **argv)
{
int i, fd, status ;
char *name ;
char *buffer ;
int64 ret64 ;

      if( argc < 2 ){
      fprintf(stderr,"Usage : example file_name\n");
      exit(-1);
   }
   name = argv[1] ;

   buffer = (char *)malloc(RECSIZE);
   memset( buffer, 0, RECSIZE ) ;

   fd = open(name, O_RDWR|O_TRUNC|O_CREAT, 0640 ) ;
   if( fd < 0 ){
      fprintf(stderr,"Unable to open file %s errno=%d\n",name,errno);
      exit(-1);
   }

/* write the file */
   for(i=0;i<NREC;i++){
      status = write( fd, buffer, RECSIZE ) ;
   }

/* read the file forwards */
   ret64 = lseek(fd, 0, SEEK_SET ) ;
   for(i=0;i<NREC;i++){
      status = read( fd, buffer, RECSIZE ) ;
   }
/* read the file backwards */
   for(i=0;i<NREC;i++){
      ret64 = lseek(fd, (NREC-i-1)*RECSIZE, SEEK_SET ) ;
      status = read( fd, buffer, RECSIZE ) ;
   }

/* truncate the file back to 0 bytes*/
   status = ftruncate( fd, 0 ) ;

   free(buffer);

/* close the file */
   status = close(fd);
}

--------------------------------------------------------------------------------
Both a script that sets the environment variables, compiles and calls the application and the example.c example are delivered and installed with the libmio, as follows: 
cc -o example example.c  -lmio 

./example file.dat

The following environment variables are set to configure MIO:

setenv MIO_STATS example.stats
setenv MIO_FILES " *.dat [ trace/stats ] "
setenv MIO_DEFAULTS " trace/kbytes "
setenv MIO_DEBUG OPEN

See the /usr/samples/libmio/README and sample files for details.

Location

/usr/lib/libmio.a