OpenVMS error reporting

[zoli]

Hello,

this question is directed to Mike, but I think the answer will be useful for everybody.

Earlier I had direct access to Scandinavian OpenVMS call center - but lately I do not work with OpenVMS directly, therefore I lost this hot wire

I have faced a weired C related problem and I am convinced that this is a C RTL (run time library) bug.

(just in short - reading an RMS file with an unusual organization fail while using C functions like open/read but works perfect while using sys$qio calls)

The question is where can I submit a such, very much development related bug?

Thank you in advance.

[miker_alpha]

Lets see if I understood the question...
Languages like COBOL BASIC and DCL on OpenVMS have intrinsic (built-in) support for RMS file structures, the most useful (in my opinion) are indexed files, which are conceptually similar to ISAM files (Indexed Sequential Access Method - an IBM acronym)

C -- at least in the versions I knew -- has no support for RMS file structures. So to do file access to not-a-stream-of-bytes files you have to do the equivalent of peeking and poking into the file control structures -- the details of which are a: unknown to me and b: very hairy
(There are definitely examples online, and probably in the SYS$EXAMPLES directory)

C does not have a happy solution for record-based IO
On the other hand, linking to modules in a record-oriented language for a few basic functions (Open, Find, Read, Update) using a specific file mapping (i.e. not trying to solve a generic problem, just for a specific file) shouldn't be too difficult.

Pretty good example using RMS from C at
http://h71000.www7.hp.com/wizard/wiz_0367.html

What is the original problem?
What is an "unusual organization"?

Two places where OpenVMS gurus hang out on the net are
NNTP newsgroup news://comp.os.vms -- generally very knowlegable and helpful

There is a forum at www.itrc.hp.com , but I have no experience of the level or speed of replies there.

More details?

Good luck
Mike

[zoli]

Hello Mike,

thank you for your answer.
To be honest I was expecting an url for bug submission.
Because earlier my company had some kind of development support that I could connect by e-mail.
There I have reported dozen of OpenVMS bugs that had been solved.

What is the original problem?
What is an "unusual organization"?
More details?

You are right that C does not have default support, but there are very neat libraries provided along with C compiler that does the job. The problem is that there are two approach:
1. using native VMS SYS$QIO system calls. This is the right approach and this works perfect.
2. another method is using standard C library calls like open(), read() fopen, fread etc. At the bottom these C functions will use SYS$QIO system calls as well-but at the top they provide a nice, unix compatible programming environment for the developer.

Unfortunately, I need to solve my problem with method 2. because the code is designed and written to be portable. (of course I'm talking about Vim)

The problem is that I have got a file that Vim truncates - LSE reads well.
So I made a two test programs one that uses VMS system calls and another that uses standard C calls - and guess what. The second test failed.

This points to a C RTL bug.

The file organization is he following:

Code: Select all

SYSTEM@alpha$ sho def
  DKA0:<WORK>
SYSTEM@alpha$ ana/rms incomplete.log

Check RMS File Integrity                     24-JAN-2008 22:59:33.03   Page 1
DKA0:<WORK>INCOMPLETE.LOG;1


FILE HEADER

        File Spec: DKA0:<WORK>INCOMPLETE.LOG;1
        File ID: (64576,4,0)
        Owner UIC: [1,1]
        Protection:  System: RWED, Owner: RWED, Group: RE, World: R
        Creation Date:   24-JUL-2007 11:23:20.30
        Revision Date:    6-SEP-2007 09:58:19.02, Number: 4
        Expiration Date: none specified
        Backup Date:     none posted
        Contiguity Options:  none
        Performance Options: none
        Reliability Options: none
        Journaling Enabled:  none


RMS FILE ATTRIBUTES

        File Organization: sequential
        Record Format: variable-with-fixed-control
        Record Attributes:  print
        Maximum Record Size: 0
        Longest Record: 145
        Fixed Control Size: 2
        Blocks Allocated: 144, Default Extend Size: 0
        End-of-File VBN: 101, Offset: %X'00F4'
        File Monitoring: disabled
        File Length Hint (Record Count):    -1 (invalid)
        File Length Hint (Data Byte Count): -1 (invalid)
        Global Buffer Count: 0


The analysis uncovered NO errors.


ANA/RMS incomplete.log


This is a nice file... unfortunately C sees the EOF some 10 lines before the real EOF.

... and this is a major issue.

The only luck in this issue is that this record organization Record Format: variable-with-fixed-control with Fixed Control Size: 2 is that rare, that in fact, this is the first this kind of file that I have seen.

Any help would be appreciated.
Thank you in advance.

[miker_alpha]

As far as I know there is no place for an unsupported client (oxymoron?) to post a bug report.
OTOH, if you can build a *small* replicator (i.e. a couple of dozen lines that show the problem) you could post it on either of the two sites I mentioned before (comp.os.vms or itrc...) Then if a supported client tries it, they can escalate to support.
Also I don't know what the position is in Sweden, but here there is minimal phone support for programming problems even for users with no contract.

The file seems to be in VFC format i.e. built of records, at the start of each record are two length bytes. Surely that is not a native Un*x format.

Also - What happens in C if you read the EOF and just keep reading?
The EOF could just be a ^Z (ASCII 26) as data in the file. Can you do that? Of course the file has to be opened as a binary.

Specific to VMS -- not portable -- there is a block count for the file, and a byte count for the last block. Perhaps TYPE/LSE/etc. use that.
You can see these in the anal/file. Does your C program stop before?

Mike

[miker_alpha]

I found "among my souvenirs..." a BASIC file with some explanations of what a VFC file record looks like
HTH
___ START CODE ___

Code: Select all

PROGRAM create_vfc
!
! This program demonstrates a method of creating a VFC-formatted
! file from BASIC.  It does so by using a USEROPEN to tell RMS
! that this is a VFC file and by loading the 'fixed control'
! portion of each record with the appropriate carriage control
! characters.
!
OPTION TYPE = EXPLICIT
 
DECLARE STRING vfc_file
 
MAP (vfc_buf) STRING vfc_record = 255%
 
!
! Use a MAP area to set up the fixed control areas for
! the file.  RMS will read these fields for the carriage control
! information for each record on a PUT operation.
!
MAP (info_buf) WORD rhb
MAP (info_buf) BYTE prefix, suffix
 
!
! Get the name of the VFC file to create.  The useropen will
! get/load the necessary information RMS needs to process the file.
!
INPUT "What do you want to name the VFC file"; vfc_file
 
WHEN ERROR IN
    OPEN vfc_file FOR OUTPUT AS FILE #1%,  &
                  USEROPEN load_rhb,       &
                  MAP vfc_buf
 
!
! Following is a diagram of the coding schemes of the control
! areas (from the RMS manual).  By loading the prefix/suffix
! according to this table, we can create VFC carriage control
! for each record.  This table can be found under the topic of
! FAB$V_PRN in the RMS manual of your VMS documentation set.
!
!--------------------------------------------------------------------
!Bit 7   Bits 6-0   Meaning
!--------------------------------------------------------------------
!  0       0        No carriage control is specified, that is NULL.
!
!  0       1-7F     Bits 6 through 0 are a count of new lines (line
!                   feed followed by a carriage return).
!--------------------------------------------------------------------
!
!
!--------------------------------------------------------------------
!Bit 7   Bit 6   Bit 5   Bits 4-0   Meaning
!--------------------------------------------------------------------
!  1       0       0       0-1F     Output the single ASCII control
!                                   character specified by the
!                                   configuration of bits 4 through 0
!                                   (7-bit character set).
!
!  1       1       0       0-1F     Output the single ASCII control
!                                   character specified by the
!                                   configuration of bits 4 through 0
!                                   that are translated as ASCII
!                                   characters 128 through 159 (8-bit
!                                   character set).
!
!  1       1       1       0-1F     Reserved.
!--------------------------------------------------------------------
!
! There are many different carriage control options available through
! the use of VFC formats.  This program demonstrates only a few.
! The intent of this program is to show how this can be done, not
! to show all the different possibilities.  Remember, the values
! loaded into the PREFIX represent the carriage control to be applied
! BEFORE the record is displayed.  The values loaded in the SUFFIX
! will be applied AFTER the record is displayed.
!
!
! For the first record, let's just place the record on the current
! line (without any special carriage control).  It is not unusual
! to load the suffix with a carriage return on operations like
! this.  To load the suffix with a carriage return, we must refer to
! the table.  To accomplish this, we need to load bit 7 of the
! SUFFIX variable with a 1, bit 6 with 0, bit 5 with 0, and
! bits 4 through 0 (as a group) with the value of 13 (the ASCII
! representation of a carriage return).  An easy way to set the bits
! is to use EXPLICIT LITERAL NOTATION.  This is documented in the
! VAX BASIC Reference manual.  This approach requires knowledge of
! the binary numbering system.  Please note that bits are numbered
! starting with zero FROM THE RIGHT TO THE LEFT.
!
!        bit 7  bit 0
!          |      |
!          v      v
prefix = B"00000000"B
 
suffix = B"10001101"B  ! Notice that bit 7 is set, bits 6 and 5 are 0
                       ! and bits 4-0 represent the binary
                       ! equivalent of a decimal 13.
 
vfc_record = "This is being displayed on the first line"
PUT #1, COUNT LEN(TRM$(vfc_record))

!
! For the second record, let's set it up so that the record is
! displayed after 4 blank lines.  To do this, we must refer to the table
! shown above.  Notice in the table that when bit 7 is zero, we can
! use bits 6 through 0 (as a group) to represent the number of line-feed/
! carriage-returns we want issued before/after the record is displayed.
! Since we want this to be issued BEFORE the record is displayed, we
! will load the PREFIX byte of the control header.  If we wanted this
! carriage control to be issued AFTER the record is displayed, we would
! have to load the SUFFIX.
!
! This is fairly easy because there is little bit manipulation to do.
! all we have to do is load PREFIX with the value of 5.  The value of
! 5 in the prefix indicates that we want to have 5 LF/CR's issued
! BEFORE the record is displayed.  We are not using the value 4
! because no carriage control has been issued following the last record
! written (except for the carriage return we loaded into the suffix).
! We must advance to the next line before specifying the 4 blank lines
! (the cursor, from a terminal perspective, will still be positioned
! on the last record in column 1).  It is important to understand
! that no carriage control is applied unless we specify it in the fixed
! control areas.  To get the 4 blank lines we want, we must account for
! one extra line to force the previous "cursor" position off of the
! previous record to the next line and THEN add 4 more lines to that.
!
prefix = B"00000101"B     ! The equivalent of a decimal 5.
                          ! Bit 7 is zero and bits 6-0 equal 5.
 
suffix = B"10001101"B     ! This is for another CR at the end.
 
vfc_record = "This line comes 5 LF/CR's after the previous record."
PUT #1, COUNT LEN(TRM$(vfc_record))
 
!
! The next record we will write will be a little harder.  This time
! we will set it up to issue a FORM FEED, print the record, and then
! another 5 lines.  This means we have to load bit 7 of the PREFIX with 1
! and bits 4 though 0 with 12 (the ASCII representation of a form feed).
! We will also need to load the SUFFIX with 6 LF/CR's.
!
prefix = B"10001100"B  ! Bit 7 is set, bits 6 and 5 are not.
                       ! Bits 4-0 represent an ASCII 12.
 
suffix = 6%            ! The equivalent of B"00000110"B
                       ! Bits 6-0 represent the number of CR/LF's we want
 
vfc_record = "This line should be on the next page followed by 6 LF/CR's"
PUT #1, COUNT LEN(TRM$(vfc_record))
 
!
! Finally, let's write a record on the current line and issue a CR/LF
! after it.
!
prefix = 0%            ! The equivalent of B"00000000"B.
suffix = B"10001101"B  ! This is for another CR at the end.
vfc_record = "This line should be 5 lines down with CR following. OOO"
PUT #1, COUNT LEN(TRM$(vfc_record))

prefix = 0%            ! The equivalent of B"00000000"B.
suffix = 1%            ! The equivalent of B"00000001"B.
vfc_record = "____ ____ ______ __ _ _____ ____ ____ CR __________ ///"
PUT #1, COUNT LEN(TRM$(vfc_record))
 
USE
    !
    ! Feel free to add more comprehensive error handling...
    !
    PRINT "Error:"; ERR; "  Error text: "; ERT$(ERR)
    CONTINUE end_program
END WHEN
 
end_program:
    CLOSE #1
    END
 
 
FUNCTION LONG load_rhb (fabdef fab, rabdef rab, LONG lun)
    !
    ! This function loads the RHB field of the RAB with the
    ! address of our own variable.  RMS uses this field to
    ! determine the carriage control for each record.
    !
 
    OPTION TYPE = EXPLICIT
 
    %INCLUDE "$rabdef" %FROM %LIBRARY
    %INCLUDE "$fabdef" %FROM %LIBRARY
 
    EXTERNAL LONG FUNCTION sys$create, sys$connect
 
    DECLARE LONG stat
 
    !
    ! We want this function to share the address of the control
    ! area with the main program.  This is done by placing the
    ! target variable into a MAP area.  The RHB is broken down
    ! into the PREFIX and the SUFFIX in the main program.
    !
    MAP (info_buf) WORD rhb
 
    !
    ! The RAB$L_RHB field is a reference that RMS uses to an
    ! area in memory that it will read the prefix and suffix
    ! from.  If we load it with the address of our own variable
    ! we can gain access to the prefix and suffix values.
    ! See the RMS Reference manual for more information on
    ! the RAB$L_RHB field.
    !
    rab::rab$l_rhb = LOC(rhb)
 
    !
    ! We also need to create a file with PRINT record format.
    ! This can be done by loading the FAB$B_RAT field with
    ! the FAB$M_PRN symbolic constant.  See the RMS documentation
    ! for FAB$B_RAT for more information.
    !
    fab::fab$b_rat = fab$m_prn
 
    !
    ! Make sure that a VFC file is created by loading FAB$C_VFC into
    ! the FAB$B_RFM field.  See the RMS documentation for more
    ! information on the FAB$B_RFM field.
    !
    fab::fab$b_rfm  = fab$c_vfc
 
    stat = sys$create (fab)
    IF (stat AND 1%) <> 0% THEN
        stat = sys$connect(rab)
    END IF
END FUNCTION stat


Mike


[/code]

By the way: When a file is Opened/Written/Closed using DCL, the default is VFC structure.
If you try reading INCOMPLETE.LOG line by line in DCL do you get the complete file?

[miker_alpha]

Another possibility - convert the file?

Code: Select all

$! Convert a VFC file to Variable, Stream
$!
$ set NoOn   ! ignore errors
$ if f$search( "INCOMPLETE.LOG" ) .eqs. "" then exit
$ if f$file_attr( "INCOMPLETE.LOG", "RFM" ) .eqs. "VFC"
$ then
$   create TEMPORARY.FDL
SYSTEM
   SOURCE                  OpenVMS
FILE
   BEST_TRY_CONTIGUOUS     no
   CONTIGUOUS              no
   EXTENSION               0
   GLOBAL_BUFFER_COUNT     0
   ORGANIZATION            sequential
RECORD
   BLOCK_SPAN              yes
   CARRIAGE_CONTROL        carriage_return
   FORMAT                  variable
$EOD
$!
$   convert/fdl=TEMPORARY.FDL -
                  INCOMPLETE.LOG; INCOMPLETE.TMP;0
$   rename/log INCOMPLETE.TMP; INCOMPLETE.LOG;0
$   delete/nolog TEMPORARY.FDL;*
$ endif
$!


... This is a system mangler-type solution -- not a programmer's

Mike

[zoli]

Thank you Mike,

Sure, I was able to read the whole file adding some non standard parameters to open like "rat=cr", "rfm=stmlf", "mrs=0" etc. But this is not the right solution, because then I need to make a detection for every single file in order to set up those parameters correctly.

I would prefer to relay on C's open() function's own intelligence to open the file with correct setup and be able to read the file to the end using read() function.
Now it works in 99% cases and I have find just this singe file when it fails.

post it on either of the two sites I mentioned before (comp.os.vms or itrc...) Then if a supported client tries it, they can escalate to support.

I am a bit scared with support escalation that you have described.
Seems, there is no other effective method, but pull my wires in Budapest and Vienna development and submit this bug through them

Thank you for positive approach and investigation that you have put in this issue.

[miker_alpha]

??

[zoli]

I sent the case to Budapest office and they should handle this issue with HP's openVMS support.

No feed back so far.