EP1354270A4 - Real-time program audit software - Google Patents
Real-time program audit softwareInfo
- Publication number
- EP1354270A4 EP1354270A4 EP00948614A EP00948614A EP1354270A4 EP 1354270 A4 EP1354270 A4 EP 1354270A4 EP 00948614 A EP00948614 A EP 00948614A EP 00948614 A EP00948614 A EP 00948614A EP 1354270 A4 EP1354270 A4 EP 1354270A4
- Authority
- EP
- European Patent Office
- Prior art keywords
- program
- audit
- options
- execution
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
- G06F11/3612—Software analysis for verifying properties of programs by runtime analysis
Definitions
- the present invention relates generally to a method and system for auditing and analyzing the execution of computer programs in real-time, and particularly to a method and system for generating source program auditing statements for debugging an application program.
- source programs are written in a source programming language by computer programmers.
- source programs can be generated by a Fourth Generation Language (4GL) or a computer-aided software engineering (CASE) tool.
- Programs created using a Fourth Generation Language or a CASE tool are automatically translated into source statements of a conventional programming language as an intermediate step to compilation and execution.
- Conventional programming languages include Common Business-Oriented Language (COBOL) , formula translation (FORTRAN) , Pascal, and Report Program Generator
- the programmer using a conventional or alternative programming language, or the application designer using a 4GL or a CASE tool uses available programming tools such as objects, functions, diagrams, routines, and/or operation codes to translate a perceived or defined need into a programmed solution.
- the output program must be tested to determine if the program works as designed.
- the program is executed against data to verify the efficacy and accuracy of the program as executed.
- both the programmer and the application designer are generally referred to as the programmer .
- the quickest and easiest part of programming is writing or generating the source program, while the most difficult and time-consuming part is making the program work correctly.
- the computer program needs to work correctly both initially and years later. Errors in programming result from a variety of sources.
- U.S. Patent Number 5,559,884, issued on September 24, 1996 to Robert L. Davidson et al . discloses a method and system for generating and auditing a signature for executable modules. Davidson et al . do not suggest real-time program audit software according to the claimed invention.
- U.S. Patent Number 5,574,898, issued on November 12, 1996 to David B. Leblang et al . discloses a data processing system and method which feature an object selector including an auditor for recording, as an audit record, which versions of objects are accessed by a processor during a data processing process. Each derived object is associated with an audit record.
- a system build process starts the auditor prior to executing commands which produce derived objects, and stops the auditor when those commands are completed.
- the process records any arbitrary sequence of commands.
- the invention of Leblang et al . is a CASE system and cannot be utilized with other programming languages or source programs, regardless of how they are generated. Also, the invention of Leblang et al . audits which version of a file is being utilized; it is a CASE version control system.
- the program also generates a configuration record which provides a complete record of software builds which includes a listing of all source file versions used, versions of build tools, and all build options specified. Configuration records can then be compared by showing the differences between two builds of the same program. Other commands label the builds with version labels on object versions listed in the record.
- the invention of Leblang et al is a CASE system and cannot be utilized with other programming languages or source programs, regardless of how they are generated.
- the invention of Leblang et al . audits which version of a file is being utilized; it is a CASE version control system.
- the program also generates a configuration record which provides a complete record
- Leblang et al . is designed to monitor the version of a software program, and the objects used within that software program, during the design of the program. It is a tool to be used by computer programmers who are utilizing CASE tools to write a software program and, therefore, is not as useful as the present invention in that it is limited to CASE tool program design. Also, the invention of Leblang et al . is not useful for monitoring the execution of a program in a remote time after the creation of the program. The invention of Leblang et al . is distinctly different from the present invention in that it was not designed to monitor the execution of a program utilizing any type of programming language; the current invention is designed to monitor any type of machine code as it executes. Leblang et al . do not suggest real-time program audit software according to the claimed invention.
- U.S. Patent Number 5,754,763, issued on May 19, 1998 to Thomas W. Bereiter shows a software auditing program which is designed to monitor the number of users who simultaneously invoke one or more application programs which occur in response to system management tasks.
- the auditing program therein counts the number of simultaneous invocations in order to determine whether an authorized number of copies of each program within the managed region has been exceeded.
- the protocol requires a dedicated license server.
- the sole purpose of the invention of Bereiter is to use a license server to identify violations of the licenses of an organization.
- the program therein does not audit any other activity nor does it aid in the actual execution of a program. It does not aid in the monitoring of program errors and trouble shooting during the execution of a program or system, as does the current invention. Bereiter does not suggest real-time program audit software according to the claimed invention.
- Bailey et al . also make a record of the backups in order to verify that the most recent files are in use and are concurrent with the backup records.
- the invention of Bailey et al . is distinctly different from the current invention.
- Bailey et al . protect or audit data files, while the current invention audits program executions. These are very distinct activities.
- Bailey et al . do not suggest real-time program audit software according to the claimed invention.
- Johnson et al . also permit reconstruction of the record units of an enterprise in the event of a catastrophic event.
- Johnson et al . audit storage files but not program execution and is, therefore, distinctly different from the present invention.
- Johnson et al . do not suggest real-time program audit software according to the claimed invention.
- International Patent document WO 93/01550 published on January 21, 1993, discloses a method and system for recording the use of a licensed product, and for controlling its use. International '550 does not suggest real-time program audit software according to the claimed invention.
- the current invention is a software program, a method, and a system for generating source program audit statements which examine and verify program statements and data as the program executes.
- the audits generated by the real-time program audit provide real-time analysis of the execution of the program.
- the real-time program audit software may be used with any programming language that uses source program statements, whether the source statements are compiled into an executable object or are interpreted during program execution.
- Conventional source programs are selected for auditing based on an audit profile.
- the audit profile is customized for specific compile and initial execution options.
- the source program is expanded with the selected audit statements.
- the source program is compiled with a conventional language compiler.
- Program execution audits are based on the initial execution audit profile, customized execution audits, or dynamic audits specified during program execution.
- Program audit output is to a disk or print file, and is available for immediate online display or printing, or for expanded auditing analysis.
- Expanded auditing analysis provides for extensive analysis of the real-time program audit output data from all audited programs based on the desired analysis, which includes program, user, date and time ranges, audit code prefixes, audited file, field, label, and any execution audited data value .
- the real-time program audit software is a new approach to auditing the execution processing of programs of virtually any source programming language.
- the real-time program audit is basically a pre-compiler program that is specific for the type of programming language utilized. In other words, a different version of the real-time program audit software is required for each programming language compiler or interpreter utilized.
- Fig. 1A is a block diagram of the source program compile and initial execution audit functions program (Z$PGM01) .
- Fig. IB is a block diagram of the expansion of the conventional source program with audit statements program (Z$PGM02), and the conventional programming language compile of the expanded source program.
- Fig. 2 is a block diagram of the program execution audit options program (Z$PGM03) , including the modification of the initial execution audit options, and the conventional execution of the compiled program object with audit outputs.
- Fig. 3 is a block diagram of the create or change options and run audit analysis options program (Z$PGM04) , including the creation or change of audit analysis reporting options, and the processing of the audit analysis report .
- Fig. 4 is a block diagram of the create or change options and run formatted audit analysis reporting options program (Z$PGM05) , and the processing of the formatted audit analysis report .
- Fig. 5 is a block diagram of the modify source program compile and initial execution audit options program (Z$PGM06) , including the capability to undo all or selected audit source statements.
- Fig. 6A is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 6B is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 6C is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 6D is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 7A is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 7B is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 7C is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 8 is part of a flowchart of the pseudo-code for the create or change compile and initial execution audit options program (Z$PGM01) .
- Fig. 9A is part of a flowchart of the pseudo-code for the expand source program with audit statements program (Z$PGM02) .
- Fig. 9B is part of a flowchart of the pseudo-code for the expand source program with audit statements program (Z$PGM02) .
- Fig. 9C is part of a flowchart of the pseudo-code for the expand source program with audit statements program (Z$PGM02) .
- Fig. 10A is part of a flowchart of the pseudo-code for the expand source program with audit statements program (Z$PGM02) .
- Fig. 10B is part of a flowchart of the pseudo-code for the expand source program with audit statements program (Z$PGM02) .
- Fig. IOC is part of a flowchart of the pseudo-code for the expand source program with audit statements program (Z$PGM02) .
- Fig. 11 is part of a flowchart of the pseudo-code for the expand source program with audit statements program (Z$PGM02) .
- Fig. 12A is part of a flowchart of the pseudo-code for the modify the initial execution audit options program (Z$PGM03) .
- Fig. 12B is part of a flowchart of the pseudo-code for the modify the initial execution audit options program (Z$PGM03) .
- Fig. 13A is part of a flowchart of the pseudo-code for the create or change options and run the audit analysis reporting program (Z$PGM04) .
- Fig. 13B is part of a flowchart of the pseudo-code for the create or change options and run the audit analysis reporting program (Z$PGM04) .
- Fig. 13C is part of a flowchart of the pseudo-code for the create or change options and run the audit analysis reporting program (Z$PGM04) .
- Fig. 14A is part of a flowchart of the pseudo-code for the create or change options and run the formatted audit analysis reporting program (Z$PGM05) .
- Fig. 14B is part of a flowchart of the pseudo-code for the create or change options and run the formatted audit analysis reporting program (Z$PGM05) .
- Fig. 14C is part of a flowchart of the pseudo-code for the create or change options and run the formatted audit analysis reporting program (Z$PGM05) .
- Fig. 15A is part of a flowchart of the pseudo-code for the modify source program compile and initial execution audit options program (Z$PGM06).
- Fig. 15B is part of a flowchart of the pseudo-code for the modify source program compile and initial execution audit options program (Z$PGM06) .
- Fig. 15C is part of a flowchart of the pseudo-code for the modify source program compile and initial execution audit options program (Z$PGM06) .
- the present invention is real-time program audit software that is a software program, a method, and a system for generating source program audit statements which provide analyses of program statements and data as the source program executes. These audit statements provide for real-time analysis of the execution of the program.
- the real-time program audit software is applicable to virtually any programming language that uses source program statements, whether the source statements are compiled into an executable object or are interpreted during program execution.
- Conventional source programs are selected for auditing based on an audit profile; the audit profile is customized for specific compile and initial execution options; the source program is expanded with the selected audit statements; and the source program is compiled with a conventional language compiler.
- Program execution audits are based on the initial execution audit profile, customized execution audits, or dynamic audits specified during program execution.
- Program audit output is to a disk or print file, and is available for immediate online display or printing, or for expanded auditing analysis.
- Expanded auditing analysis provides for extensive analysis of the real-time program audit output data from all audited programs based on the desired analysis, which includes: program, user, date and time ranges, audit code prefix, audited file, field, label, and any execution audited data value .
- program, user, date and time ranges includes: program, user, date and time ranges, audit code prefix, audited file, field, label, and any execution audited data value .
- Conventional programming language is defined to be any artificial language that can be used to define a sequence of instructions that can ultimately be processed and executed by a computer; a translation process, from the source code expressed using the programming language to the machine code that the computer needs to work with, must be automated by means of a compiler.
- programming language refers to any series of source codes regardless of whether the program statements are compiled or interpreted.
- Source code is human-readable program statements written in a high-level or assembly language that are not directly readable by a computer.
- a "source program” is the source code version of a program.
- Object code is the code, generated by a compiler or an assembler, that was translated from the source code of a program. Object code generally refers to machine code that can be directly executed by the system' s central processing unit, but it can also refer to assembly language source code or a variation of machine code.
- An "interpreter” is a program that translates and then executes each statement in a program written in an interpreted language.
- a “compiler” is any program that transforms one set of symbols into another by following a set of syntactic and semantic rules.
- a compiler is a program that translates all the source code of a program written in a high-level language into object code prior to execution of the program. As used herein, "compiler” refers to both interpreter and compiler.
- Real-time refers to a time frame wherein the computer responds to situations as they occur.
- the analyses which occur in the present invention occur at the rate at which the audited program is executed.
- Real-time operations are those in which the machine's activities match the human perception of time or those in which computer operations proceed at the same rate as a physical or external process.
- Operaation code is the portion of a machine language or assembly language instruction that specifies the type of instruction and the structure of the data on which it operates .
- validating refers to accepting a pre-chosen variable or changing that variable to a preferred variable.
- Variables include program language, conventional source program, and audit profile.
- the present invention provides a method and system for generating computer source code audit statements from input source programs of virtually any commercial programming language.
- the generated audit source statements are inserted into a copy of the input source program (the conventional source program) , become an integral part of the copied source program (the expanded source program) , and are available for optional auditing at any time during program development, program testing, implementation, or in a production environment, without intervention by the programmer, or anyone else.
- the generated source code audit statements then may be utilized during the execution of the compiled program object, or execution of an interpreted source program, to provide a real-time audit of the detailed processing of the program against the data processed by the program.
- the resulting real-time audit output may be viewed online as the program execution takes place, or may be viewed later both online or in printed audit analysis reports. Audits may be specified at up to ten levels, providing for no auditing, auditing of only key program functions, to very comprehensive auditing of virtually every executable instruction as it is executed, with all the data processed by the instruction. The level of auditing may be changed dynamically as the program is executing by using dynamic audits.
- this invention provides for the insertion of source program documentation statements into the input conventional source program. This provides comprehensive and useful program documentation not normally found in conventional source programs written by most commercial programmers, and not available in most commercial software products .
- the resulting expanded source program becomes much easier to read and comprehend, particularly for programmers unfamiliar with the program, and the resulting source program logic is much easier to understand by viewing the audit output of the actual processing being performed against the data being processed.
- the real-time program audit software provides a separate licensed implementation for each source programming language implementation supported. Thus, each programming language vendor compiler that supports specific language functions and operation codes could have a licensed implementation of the real-time program audit software which would support that specific language implementation. Each programming language requires a separate version of the real-time program audit software.
- Fig. 1 shows a block diagram of the initial input and output files needed for the real-time program audit software. It shows the source program compile and initial execution audit functions used.
- the program module create or change compile and initial execution audit options program 100 handles the initial input/output files for the real-time program audit software.
- the create or change compile and initial execution audit options program 100 (Z$PGM01) allows the programmer to quickly and easily audit and document the conventional source program 27 or select additional audits for a previously expanded source program.
- This program is a setup program for the expand input with audits program, the pre-compiler program 200 (Z$PGM02) , and provides all the information and options needed to expand the source program with audit statements.
- the select compile and initial execution audit options for program 100 represents the screen input.
- the operation code audit profile and master information file 21 contains an audit profile of virtually every operation code and file description format used by programmers in writing source programs in the vendor programming language implementation.
- the operation code audit profile and master information file 21 determines exactly how the operation code is to be audited, if selected for auditing, including all variable names and status codes used in the instruction.
- the profile also determines whether the audit is to be inserted before the source statement (as in branches) , or after the source statement (as in file I/O) .
- the operation code audit profile and master information file 21 contains the standard audit prefix code for the operation code, such as Z$R for READ and Z$W for WRITE.
- the compile and initial execution audit profiles 23 master file contains a default compile audit profile, and a default initial execution audit profile for typical programming environments such as: initial compile, where comprehensive documentation would be appropriate; initial test, where audits of all file input and output would be appropriate; pilot implementation, where comprehensive auditing would be appropriate; and for production implementation, where selected auditing such as key event auditing may be appropriate. Production implementation auditing is critical for enhanced error correction and for auditing by internal and external auditors.
- the data file names, attributes, keys, fields length, and types 25 master file contains all the necessary information about each file used in the source program to insert the audit and documentation statements into the expanded source program. This information is automatically generated by analyzing the compile listing output of the conventional input source program.
- the conventional source program library 27 is the existing source program library in the programming language at the licensed customer site.
- Previously expanded conventional source program library files are included as a conventional source program library 27 because they are handled in the same manner as if unexpanded. Programs to be expanded with audit and documentation statements should compile successfully before program expansion, including already expanded conventional programs.
- Conventional data files 29 are the existing data files used in the conventional source programs at the licensed customer site.
- An appropriate audit profile is selected for the selected program name, together with an initial execution profile. The default audit profile, documentation options, and initial execution options may be utilized, or the profile options may be extensively modified for each program file, operation code, field and label used in the conventional source program.
- the ability to bypass auditing of labeled subroutines or procedures that are specified by the programmer or for the entire installation is provided.
- This provides the ability to bypass the auditing for very repetitive routines such as date validation and field names defined as constants, which can greatly reduce the amount of audit output, allowing focus on key program processing routines.
- the selected compile audit options file 22 (Z$AUDITC) , and the program file and field information file 24 (Z$AUDITM) are output, and used in the next program, the expand input with audits program, the pre-compiler program 200, to expand the source program with audit statements.
- the initial execution audit options file 26 (Z$AUDITO) is also output, and is used during program execution to create audit output.
- the selected compile audit options file 22 provides output which can also be used as shown in Fig. 2. This is denoted by the circled capital A.
- the initial execution audit options file 26 also provides output which can be used, as denoted by the circled capital B.
- Advanced and online review of output audits is also provided, by optionally displaying the source program variable names on a line above the actual audit data being executed. This optional review is available only if the expanded source program is online during review of the audit output file, which is normally the case.
- the ability to select and sequence virtually any file record (data) field when auditing file record processing is provided, as well as the ability to select and sequence virtually any (data) field, when auditing any label in the program.
- the ability to audit any changed (data) field in a specified labeled subroutine or procedure is provided, in addition to the ability to audit any specified (data) field, including auditing only when the field is changed.
- the submit program to insert audit statements 28 represents screen input.
- the programmer decides whether to submit the conventional program to the expanded with audit statements by real-time program audit software.
- the block diagram shown on Fig. 1A continues to Fig. IB, as shown .
- the expand input with audits program provides a one-pass expansion of the input conventional source program 27, utilizing the selected compile audit options file 22, and program file and field information file 24 from the previous program.
- This expand input with audits program 200 is the major module in this invention, and it may be considered to be a pre-compiler to the conventional programming language compiler 37.
- the expanded source program audit messages 31 are read into the pre-compiler program 200. Each conventional source statement read is examined for insertion of audit and/or documentation statements, using the selected options. Every data field for audited statements is audited, and a unique audit code is assigned to each audited statement, by suffixing the standard assigned audit prefix code with a sequential number starting with 001.
- the pre-compiler program 200 is output to a separate expanded source program with selected audit statements library 33, leaving the input conventional source program unchanged.
- Each of the audit statements are assigned a unique fifteen digit number to provide for a second level file of very detailed audit information for every audited source statement executed.
- This optional detailed auditing provides for auditing all of the one hundred RPG indicators, and the command keys, and other key information at every audited statement executed in the expanded source program for RPG language implementations. Standard audit printer and disk file definitions are copied into the program, together with routines to initiate and change auditing during program execution.
- the pre-compiler program 200 also includes an express expand source option which utilizes default audit options normally utilized by the programmer.
- This express expand option allows a programmer to key only the program name and then to press a command key to expand the input source program with realtime program audits.
- This express option is in addition to the option to fully customize the audits by file, record format, operation code, field, and program label.
- the expanded source program with selected audit statements library 33 and the conventional (existing) program language compiler files 35 are read into the conventional programming language source program compiler 37.
- the conventional programming language source program compiler 37 compiles the expanded source program, producing an expanded executable program object with selected audits 32 having the compiled audit statements.
- the conventional programming language source program compiler 37 also produces a program source compile listing with audit statements document 39. The source program compile listing may be utilized as the basis for creating real-time programming audit work files.
- the compile listing completely defines the program files, fields, labels, and operation codes including copybooks utilized in the source program.
- the expanded source program with selected audit statements library 33 and the expanded executable program object with selected audits 32 are also referred to in other block diagrams denoted by the circled capital letters .
- the expanded source program with selected audit statements library 33 is utilized in Fig. 5 as denoted by the circled capital C
- the expanded executable program object with selected audits 32 is utilized in Fig. 2 as denoted by the circled capital D. It is possible to include several commonly utilized levels of compilers in the same language implementation.
- RPGIII RPG/400
- ILE RPG RPG
- both COBOL/400 and COBOL ILE can be provided in the same implementation.
- Fig. 2 shows a block diagram of the program which is responsible for the execution of audit options.
- the modify initial execution audit options program 300 (Z$PGM03) allows the programmer to quickly and easily modify the selected initial execution options for compiled audit statements, even during program execution.
- the selected compile audit options file 22 is utilized by the modify initial execution audit options program 300.
- the modify initial execution audit options for program 41 represents screen input. Here the programmer decides whether to change the selections previously made pertaining to audit options for the program and to write these selections to the modify initial execution audit options program 300.
- the modify initial execution audit options program 300 is the program that allows the dynamic modification of the auditing options and audit output as the program is in program execution, by modifying the initial execution audit options for program file 26 into the current execution audit options file 43.
- the initial/current execution audit options program is utilized to create the initial execution audit options for program file 26 and the current execution audit options file 43.
- the current execution audit options file 43 and the expanded executable program object with selected audits file 32 are used in conventional program execution by the executable program object with selected audits 47.
- Data files used in the program conventional processing 45 also write to, and are written to, by the executable program object with selected audits 47.
- the output from the executable program object with selected audits 47 includes an audit execution analysis report 49 and an audit execution output file 42.
- the audit execution output file 42 is also used in Fig. 3 as denoted by the circled E.
- Fig. 3 is a block diagram depicting the create or change options and run audit analysis reporting program 400, and includes the creation, change, and processing of audit analysis reporting options.
- the create or change options and run audit analysis reporting program 400 allows the programmer or any authorized user to analyze the audit execution output file 42 (Z$AUDITF) information, if this file was output during program execution.
- the display or print execution audit analysis 51 screen display allows the user to decide which menus to choose .
- the requestor may select from a menu of audit analysis functions and create printed or displayed audit output or create formatted disk audit output 59 (Z$AUDITE) for further analysis.
- the create or change options and run audit analysis reporting program 400 (Z$PGM04) also produces an audit analysis reporting document 57 (Z$AUDITP) .
- a display of audit analysis 55 may be produced.
- An audit analysis reporting options file 53 (Z$AUDITA) provides the ability to save and retrieve reporting options by a name assigned to the audit analysis request.
- Typical audit output would be for a specific job execution of the program, or by user, terminal, date, or time range.
- the printed and disk audit outputs may be viewed and scanned using standard system utility programs and available utility programs in real-time as the program executes, or later for error resolution or analysis.
- Fig. 4 shows a block diagram of the create or change options and run formatted audit analysis reporting program 500 and the processing of the formatted audit analysis report 65.
- the display or print formatted audit analysis 61 screen display permits the user to select options for the create or change options and run formatted audit analysis reporting program 500 (Z$PGM05) .
- the create or change options and run formatted audit analysis reporting program 500 (Z$PGM05) allows the programmer or any authorized user to analyze the extracted audit execution file 59 (Z$AUDITE) , if the extracted audit execution file 59 (Z$AUDITE) was output by the previous program.
- a formatted audit analysis reporting options file 63 (Z$AUDITS) provides the ability to save and retrieve formatted reporting options by a name assigned to the formatted audit analysis request.
- the requestor may select from a menu of formatted audit analysis functions, presented by the display of formatted audit analysis 67, and create a printed audit output .
- the formatted audit analysis reporting 65 allows the user to print audit analysis. Typical audit output would be for a specific application to selected transaction types over a range of dates and times. This focused audit output is most useful for internal and external auditors in verifying the detailed computations, processing, and transactions behind more summary output. These detail transactions may be transient computations that otherwise would never have been written to disk and saved without the audits, and not available on any disk journal.
- the modify compile and initial execution audit options program 600 (Z$PGM06) allows the programmer to undo or partially undo the previous expansion of source programs with audit statements, and therefore remove the expanded source program with selected audit statements file 33.
- the input/output to the modify compile and initial execution audit options program 600 is shown in the block diagram in Fig. 5.
- the expanded source program is read from the expanded source program library and contains the expanded source program with selected audit statements file 33. All audit and documentation statements are summarized and displayed to the programmer on screen by the modify compile and initial execution audit options for program 71 screen display. The programmer may then undo all or some of the auditing or documentation selections. The initial execution options for the program are similarly read, summarized, and displayed to allow the undoing of these selections.
- the source program may then be expanded with additional audit, documentation, and initial execution options with the expand input with audits program 200 using input/output that is very similar to that shown Fig. 1A, and the same files are frequently used, as shown by both Fig.
- FIG. 5 flows to Fig. IB in the exact same manner as does Fig. 1A.
- the operation code audit profile and master information file 21 is utilized several times and is the same file throughout even though its position in the block diagrams are not identical.
- the source program may alternatively be left as a conventional program.
- Fig. 6A shows the initial selection on the screen display corresponding to the select compile and initial execution audit options program 20 shown in the block diagram of Fig. 1A.
- the initial display 101 shows the initial screen images.
- the initial display 101 prompts the user to enter the name of the conventional source program 27 and select the default compile audit profile from the compile and initial execution audit profiles 23 master file.
- the initial steps involve validating the selection of the program 102 to be audited from a display of conventional source programs 27 listed preferably in a library format, validating a selected program language 103 to determine that the program language is acceptable for auditing by the audit pre-compiler, and validating the entered audit profile 104 or allowing selection of the audit profile.
- Fig. 6B includes the steps of retrieving and saving operation codes 105,106,107. Retrieving and saving the default audit profile operation codes and the default initial execution audit options 105 are executed at this time.
- This step 105 includes retrieving and saving the default audit profile operation codes which are to be audited and the default initial execution audit options for the selected audit profile.
- the next step 106 involves retrieving and matching the operation codes to be audited.. The selected program source operation code is retrieved and matched to the default audit profile operation code used. The selected operation codes are to be audited unless deselected subsequently during execution of the real-time program audit software.
- Next is the step 107 of saving the programs unselected operation codes. This step 107 involves saving the program operation codes which are actually used in the program but are not selected for auditing in the default audit profile. These saved program operation codes may be selected for additional auditing later during the execution of the real-time program audit software .
- the steps 108,109,110 involve the matching of program operation codes used by the conventional source program 27 but not in the default audit profile with the master list of all operation codes valid for auditing. Also saved are any program operation codes used by the program but that are not valid for auditing, for later optional display 108. Also shown is the step 109 of retrieving or creating the data file names, attributes, keys, fields, lengths, types 25 master file for each file used.
- the next step 110 creates a cross reference which involves creating a separate file and field label files on a disk with one keyed record per file. This step 110 creates the program file and field information file 24; it creates a keyed record per field including program defined constants and internally described file fields.
- the next step 111 deletes all unreferenced fields that are not actually utilized in the conventional program from the program file and field information file 24.
- This step 111 deletes all unreferenced fields which are not used from the disk cross reference file.
- the next step 112 displays a menu of the functions specified in the selected audit profile and allows the selection of the menu items for auditing profile overrides. These menu options are grouped by files, operation codes, and fields.
- the next step 113 allows a command key to be pressed to accept the audit profile options as displayed. This creates the selected compile audit options file 22 and the initial execution audit options file 26. An option is provided for submitting the source program directly to the basic audit pre-compiler program for insertion of the auditing statements utilizing the submit program function.
- a step 114 is provided which enables the user to display the menu options in order to override the default audit options for the program.
- the menu override options include: Files, Operation codes, Fields and Labels, and initial execution options. If the override option File is chosen 115, the display will show a summary of the file and audit status utilized.
- Step 116 enables a user to select or deselect files for auditing, to modify auditing levels, and to permit a command key to display a second line with each operation code used by the file and its auditing status.
- Fig. 7B shows selections 117,118,119 pertaining to auditing the operation codes.
- Step 117 enables the programmer to select (or unselect) to audit each operation code used in each file, and to determine the auditing level. It is important to emphasize that all key fields used in the audited file's operation codes will be audited automatically. The programmer may also select additional non-key fields for automatic auditing. If the programmer chooses to use operation code overrides, step 118 displays each operation code. The display will include the auditing status of the operation code and the level of auditing selected. It will show the assigned audit code prefix and apparent duplicate field names. Step 119 enables the programmer to select (or unselect) each operation code used for auditing, the auditing level, and to change previously assigned audit prefix codes. Fields used in non-file audited operation codes will be audited automatically. Audit prefix codes will be suffixed in the pre-compiler program 200 to uniquely identify an audit point.
- Fig. 7C shows additional audit options 120,121,122,123,124.
- a display of each Field and Label used in the program is shown as a single line on a Field summary screen, and a display of the auditing status and which of the ten levels of auditing of each Field and Label used in the program is also shown.
- Step 121 enables the programmer to select (or unselect) each Field or Label used for auditing, to modify the audit levels, and to allow auditing when a field is used or is modified.
- step 122 displays the default initial execution profile options. The execution options may be modified and the auditing level may be changed from level 0 to level 9.
- Step 123 requires the programmer to validate all default profile and override entries. Error messages are displayed as required, wherein the programmer accepts and revalidates input until all input is correct.
- Step 124 provides a command key which may be utilized to exit the program and to submit the source program for expansion of the auditing statements.
- Fig. 8 shows the output for the create or change compile and initial execution audit options program 100.
- step 125 formats and creates the output files required for the program to be utilized in expanding the conventional source code with audit statements.
- the output files which are produced include selected basic audit compile options for source program expansion to the selected compile audit options file 22, program file information, key fields information and field cross reference for source program expansion to the program file and field information file 24, and basic audit initial execution options for the object (executable) program to the initial execution audit options for program file 26.
- Step 127 prints a summary of the audit options, the program's initial execution options, informational messages, warning messages, error messages, counts of the input conventional source statements, files, fields, and labels used.
- Step 128 displays a message that the source program has been submitted for expansion of the auditing statements.
- the last step 129 returns to the calling program.
- the steps required to create or change compile and initial execution audit options are complete.
- the create or change compile and initial execution audit options program 100 has been executed fully.
- the first four steps 201,202,203,204 in the pre-compiler program 200 are depicted in Fig. 9A.
- the pre-compiler program 200 reads and stores the selected audit compile options which were created by the create or change compile and initial execution audit options program 100. Additional information for the expansion of the conventional source program is stored in the operation code audit profile and master information files 21.
- the pre-compiler program 200 reads and stores the program file information, the key fields information, and the field cross reference information that were created by the create or change compile and initial execution audit options program 100.
- the pre-compiler program 200 reads the conventional (existing) source statements for the program from the conventional source program 27 library, and in a single pass of the conventional source program statements, expands the source by inserting the basic audit source statements and expanded documentation statements.
- the pre-compiler program 200 determines if expanded documentation is selected. For each file used, comment lines are created with the full file name and a list of the file field names used in the file key. Auditing may be limited to only expanded source program documentation .
- Fig. 9B shows the auditing to disk selections 205,206 for the pre-compiler program 200.
- the programmer must insert the File description source statements for the audit execution output file 42 into the source program statements.
- Audit execution output file 42 source statements are copied from the audit master information. Related files, records, fields, and standard processing routines for this function are also copied. All audit disk output will be written to the audit execution output file 42 disk.
- the programmer may select option 206, to effect the basic auditing of operation codes or data fields to disk. In this case, the programmer must insert the file description source statements for the audit execution output file 42 into the source program statements.
- the program copies the current execution audit options file 43 source statements from the audit master information. Also, the program copies related file, record, field, and standard processing routines for this function.
- the current execution audit options file 43 is the audit execution options disk files which controls which of the audit options are active during program execution.
- the programmer inserts the file description source statements for the audit execution analysis printer file into the source program statements.
- the program copies the audit execution analysis report 49 file source statements from the audit master information and copies related files, records, field information, and processing routines for this function.
- This print record output is formatted in a manner similar to the previous disk record output.
- the audit execution analysis report 49 printer file provides real-time analysis of audit output during program execution from a printer spool file. This output is for that specific program execution, for only that specific user, and for only that job.
- step 208 the auditing and/or documentation options selected in the previous program, the create or change compile and initial execution audit options program 100, for each executable source statement read are determined. This determination involves consideration of the statement operation code, the statement variable names, and the conditions, such as indicators or switches.
- the auditing and/or documentation source statements are inserted using the audit operation code profile for the source statements being processed.
- the resulting audits may be output to disk via the audit analysis reporting options 53 file and/or to a printer via the audit analysis reporting 57 program based on selections in the create or change compile and initial execution audit options program 100.
- a unique audit name for the audited source statement is generated for each source statement to be expanded with audit statements .
- the assigned three character audit prefix code for the operation code from the create or change compile and initial execution audit options program 100 is utilized.
- the audit prefix code is suffixed with a three digit sequential number each time the operation code is used in the source program. This method results in a unique identifier for every source statement audited during the execution of the program and the audited variable names identify the data being processed.
- step 210 changes to the conventional (existing) source statements can occur only if a branch is made to a statement label, and auditing is selected to audit the branch to that label, which results in an audit statement that is inserted at the label statement. These changes are normally made only to the copies of the expanded source program.
- step 211 the audit operation code profile for the source statement being processed inserts documentation comment statements for each source statement expanded.
- step 212 the entire expanded source program results are output to an expanded source program library.
- a new source program is created in an expanded source program library with the same program name.
- the entire input conventional source program is written to the expanded source program library including the inserted audit and documentation statements.
- the input conventional source library remains unchanged.
- step 213 the inserted audit and documentation source statements and the input of conventional program source statements are counted by program source statements and comment statements. Also, audit and documentation expansion statement errors, if any, are counted by error types.
- step 214 the expanded source program audit messages 31 printer file is utilized to print an expanded source program audit messages report. Needed audit expansion or documentation messages may then be printed. The counts of input program source statements and comment statements, and the counts of the inserted audit and documentation program source and comment statements are printed at the end of the basic audit source program expansion. The elapsed time of the execution of the audit source program expansion program is also printed.
- step 215 if the expand source program with audits is successful, then the expanded source program is allowed to be submitted to the conventional programming language source program compiler 37.
- An executable program object with the basic audit functions in an expanded program object library is created by successful expanded source program compilation.
- the output conventional object library remains unchanged, which allows program execution to be performed by the object program either before or after basic audit source program expansion.
- Successful completion of the expand source program with audits means that no significant errors are detected, and inserted basic audit source statements should compile successfully in the conventional language compile of the expanded source program.
- the flowchart of the pseudo-code for the expand input with audits program, the pre-compiler program 200 is continued in Fig. 11.
- step 216 in the event that the expanded source program with audits is unsuccessful, then an error severity code is returned.
- the error severity code for the job is similar to the error severity code returned by conventional language compilers.
- step 217 once the pre-compiler program 200 is finished, the program is exited and returned to the calling program.
- the flowchart of the pseudo-code for the modify initial execution audit options program 300 is depicted in Figs. 12A and 12B.
- the modify initial execution audit options program 300 is the program that allows the dynamic modification of the auditing options.
- the first step 301 is to input the name of the conventional source program 27 by entering the name when prompted by the initial display screen, the modify initial execution audit options for program 41. It is important to note that the program execution options for a program may be changed while the program object is executing when confirmed later during the execution of modify initial execution audit options program 300.
- Step 302 requires the programmer to validate that the modify initial execution audit options program 300 has selected the audit compile options in the selected compile audit options file 22.
- Step 303 retrieves any initial or current initial execution options for the program from the initial execution options for program file 26 (Z$AUDITO) , together with the compile audit options for the program from the selected compile audit options file 22 (Z$AUDITC) .
- Step 304 displays the initial or current program execution audit options and all of the compile audit options are displayed.
- Step 305 allows all the compile options to be turned on or off as execution options, and allows for the ten auditing levels to be changed.
- Step 306 causes the current execution audit options for the program to be updated to the current execution audit options file 43 (Z$AUDITO) .
- the changed execution audit options will take effect immediately if the program is being executed.
- Step 307 enables a command key to be used to confirm that the audit execution options are complete and may be used for current auditing. It is indicated that the execution audit options have been changed for the program in the current execution audit options file 43 (Z$AUDIT0) , so that if the program is currently executing, and no auditing is taking place, the program is to start auditing with the current execution options.
- a command key is used to end the modify initial execution audit options program 300.
- step 309 the calling program is returned to once the modify initial execution audit options program 300 ends.
- 13A, 13B, and 13C are a flowchart of the pseudo-code for the create or change options and run audit analysis reporting program 400 (Z$PGM04) .
- step 401 the initial screen is displayed.
- This display or print execution audit analysis 51 screen display provides for the selection of the desired audit analysis function.
- Audit analysis reporting is provided for using a selection capability from a menu of functions.
- Output provided by the selection includes printing of audit execution output file information and creation of extracted audit execution output file information for formatted reporting. The printed and disk input and output may be viewed and scanned online using standard system utility programs and available utility programs.
- An option 402 is provided which allows for the entry of an audit analysis report name or the selection of a name from a display of existing audit analysis report names.
- the audit report name provides the ability to save previously entered audit report parameters for use again in the same, or a similar, audit analysis report.
- step 403 when a new audit analysis report name is entered, a menu for audit analysis reporting selection options which allows the selection of the form of output is provided. The selection of the desired audit execution output file information is also provided. Selection parameters include all keyed fields of the audit output file including Job, Program, User, Terminal, Date, and Time.
- step 404 when an existing audit analysis report name is entered, the create or change options and run audit analysis reporting program 400 retrieves the existing options for the audit analysis report name from the analysis reporting options file 53.
- the same menu of audit analysis reporting selection options is provided and indicates which options were previously selected. The same selection options that were available when creating a new audit analysis name are allowed.
- Step 405 provides a command key which allows exiting the program without updates. All selections are validated and, when there are no errors, a command key to save the audit analysis selected options to the analysis reporting options file 53 is provided by the entered audit analysis report name.
- Step 406 provides a command key to allow the audit analysis report to be run. As shown in step 407, if the audit analysis report is to be run, the selected audit analysis report options are processed against the audit execution output file 42
- Step 408 provides a command key to end the program, and step 409 returns to the calling program.
- Figs. 14A, 14B, and 14C are a flowchart of the pseudo-code for the create or change options and run formatted audit analysis reporting program 500 (Z$PGM05) .
- step 501 the initial screen is displayed providing for the selection of the desired formatted audit analysis functions.
- the formatted audit analysis reporting using a selection capability from a menu of selection and formatting functions is provided. It is important to note that printed and disk input and output may be viewed and scanned online using standard system utility programs and available utility programs.
- step 502 the entry of a formatted audit analysis report name or the selection of a name from a display of existing formatted audit analysis report names is allowed.
- the formatted audit report name provides the ability to save previously entered formatted audit report parameters for use again in the same or similar formatted audit analysis report.
- step 503 if a new formatted audit analysis report name is entered, a menu of formatted audit analysis reporting selection options is provided which allows for the selection of the form of output and for the selection for the desired extracted audit execution file information.
- the selection parameters include all keyed fields of the extracted audit execution formatted file including Job, Program, User, Terminal, Date, Time, and other parameters selected when the file was formatted.
- the existing options for the formatted audit analysis report name are retrieved from the formatted audit analysis reporting options file 63.
- step 505 a command key is provided to allow exiting the program without updates, all selections are validated, and when there are no errors, another command key is provided to save the formatted audit analysis selected options to the formatted audit analysis reporting options file 63 (Z$AUDITS) by the entered formatted audit analysis report name.
- Step 506 provides a command key to allow the formatted audit analysis report to be run .
- step 507 if the formatted audit analysis report is to be run, then the selected formatted audit analysis report options are processed against the extracted audit execution formatted file (Z$AUDITE) to produce the desired display or printer file (Z$AUDITR) of the formatted audit analysis reporting 65.
- Step 508 provides a command key to end the program and step 509 returns to the calling program .
- Figs. 15A, 15B, and 15C are a flowchart of the pseudo-code for the modify compile and initial execution audit options program 600.
- step 601 the initial screen is displayed and the program allows a name to be entered. This program is utilized to input previously expanded source programs and to allow the removal of all or selected audit source statements. This allows the complete undoing of the expanded source audit statements back to the original conventional input source program, or the partial undoing of the expanded source audit statements. The program may then be expanded with additional audit statements.
- step 602 the expanded source program is retrieved from the expanded source program library into a work file, the source program is scanned for all audit statements, and the audit functions are summarized in the same display as used in the create or change compile and initial execution audit options program.
- step 603 the audit functions in the expanded source program are displayed, and any or all of the audit functions are allowed to be removed from the work file of the input expanded source program.
- step 604 the initial (current) execution options are retrieved and displayed for the expanded source program from the initial execution audit options file 26
- Step 605 provides a command key to allow exiting the program without updates, all selections are validated, and when there are no errors, a command key is provided to save- the source program from the work file back into the expanded source program library. Any changes to the initial execution is updated to the initial execution audit options for program file 26.
- Step 606 produces a variety of printouts. A summary of the auditing options removed and the audit options -remaining in the expanded source program are printed. Informational, warning, and error messages are printed as required, including counts of the input and output source statements.
- Step 607 provides a command key to call the create or change compile and initial execution audit options program to allow additional audit functions to be selected.
- Step 608 provides a command key to end the program, and step 609 returns to the calling program.
- the invention utilizes the expanded source program and the audit statements as the basis for analysis of the program execution audit output.
- the invention provides for more rapid understanding and correction of errors, problems, or programming logic than is possible with the conventional source program.
- the source program is expanded with standard comprehensive program documentation. Examples of the documentation provided are file names, file keys, and call program names. This expansion results in source programs being significantly faster to write and test by the original programmer. Also, the program is much easier to read and understand by programmers unfamiliar with it, and therefore programmers can more easily modify the program.
- the invention provides default audit compile profiles which provide for typical auditing environments such as first program test audits, completed program test audits, pilot production audits, and production environment audits.
- the default audit profiles simplify the process of selecting auditing options to be included in the expanded source program.
- the invention allows a selected audit profile to be modified for the selected program and used when the conventional source program is expanded with audit statements .
- the invention allows virtually any executable source program statement to be audited.
- the invention also allows virtually any file in the source program to be audited, together with desired file operation codes, and the data for all key fields used in the audited file operations.
- the invention provides ten levels of auditing which may be dynamically changed during program execution. Any executable operation code may be audited together with all fields referenced in each audited statement.
- the invention provides standard audit operation code audit prefix codes beginning with the characters Z$, and with a suffix code unique to the operation code.
- the invention allows virtually any field or label used in the source program to be audited when either referenced or modified, together with the data value of each audited field.
- the invention provides select and omit functions which qualify the auditing of operation codes by field name(s) .
- the invention provides program default initial execution profiles, which are provided for typical initial execution environments, such as first program test, completed program test, pilot production, and production environment.
- the default initial execution profiles simplify the process of auditing the execution of programs by providing frequently selected audit execution options.
- the invention allows a selected execution profile to be modified for the selected program and used during the execution of the program.
- the invention provides for dynamic modification of program auditing during program execution by changing execution auditing options, or turning auditing on or off completely. Audits must have been previously selected for potential audit and included in the expanded source program.
- the invention optionally outputs audit data to a disk, and provides for the retention of the audit data for both test and production environments.
- the invention also provides an audit database in date and time sequences from all audited programs. This provides for analysis of audited statements across all audited programs by date and time sequence. The analysis may include program, user, date and time range, audit code prefix, file, field, label, and/or any execution audited data value.
- the invention provides formatted audit reporting of historical audit information for use by internal and external auditors at a level of detail and auditability not possible previously.
- the invention reduces the cost of developing and maintaining source programs by significantly reducing the skills and training needed by programmers, and reduces much of the risk involved in implementing and maintaining programs.
- the invention also reduces the number of programmers required to develop and maintain programs.
- the invention provides the real-time detailed audit information needed for advanced event detection and event action software functions. This may be advantageous in a large variety of computer operation settings and is particularly useful with multiple programmers and users that are utilizing the same programming language on the same compiler on a remote computer.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Stored Programmes (AREA)
Abstract
Description
Claims
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US39831099A | 1999-09-20 | 1999-09-20 | |
US398310 | 1999-09-20 | ||
US09/611,210 US6775827B1 (en) | 1999-09-20 | 2000-07-06 | Real-time program audit software |
US611210 | 2000-07-06 | ||
PCT/US2000/018816 WO2001022644A2 (en) | 1999-09-20 | 2000-07-11 | Real-time program audit software |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1354270A2 EP1354270A2 (en) | 2003-10-22 |
EP1354270A4 true EP1354270A4 (en) | 2004-09-15 |
Family
ID=27016203
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP00948614A Withdrawn EP1354270A4 (en) | 1999-09-20 | 2000-07-11 | Real-time program audit software |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP1354270A4 (en) |
AU (1) | AU778165B2 (en) |
CA (1) | CA2385433A1 (en) |
WO (1) | WO2001022644A2 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2347647A1 (en) | 2001-05-15 | 2002-11-15 | Ibm Canada Limited-Ibm Canada Limitee | Storing and restoring snapshots of a computer process |
US7953688B2 (en) | 2007-05-08 | 2011-05-31 | Sharon Sadeh | Method and system for facilitating a compliance audit using a rule set |
CN112732640B (en) * | 2020-12-28 | 2023-07-25 | 杭州迪普科技股份有限公司 | Method and device for detecting leakage of file descriptor |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6011920A (en) * | 1995-04-05 | 2000-01-04 | International Business Machines Corporation | Method and apparatus for debugging applications on a personality neutral debugger |
JP3290567B2 (en) * | 1995-08-24 | 2002-06-10 | 富士通株式会社 | Profile instrumentation method |
US5771385A (en) * | 1996-03-29 | 1998-06-23 | Sun Microsystems, Inc. | Setting and getting system debug flags by name at runtime |
JPH1063550A (en) * | 1996-08-23 | 1998-03-06 | Fujitsu Ltd | Executing performance analytic display method and medium stored with program executing the method |
US6071316A (en) * | 1997-09-29 | 2000-06-06 | Honeywell Inc. | Automated validation and verification of computer software |
-
2000
- 2000-07-11 CA CA002385433A patent/CA2385433A1/en not_active Abandoned
- 2000-07-11 AU AU62088/00A patent/AU778165B2/en not_active Ceased
- 2000-07-11 EP EP00948614A patent/EP1354270A4/en not_active Withdrawn
- 2000-07-11 WO PCT/US2000/018816 patent/WO2001022644A2/en not_active Application Discontinuation
Non-Patent Citations (6)
Title |
---|
AGRAWAL H ET AL: "DEBUGGING WITH DYNAMIC SLICING AND BACKTRACKING", SOFTWARE PRACTICE & EXPERIENCE, JOHN WILEY & SONS LTD. CHICHESTER, GB, vol. 23, no. 6, 1 June 1993 (1993-06-01), pages 589 - 616, XP000655847, ISSN: 0038-0644 * |
ANONYMOUS: "PL/I Trace Program. September 1970.", IBM TECHNICAL DISCLOSURE BULLETIN, vol. 13, no. 4, 1 September 1970 (1970-09-01), New York, US, pages 855 - 857, XP002288961 * |
COPPERMAN M ET AL: "POOR MAN'S WATCHPOINTS", ACM SIGPLAN NOTICES, ASSOCIATION FOR COMPUTING MACHINERY, NEW YORK, US, vol. 30, no. 1, 1995, pages 37 - 44, XP000504825, ISSN: 0362-1340 * |
See also references of WO0122644A3 * |
TOLMACH A P ET AL: "Debugging Standard ML without reverse engineering", 1990, NEW YORK, NY, USA, ACM, USA, 1990, pages 1 - 12, XP002288960, ISBN: 0-89791-368-X * |
YAN J C: "Performance tuning with AIMS - an Automated Instrumentation and Monitoring System for multicomputers", SYSTEM SCIENCES, 1994. VOL.II: SOFTWARE TECHNOLOGY, PROCEEDINGS OF THE TWENTY-SEVENTH HAWAII INTERNATIONAL CONFERENCE ON WAILEA, HI, USA 4-7 JAN. 1994, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, 4 January 1994 (1994-01-04), pages 625 - 633, XP010096966, ISBN: 0-8186-5060-5 * |
Also Published As
Publication number | Publication date |
---|---|
AU6208800A (en) | 2001-04-24 |
EP1354270A2 (en) | 2003-10-22 |
AU778165B2 (en) | 2004-11-18 |
WO2001022644A3 (en) | 2003-08-28 |
WO2001022644A2 (en) | 2001-03-29 |
CA2385433A1 (en) | 2001-03-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6775827B1 (en) | Real-time program audit software | |
US6041330A (en) | System and method for generating year 2000 test cases | |
US6560776B1 (en) | Software installation verification tool | |
US8296734B2 (en) | System and method for testing a software product | |
US6421822B1 (en) | Graphical user interface for developing test cases using a test object library | |
US7555749B2 (en) | Software updating system and method | |
AU2012100128A4 (en) | A computer implemented system and method for indexing and optionally annotating use cases and generating test scenarios therefrom | |
US7475289B2 (en) | Test manager | |
US6332211B1 (en) | System and method for developing test cases using a test object library | |
KR19990023214A (en) | Database facilitating software installation and testing for custom computer systems | |
CN114840489A (en) | Database version changing method and device, storage medium and electronic device | |
WO2006135607A2 (en) | Method and system for rapid deployment and development of application and test software and documentation | |
AU778165B2 (en) | Real-time program audit software | |
JPH11224186A (en) | Device and method for analyzing software | |
US5961557A (en) | Design support system and method therefor | |
US8150821B2 (en) | System and method for using generic utilities to perform database utilities on mainframe operated DB2 databases | |
US8839097B2 (en) | System and method for information encapsulation for providing multiple deliverable formats from one information source | |
WO2008014387A2 (en) | Custom database system and method of building and operating the same | |
CN116204219A (en) | Processing method for software version update, electronic equipment and storage medium | |
JP5039683B2 (en) | Test support system | |
JPH08314758A (en) | Program test progression managing method | |
CN118467556A (en) | Database upgrading and rollback method and system thereof | |
CN118426811A (en) | BIOS updating method, device, equipment and readable storage medium | |
JP4467363B2 (en) | Product development support system and method, product test support system, and computer program | |
JPS60136836A (en) | Patch post-processing method of program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20020419 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK RO SI |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: 7G 06F 11/36 A |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20040803 |
|
17Q | First examination report despatched |
Effective date: 20041102 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20050513 |