AU711020B2 - System for virtually converting data - Google Patents

Description

Field of the Invention The present invention relates to software systems and in particular to software systems for virtualising arbitrary data within software system program and data files to any desired format.

Background of the Invention The Information Technology (IT) industry currently faces a huge problem as the year 2000 approaches. The problem is that programs and applications that represent dates using only two digits '96' rather than 1996) to represent the year will break down when they try to perform arithmetic operations, comparisons, or sorting of dates that lie beyond the year 2000. Incorrect results will be obtained when programs try to process dates that lie outside the range of January 1, 1900 to December 31, 1999. The scope of the year 2000 date change is very wide and includes both hardware and software from microcode on up to application program code on both 15 mainframes and desktop computers. It also includes both programs and database files and spans all computer platforms.

The year 2000 problem stems mostly from application programs and their corresponding data that use 2 digits to represent the year rather than 4 digits. A large number of these programs were written by application programmers back in the 1960's and 1970's who never expected that their software would still be used 20 or 30 years later in the year 2000. In addition, these programs were written at a time when every byte of storage memory space was valuable. Every opportunity to reduce the storage requirements of programs and databases was taken advantage of. As a result, 25 2 digits to represent year data was widely used and quickly became the customary practice of the trade. Rather than being tossed out, as time passed, new features were added and updates made to these legacy applications, mostly written for mainframe and minicomputers, in order to keep them current.

The year 2000 problem is compounded when one considers the possible ramifications of using 2-digits to store the year. Any arithmetic calculation that subtracts year data runs the risk of being off when using years 2000 and beyond. Two-digit date information cannot be compared correctly if the dates are in different centuries. In addition, sorting algorithms will not sort correctly records that include years 2000 and beyond.

For example, the year 2003, represented by 03, will be sorted before the year 1996, represented by 96. In addition, leap year and other day/week month calculations will be off. For example, the year 2000 is a leap year whereas 1900 is not. In addition, the dates of the week and month are not the same in, for example, the years 2001 and 1901.

Several solutions to this problem have already been proposed. One solution is to expand the date field so it stores a 4-digit year field. This appears to be the appropriate solution for making the software system year- 2000 compliant. No change to the logic of the program is required, only obvious changes to date related portions. Although elegant, a disadvantage of this approach is that changing the data field length requires simultaneous conversion of both program and data files. All software programs that reference the converted data file must also be converted to handle the 4-digit year format. This is either impossible or extremely difficult to implement when considering that the typical data processing shop may have a combined 15 number of modules and database files in the range of 15,000 to 30,000. Thus, the link between program and data files makes it necessary to build encapsulated clusters of programs and data files that can be converted and then implemented, independent of other programs and files in the system.

Due to the interrelated nature of most systems, these clusters tend to become extremely large and may contain hundreds and hundreds of programs and data files.

Another solution is to use a windowing technique which externalises the 2-digit year into a 4-digit representation. This technique uses a fixed or rolling window of 100 years to determine whether the 2-digit year is 19xx or 20xx. A number of years to be used for each century is specified. For example, numbers 00 to 20 might represent years 2000 to 2020. Thus the numbers 21 to 99 represent the years 1921 to 1999. An advantage of this technique is that there is no need to expand the 2-digit year to a 4-digit format. A disadvantage is that a problem exists if years beyond the 100 year range are needed. In addition, any databases that use the 2-digit year as an index will not work because of the discontinuous step from 20 to 21. And from 99 to 00.

A third solution is to encode and compress the 2-digit date field to hold 4-digits. An advantage of this solution is that there is no need to expand the 2-digit year data format to 4-digit year format. A disadvantage of this solution is that all programs that access this data file must be modified to handle the compressed/encoded 4-digit year format.

Illustrated in prior art FIG. 1 is a diagram showing the potential problems involved in using some of the prior art solutions discussed above.

Depicting a small group of programs, Programs A 12, B 14, C 16 and D 18 are shown accessing one or more related data Files A 20, B 22, C 24 and D 26.

Program A, for example, is shown with links to Files A, B and C. If Program A is converted to 4-digit year format, before it can be released into production, data Files A, B and C must also be converted, otherwise Program A will not work. Program B, however, accesses Files A, C and D, thus changing Program A also effects the operation of Program B. Therefore, it can be seen in a typical scenario the conversion of one program or data file cannot be done independently of other programs and/or data file, since they typically are interdependent among one another.

.15 Summary of the Invention According to a first aspect, the present invention provides a data adapting system for virtualising an arbitrary item of data derived from a data source, having an original format, so as to be presented in a desired format to a program, said system utilising a data conversion definition, said data conversion definition containing for each record type in said data source including said data item, indicia of fields in said record type used to identify said record type, including the values of the fields, and data fields and their corresponding format, at least one of said data fields corresponding to the data item to be virtualised, said system comprising: at least one conversion routine mapping the data fields within said records of said data source to the desired format in accordance with the data conversion definition; and a data adaptor operative to: intercept read requests from said program and, utilising said conversion routine, substitute in a data item read from said data source, data having said desired format for data having said original format, and intercept write requests from said program and, utilising said conversion routine, substitute in a data item written to said data source, data having said original format for data having said desired format.

According to a second aspect, the present invention provides a data adapting system for providing a program with date information in a virtual second date format derived from data in a first date format stored in a data source, said system utilising at least one data conversion definition containing indicia of record types in said data source, fields in said record type used to identify said record type and the relevant values of the identifying fields, and date fields and their corresponding format, said date fields corresponding to date information to be virtualised by said system, said system comprising: at least one conversion routine mapping the data fields within said records of said data source to the second format in accordance with the data conversion definition; and a data adaptor for: intercepting read requests from said program and, utilising said 15 conversion routine, substituting in a record read from said data source, information in the second date format for information in the first date format, and intercepting write requests from said program and, utilising said S. S conversion routine, substituting in a record written to said data source, information in the first date format for information in the second date format.

According to a third aspect, the present invention provides a method of virtualising an arbitrary item of data, having a first format, contained in a data source, so as to be presented in a second format, said method utilising a data conversion definition, said data conversion definition containing indicia 25 of record types in said data source, fields in said record type used to identify said record type and the relevant values thereof, and data fields and the corresponding format thereof, said data fields corresponding to the data to be virtualised, said method comprising the steps of: generating a conversion routine for mapping the first format data fields within said records of said data source to the second format in accordance with the data conversion definition; intercepting read requests from said program; substituting, utilising said conversion routine, in a data item read from said data source, data having said second format for data having said first format; intercepting write requests from said program; and substituting, utilising said conversion routine, in a data item to be written to said data source, data having said first format for data having said second format.

According to a fourth aspect, the present invention provides a system for transforming data from a data source having a first format into data having a second format for use by a program, said system comprising: a. at least one conversion routine for mapping the first format into the second format in accordance with a data conversion definition; and b. a data adaptor operative to: intercept read requests from said program and, utilising said conversion routine, substitute for data in said first format read from said data source, data having said second format, and ii.) intercept write requests from said program and, utilising said conversion routine, substitute in data written to said data source, data having said first format for data having said second format.

The present invention solves the problem inherent in the prior art solutions by providing a mechanism of virtualising any arbitrary data field in V a data file so as to be presented having any desired format. This mechanism is directly applicable to the year 2000 data problem described above. In particular, in the year 2000 context, the present invention operates by *severing the link between the program conversion process and the data file conversion process.

The present invention assumes control each time a converted program reads or writes a record of a data file that has not yet been converted. Converting S 25 records during I/O operations, enables a converted program to access an "unconverted data file as if it were converted. The invention converts the date fields in a read record to a new format that is implementation specific before passing it on to the program. In the converted program, the dates are defined in the new format 4-digit year format) which the program expects to see. Unconverted programs continue to access the actual unconverted data file. Thus, the present invention provides the converted program with a virtual converted view of an unconverted data file. The invention similarly converts data having the new format, 4-digit year format dates) to the old format before writing records to an unconverted data file.

SRA4,/

/X.

Using the present invention, programs can now be converted and implemented in production before the files they use are converted. This enables program conversion and data file conversion to proceed independently of one another thus making planning and management much easier. Thus, small, compact clusters that contain programs only can be built. This allows more thorough testing of the conversion process. If a problem arises with a converted program, it is much easier to go back and use earlier versions of that particular program. In addition, using smaller segments, eliminates the need to tie up many programs for long periods of time. This reduces disruptions for the entire system. The chances of upsetting the daily work while transferring the converted programs to production is now significantly less than the case where large clusters of programs and data files are transferred. The present invention thus makes software conversion easier, cheaper, quicker and most importantly safer than solutions based on the prior art.

S"Accordingly, one aspect of the present invention provides a system of virtualising any arbitrary data field of a data file as to be present in the data file in any desired format. In particular, enable legacy software applications may be adapted to the upcoming change in year from 1999 to 2000.

Embodiments of the present invention preferably provide a system of adapting legacy application programs to the year 2000 that only requires "-*conversions to application programs and allows data files to remain intact with original 2-digit year format data.

S"Other embodiments of the present invention may eliminate the S 25 requirement that all application programs and data files be converted to a new, 4-digit year, format at once. Initially, application programs can be converted and subsequently data files may be converted.

Thus, there is provided in accordance with a preferred invention, a data adapting system for virtualising an arbitrary data field, having an original format, of a data file so as to be presented in a desired format, the system utilising a data definition file created by a user, the data definition file containing all record types in the data file, fields in the record type used to identify the record type, including their values, and data fields and their corresponding format, the data fields corresponding to the data to be virtualised, the system including a conversion routine generator operative to generate a conversion routine utilising the data definition file, the conversion routine containing displacements and offsets of the data fields within the records of the data file, and a data adaptor operative to intercept input/output read requests from the program and, utilising the conversion routine, substitute in a record read from the data file, data having the desired format for data having the original format, the data adaptor operative to intercept I/O write requests from the program and, utilising the conversion routine, substitute in a record written to the data file, data having the original format for data having the desired format. In addition, the original data format is a 2-digit year format and the desired data format is a 4-digit year format.

There is also provided, in accordance with a preferred embodiment a data adapting system for providing a program with virtual 4-digit year format date information derived from 2-digit year format stored in a data file, the system utilising a Data Definition file created by a user containing all record types in the data file, fields in the record type used to identify the record 15 type, including their values, and date fields and their corresponding format, the date fields corresponding to date information to be virtualised by the system, the system including a conversion routine generator operative to generate a conversion routine utilising the Data Definition file, the conversion routine containing displacements and offsets of the date fields within the records of the data file, and a data adaptor operative to intercept *input/output read requests from the program and, utilising the *conversion routine, substitute in a record read form the data file, 4-digit year format information for 2-digit year format information, the data adaptor operative to intercept I/O write requests from the program and, utilising the conversion routine, substitute in a record written to the data file, 2-digit year ~format information for 4-digit year format information.

The system also includes a program generator means for generating, utilising the Data Definition file, a program that is operative to determine the displacements and offsets of the date fields in the date file. In addition, the program generator means includes a high level language program generator, the high level language generator utilising the Data Definition file to generate a high level program, the high level program operative to determine displacements and offsets of the date fields in the date file, the assembler program generator processing output of the high level program to generate the conversion routine.

In addition, there is also provided in accordance with a preferred embodiment of the present invention, a method of virtualising an arbitrary item of data, having an original format, so as to be presented in a desired format, the method utilising a data definition file created by a user, the data definition file containing all record types in the data file, fields in the record type used to identify the record type, including their values, and data fields and their corresponding format, the data fields corresponding to the data to be virtualised, the method includes the steps of generating a conversion routine, utilising the data definition file, the conversion routine containing displacements and offsets of the data fields within the records of the data source, and intercepting read requests from the program, substituting, utilising the conversion routine, in a data item read from the data source, data having the desired format for data having the original format, intercepting write requests from the program, and substituting, utilising the 15 conversion routine, in a data item to be written to the data source, data having the original format for data having the desired format, There is also provided in accordance with a preferred embodiment of the present invention a method of providing a program with virtual 4-digit year format date information derived from 2-digit year format stored in a data file, the method utilising a Data Definition file created by a user containing all record types in the data file, fields in the record type used to identify the record type, including their values, and date fields and their corresponding format, the date fields corresponding to date information to be virtualised by **the method, the method including the steps of generating a conversion 25 routine, utilising the Data Definition file, the conversion routine containing displacements and offsets of the date fields within the records of the data file, and intercepting input/output read requests from the program, substituting, utilising the conversion routine, in a record read from the data file, 4-digit year format information for 2-digit year format information, intercepting I/O write requests from the program, and substituting, utilising the conversion routine, in a record to be written to the data file, 2-digit year format information for 4-digit year format information.

Brief Description of the Drawings The invention is herein described, by way of example only, with reference to the accompanying drawings, wherein: FIG. 1 is a prior art illustration of a sample scenario involving multiple application programs accessing multiple data files; FIG. 2 is a high level block diagram illustrating the application of the data adaptor of the present invention to a sample data processing scenario; FIG. 3 is a high level block diagram illustrating the data adaptor system of the present invention applied to a sample data processing scenario; FIG. 4 is a high level block diagram illustrating in more detail the data adaptor system of the present invention; FIG. 5 is a high level flow diagram illustrating the initialise routine of the present invention; FIG. 6 is a high level flow diagram illustrating the 'OPEN' hook routine of the present invention; FIG. 7 is a high level flow diagram illustrating the Input/Output (I/O) read routine of the present invention; ge:: 15 FIG. 8 is a high level flow diagram illustrating the I/O write routine of the present invention; FIG. 9 is a high level block diagram illustrating in more detail the S:i" conversion routing generator of the present invention; 4*

S

FIG. 10 is a high level block diagram illustrating in more detail the COBOL program generator of the present invention; FIG. 11 is the contents of a sample Data Definition file used in o conjunction with the conversion routine generator; FIG. 12 is high level block diagram illustrating the data adaptor system of the present invention as applied to the conversion of data files to 4-digit year format; and FIG. 13 is a data diagram illustrating the 8-byte representation of 4digit year format data encoded and compressed into the original 6-byte 2digit year format.

Detailed Description of the Invention A high level block diagram illustrating the application of a data adaptor, generally referenced 10, of the present invention to a sample data processing scenario is shown in FIG. 2. For purposes of teaching the principles of the present invention, an example of the year 2000 date problem discussed above is used throughout this disclosure. The illustration of the present invention using the year 2000 problem in no way limits the suitability of the present invention to other non date related applications. In particular, the present invention can be applied to any system where it is desired to virtually change the formatting of arbitrary data fields in a data source such as a data file, computer screen memory, magnetic tape, etc.

In connection with the year 2000 date problem and with reference to FIG. 2, Program A 28 represents an unconverted program and Program B represents a program that has been converted. The term converted refers the process of changing data from one type of format to another type of format.

In the year 2000 problem example, the term converted refers to the process of changing every 2-digit year reference in the original program file to 4-digit year format. This includes any reference to dates, comparison of dates, arithmetic operations on one or more dates, etc. Unconverted data file 32 is the original date file that still retains the original 2-digit year format. Data adaptor 10 functions to intercept all I/O calls from Program B and perform a virtual change of data format. In this example, all 2-digit year dates obtained 0* 15 from unconverted data file 32 are replaced with 4-digit year format dates.

0 Thus, the fact that data adaptor 10 is intercepting all I/O calls is transparent to Program B. It expects and receives 4-digit year formatted information.

Likewise, when writing, data adaptor 10 converts the 4-digit year format into 2-digit year format before actually writing the date data to the data file. Data *0 S 20 adaptor, thus allows both unconverted programs that still expect 2-digit year *format data to execute side by side with programs already converted to handle 4-digit format year data.

An advantage of using data adaptor 10 to virtualise data having an old format to data having a new desired format converting 2-digit date to and from 4-digit dates by intercepting or filtering all I/O requests) is that it 0* permits Management Information System (MIS) personnel, or anybody else responsible for handling data formatting changes, to convert application programs at their own pace. It is no longer necessary now to simultaneously place in to production all programs and data files at once. Conversion projects may proceed at any desired pace since both converted and unconverted programs can exist simultaneously. For data files to be converted, however, all program files that access that particular data file must have been previously converted.

Referring to FIG. 3, data adaptor 10 cooperates with one or more conversion routines 46 which define the desired transformation. Data adaptor 10 and conversion routines 46 are collectively referred to as data adaptor system. 42. In systems where flexibility is desirable to facilitate conversion of a variety of file formats, data adaptor system 42 suitably cooperates with a conversion routine generator 44, employed to generate one or more conversion routines 46 from one or more data conversion definitions (rules) 38 and indicia of a converted header, a converted header file provided by the user. A data conversion definition (rule) 38 is provided for each data item format to be converted. Conversion routines 46, in turn, are used by data adaptor 10 to convert selected data items date data) to and from first, e.g. 2-digit year and second, e.g. 4-digit year, formats. A data conversion definition 38 and a converted header file 40 are input to conversion routine generator 44. Indicia of a header corresponding to the original, unconverted date, an old header file 34 is input to nonconverted program 28 which accesses an unconverted data file 32. Data adaptor 10 accesses unconverted data file 32 in addition to receiving I/O 15 requests from a converted program 48. Converted header file 40 also is used to gather with the converted program as input to the compiler.

In a preferred embodiment, data adaptor 10 utilises one or more conversion routines 46 in performing the actual conversions of data to and from unconverted data file 32. The conversion routines may be externally S* 20 generated by conversion routine generator 44, described in more detail below, from data conversion definition(s) 38, and converted header file 00o0 Converted header file 40 defines the data structure after designated data items are transformed from the first to second format, contains all the data conversion declarations of the original program with date related data expanded to 4-digit year format. Data definition 38 suitably contains explicit definitions, position, type, etc., of date related data contained in the converted header file.

Alternatively, data adaptor system 42 may include an internal (integral) conversion routine generator, or a built in hard-coded) conversion routine suitable for the type of data to be converted. Use of a hard-coded conversion routine, created either manually or by other automated means, is of particular utility in instances where the different data types and formats involved in an application are relatively few in number, and unchanging.

Both converted and non-converted programs may operate side-by-side upon the same unconverted data file 32. Non-converted program 28, using the first, 2-digit year date format, accesses unconverted data file as normal. Converted program 48, however, has been converted to handle the second format, 4-digit year formatted date information. Data adaptor using conversion routines 46, effectively performs a conversion from first format to second format as part of each read operation, and a conversion from second format to first format as part of each write operation involving the designated data item, performs 2-digit to 4-digit conversions on date file reads and 4-digit to 2-digit conversions on date file writes.

FIGs. 4 through 13 and the accompanying descriptions assume the language and architecture of IBM mainframes and the COBOL programming language using sequential files. However, other platforms, databases and programming languages can easily be adapted in place of what is disclosed by one skilled in the art.

Referring to FIG. 4, a suitable data adaptor 10 comprises an execution- 15 time data adaptor 47, conversion routines 46, an 'OPEN' hook routine 58 and an initialise routine 56. Converted program 30 comprises a sequence of instructions/statements in which all references to the designated data items have been converted from the first to the second format, all data references in converted program 30 have previously been converted from 2- S 20 digit to 4-digit year formats. FIG. 4 illustrates the typical sequence of COBOL statements used in a converted program that has been updated to handle the second, 4-digit year, format data.

In a preferred embodiment, converted program 30 triggers data adaptor 10 through a single call to initialisation routine 56, added at the beginning of the application program.

:0 No other change to the program or the program's job control language (JCL) is required to launch the data adaptor. Once flagged, the data adaptor interprets all I/O operations on converted data files and activates the conversion routines. The use of conversion logic external to converted program 30, invoked by a single call, tends to simplify the programming that would otherwise be required to permit converted program 30 to accommodated unconverted data.

The call to initialisation routine 56 identifies the files that are to be converted and associates those files with corresponding conversion routines 46. Once flagged, the data adaptor interprets all I/O operations on converted data files and activates the conversion routines. The parameters of the call 13 include pairs of file names. The first file name is the name of the data file that is to be handled and the second is the name of the conversion routine 56 associated with the data file specified in the first file name. More than one [data file, conversion routine] pair may be listed in the call to the initialise routine 56.

Initialise routine 56 in adaptor 10 functions to set up hooking or trapping 'OPEN' command or statement. When an 'OPEN' command is encountered, 'OPEN' hook routine 58 is executed. The OPEN hook routine, described in more detail below, functions to set up additional hooking for I/O reads and writes and then calls the original 'OPEN' routine.

When a read or write statement is reached, in the converted program program control is switched to data adaptor 10. The data adaptor utilises conversion routines 46 in converting or virtualising data having the original format to data having the desired new format. In particular, in the year 2000 S 15 example, 2-digit year data is converted to and from 4-digit year data. The data adaptor also calls an original I/O routine 62 to actually perform the read r the write to unconverted data file 64. Thus, the data is virtualised and the application program is unaware that the data originally had the old formatting.

20 Co-existing simultaneously with converted program 30, non-converted program 52 suitably directly accesses 2-digit year information from unconverted data file 64.

Referring now to FIG. 5, when a call to initialise routine 56 is made, the first step performed is to install hooks or software trapping using SVC screening techniques, well known to hose skilled in the art (step 72). Next, :the conversion routines are loaded into memory for speedy access (step 74).

Referring now to FIG. 6 when, as the converted program executes, an 'OPEN' command is eventually reached, program control is passed to hooked 'OPEN' routine. As described previously, hooking is one of the functions performed by initialise routine 56. In order to start converting data information from the file, a determination is made as to whether the data file included as a parameter in the OPEN statement is one of the data files listed in the call to the initialise routine (step 80). If the file is not one of the files in the set of parameters to the call statement, control is passed to original OPEN routine 60. If the file is on the call statement list, it is then determined whether the data file is to have its, date year information converted (step 82), suitably by reference to an indication of whether to convert a data file is made in the data conversion definition 38 (FIG. 3).

Even after date year information in a data file associated with a converted program is converted from first to second format, 2-digit to 4digit year format, it may still be desirable to retain the associated statements CALL initialise, OPEN, etc.) in the converted programs. Removing these statements, initially needed to set up the year information, requires additional testing and maintenance time, and there will likely be hundreds of such converted programs that do not need this extra code because all their associated data fields have already been converted to 4-digit year format.

However, although current working date fields have been converted to 4-digit year format, it is very likely that historical data will not be converted. Many data processing shops may find that the sheer number of historical data files is too voluminous to attempt full conversions. Thus, the slight additional S 15 code to trigger the mechanisms of adaptor 10 are likely to remain in S.o.

*:converted programs and a mechanism is needed to know whether to convert a given data file or not. As described in more detail below, once a data file is converted it is a simple matter to indicate this fact in the Data conversion definition 38.

20 In the exemplary context of a COBOL system, once it is determined that a given data file is to be converted, the next step is to modify the Data Control Blocks (DCBs) that were provided by the application program to S: accommodate those attribute(s) such as record length, is different where the actual presentation and virtual presentation may differ. In addition, I/O routine hooking is installed by modifying the access method routine in the corresponding DCB (step 84). The DCB exit is also modified in order to properly install I/O routine hooking (step 86). Once I/O hooking is in place, the original OPEN routine 60 is executed (step 88).

Execution-time data adaptor 47 will now be described in more detail.

Data adaptor 10 comprises an I/O read process (FIG. 7) and an I/O write process (FIG. Referring now to FIGs. 4 and 7, when a read statement/command is encountered in converted program 30, the original I/O read routine 62 (FIG. 4) is initially executed. Next, the conversion routine 46, generated previously, corresponding to the data file, is called to perform the actual first format to second format, 2-digit to 4-digit year format, conversion (step 92). The converted, 4-digit year, format data is passed to the calling converted application program 30 (step 94).

Referring now to FIGs. 4 and 8, when a write statement/command is encountered in converted program 30, the record to be written to the data file is passed to the hooked I/O write routine of FIG. 8 (step 100). The data file's corresponding conversion routine 46 is called to perform the actual second format to first format, 4-digit to 2-digit, conversion on the record (step 102). Once converted, the original I/O routine 62 (FIG. 4) writes the record to the data file 32. Thus, the I/O read and write routines of the present invention permit the data file 32, which retains its original, 2-digit year, format data, to be virtually converted to the second, 4-digit year, format.

As previously noted, conversion routines 46, may be hard-coded or generated by an internal (integral) conversion routine generator. However, in instances where a number of different data types and formats involved in an 15 application, the application operates upon data having a number of different record formats, conversion routines 46 are preferably generated by conversion routine generator 44 external to adaptor 10, to maximise flexibility. From a user's perspective, use of conversion routine generator greatly simplifies the task of defining lengths and offsets of the data fields S 20 within a data file that need to be converted; a user is able to specify lengths and offsets in the language of the application, COBOL, rather than in a more cryptic type fashion.

S. With reference to FIGs. 3 and 9, conversion routine generator 44 will now be described in more detail. With reference to FIG. 3, the input to the conversion routine generator are Data conversion definition 38 a data definition file) and converted header file 40 are provided as inputs to conversion routine generator 44. Converted header file 40 is the original header file modified in accordance with the second format, e.g. with expanded date field sizes (2-digit year fields have been expanded to 4-digit year fields). Indicia of data conversion definition 38 is input to a suitable program generator (step 110). The programming language COBOL is suitably used in the exemplary embodiment here described. However, other programming languages would work just as well. One skilled in the art could substitute other programming languages and other platforms to perform the teachings of the present invention.

The function of the program generator 110 is to generate a program in COBOL) that can be used to determine for all the data fields in the data file, the lengths and byte offsets used in the conversion routine, and whether or not they are to be converted or not to 4-digit year data).

The COBOL program source output from the COBOL program generator is compiled and linked (step 112) using a standard COBOL compiler and linker. The converted header file 40 is used as input during the step of compiling and linking. The converted header file 40 provides all the data file data definitions structures, arrays, variables, etc.) needed by the COBOL program to access the data file. The output object code is then executed (step 114).

The executed program preferably uses a novel technique to determine the offsets, displacements, size and type of the data field and other variables within the data file. The technique involves filling a record with hex FF in S 15 every byte. Data are then selectively written to the record. The record is then scanned and the position and length of the non-FF data is determined.

The non-FF data corresponds to the data fields within the record. This technique is performed by assembler routines invoked by the COBOL program.

S 20 The binary length and offset and (displacement), output of the COBOL program is input to an assembler program generator (step 116). The assembly "program generator uses the output of the COBOL program to generate the source code for the conversion routines. The assembly source code is then compiled assembled) and linked using standard development tools (step 118). The object code output from the conversion routine generator is the S" executable conversion routine.

The COBOL program generator will now be described in more detail.

A high level block diagram illustrating the syntax structure used by the COBOL program generator is shown in FIG. 10. Once the syntax structure is defined, it is a straightforward process to one skilled in the art to generate corresponding COBOL source code. A one to one relationship exists between the syntax structure illustrated in FIG. 10 and the COBOL generator source code.

Data files sometimes include records with different formats, or records with varying information. Data conversion definitions 38 include the information necessary to identify the various types of records, as well as the particular logic to be employed in the conversion. Data conversion definition 38 suitably includes a header definition 132 followed by a body 140.

Header definition 132 suitably includes a FILE-ID structure 134, a default statement 136 and a conversion protocol 138. In the exemplary context of year 2000 application, conversion protocol 138 suitably comprises a window-year protocol statement defining the lowest date assumed to be in the 20th century; all earlier dates (lower numbers) are assumed to be in the 21st century.

The body 140 of a data conversion definition 38, can be either a single type record definition 142 or a multiple type record definition 162. A single type record 142 suitably includes a record statement 144 followed by a data description 146. The data description 146 can be an array definition 148 or it can be one or more data type definitions 150 (indicated by the star symbol). An array definition 148 suitably includes an array statement 152, a 0 15 recursive data description 154 and an end array statement 156. A data type definition 150 suitably begins with a data item, date, type name (of the o0 format YYMMDD, for example) followed by a list of data names of this type 160. If desired, conversion protocols different from the protocol specified in statement 138 of the header definition 132 can be utilised for different data 20 item types. In such case, a data type protocol statement 161, windowyear protocol, may be provided as part of the data type definition.

A multiple type record definition 162 suitably includes one or more 0 record type statements 163 (indicated by the star symbol). Each record type statement 163 suitably includes a record header statement 164, recursive body 166 and end record type statement 168. A record type header 164 suitably includes either a record type statement 165 or a default type statement 167. A multiple type record 162 may have many record type statements and one default type statement as the last record type header for this multiple type record.

An example of a Data conversion definition used as input to the conversion routine generator is illustrated in FIG. 11. The initial step in applying the data adaptor entails defining the data fields. For each data file all the record types in the data file must be defined. In addition, the fields in the record that are used to identify the record type, and their corresponding values per record type are defined. The data fields and their format per record type must also be defined. These definitions are passed to the data adaptor in a special syntax that accepts the same variable names used by the organisation's programming language, such as COBOL.

With reference to FIGs. 10 and 11, the FILE-ID 134 'BANKIN' is used as the reference to the data file and its generated conversion routines. The default statements 136, under 'DEFAULTS,' are to convert the data file and use old output formatting. The window-year statement 138 indicates a year window, meaning dates 00 to 20 are translated to 2000 to 2020 and dates 21 to 99 are translated to 1921 through 1999. The 'COPY BANKIN' statement functions to include the contents of the copy book or header file associated with the 'BANKIN' data file. The record type statement 164 'BK-RECORD- TYPE' follows the record definition section. The next four TYPE statements 142 describe various date fields in the 'BANKIN' data file. The last section is a DEFAULT-TYPE statement for the case when the data field does not fit any of the previous type statements.

S 15 The advantage of using this technique to define the date fields is that the definitions are simple and easily understood. In addition, the programming language data names are used to specify the date fields, making it unnecessary for the user to manually calculate the field's position, thus saving time and eliminating errors. In addition, the converted system is 'live' in that changes can be made to the record definitions at any time. To have the data adaptor use these changes only the Data Definition file needs to be recompiled and the changes will automatically be reflected in the conversion routines rebuilt by the conversion routine generator.

Even after both program and data file have been converted, it is often desirable to retain the conversion ability, to, for example, access historical archived) files. Referring to FIG. 12 in a typical data processing scenario a conversion routine 46 has been previously generated by the conversion routine generator 44 and employed to convert all the records in an unconverted data file 32, having 2-digit year format data, into a converted data file 172 at one time.

Once converted, unconverted data file 170 is no longer needed, since no other programs need access to it. Once converted, converted program 174 directly accesses data file 172 without the necessity of data adaptor 10. The conversion process can be disabled by changing the DEFAULTS statement in the Data conversion definition to 'CONVERT NO.' Making this change will prevent the data adaptor from operating on the data file, and the programs, without any need for recompilation, will receive the records directly from the file. If the programs need to be run using historical or backup files 176 that were not converted, the data adaptor can be triggered for a specific run by adding a control statement to the job control language (JCL).

In an alternative embodiment, the data adaptor can selectively convert the data files containing records with both first and second formats. For example, it may be desirable that a program have the capacity to operate upon both converted and non-converted data, such as in an instance where data is converted on-the-fly, as accessed in normal operation. In the context of a year 2000 application such a file might have two types of records: records having dates in the original 2-digit year format; and records with expanded 4-digit year format but which are stored in place of the old date, leaving the record length unchanged. The expanded date information is suitably stored as a 6-byte (11 digits) packed decimal number. A data 15 diagram illustrating the 8-byte representation of 4-digit year format data encoded and compressed into a packed 6 byte format is shown is FIG. 13.

Date field 180 is the 2-digit year format of the original unconverted data. Data field 182 is correspondingly converted 4-digit year data, as expected by the program stored in packed format in the same length (6 byte) 20 field as the original data field. Thus, the field length and record length of the data file remains unchanged. The data file contains a mixture of both year 0 *formats. The data adaptor uses the internal representation of these two types to distinguish between them. The program always receives expanded 4-digit year format data as the method in which the dates are stored is transparent to it. Alternatively, all of the records in data file 32 may be converted at one .time, as described above, and the conversion feature activated only when unconverted data must be accessed.

While the invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications and other applications of the invention may be made.

Claims (29)

1. A data adapting system for virtualising an arbitrary item of data derived from a data source, having an original format, so as to be presented in a desired format to a program, said system utilising a data conversion definition, said data conversion definition containing for each record type in said data source including said data item, indicia of fields in said record type used to identify said record type, including the values of the fields, and data fields and their corresponding format, at least one of said data fields corresponding to the data item to be virtualised, said system comprising: at least one conversion routine mapping the data fields within said records of said data source to the desired format in accordance with the data conversion definition; and a data adaptor operative to: intercept read requests from said program and, utilising said conversion 15 routine, substitute in a data item read from said data source, data having said desired format for data having said original format; and intercept write requests from said program and, utilising said 0.4. conversion routine, substitute in a data item written to said data source, data 0:6 to: having said original format for data having said desired format. 20 2. The system according to claim 1, wherein said original data format is a

2-digit year format and said desired data format is a 4-digit year format.

3. The system according to claim 1, wherein said data source comprises a S"data file.

4. The system according to claim 1, wherein said data source comprises computer screen memory. A data adapting system for providing a program with date information 0in a virtual second date format derived from data in a first date format stored in a data source, said system utilising at least one data conversion definition containing indicia of record types in said data source, fields in said record type used to identify said record type and the relevant values of the identifying fields, and date fields and their corresponding format, said date fields corresponding to date information to be virtualised by said system, said system comprising: at least one conversion routine mapping the data fields within said records of said data source to the second format in accordance with the data conversion definition; and a data adaptor for: intercepting read requests from said program and, utilising said conversion routine, substituting in a record read from said data source, information in the second date format for information in the first date format, and intercepting write requests from said program and, utilising said conversion routine, substituting in a record written to said data source, information in the first date format for information in the second date format.

6. The system according to claim 5, further comprising a conversion routine generator for, utilising said data conversion definition, generating said conversion routine

7. The system according to claim 6, wherein said conversion routine generator comprises: a first program generator utilising said data conversion definition 15 to generate a first program, said first program operative to determine the mapping of said date fields in said data source into the second format; and second program generator, said second program generator :processing the output of said first program to generate said conversion routine.

8. A method of virtualising an arbitrary item of data, having a first format, contained in a data source, so as to be presented in a second format, said method utilising a data conversion definition, said data conversion definition *containing indicia of record types in said data source, fields in said record type used to identify said record type and the relevant values thereof, and 25 data fields and the corresponding format thereof, said data fields corresponding to the data to be virtualised, said method comprising the steps of: generating a conversion routine for mapping the first format data fields within said records of said data source to the second format in accordance with the data conversion definition; intercepting read requests from said program; substituting, utilising said conversion routine, in a data item read from said data source, data having said second format for data having said first format; intercepting write requests from said program; and substituting, utilising said conversion routine, in a data item to be written to said data source, data having said first format for data having said second format.

9. The method of claim 8, wherein one of said data formats is a 2-digit year format and the other is a 4-digit year format. The method of claim 8, wherein the conversion routine generator generates a plurality of conversion routines in accordance with the data conversion definition.

11. The method of claim 8, wherein the data conversion routine includes indicia of field lengths and offsets for converting data from the original to the desired format.

12. The method of claim 8, wherein said first data format is a first date format and said second data format is a second date format.

13. The method of claim 12, wherein one of the date formats is a 2-digit 15 year format and the other is a 4-digit year packed decimal format.

14. The method of claim 8, wherein said data source comprises a data file.

15. The method of claim 8, wherein said data source comprises computer screen memory.

16. The method of claim 8, wherein the step of generating said conversion routine comprises: utilising said data conversion definition to generate a high level program, said high level program operative to determine the mapping of said S: date fields in said data source into the second format; and processing output of said high level program to generate said conversion routine.

17. The method of claim 8, wherein said data conversion definition further comprises, for at least one of the data fields to be modified, indicia of a conversion protocol.

18. The system of claim 1 further comprising a conversion routine generator operative to generate at least one conversion routine in accordance with the data conversion definition.

19. The system of claim 1 further comprising a conversion routine generator operative to generate a plurality of conversion routines in accordance with the data conversion definition. 23 The system of claim 18, wherein the data conversion routine includes indicia of field lengths and offsets for converting data from the original to the desired format.

21. The system of claim 1, wherein the data conversion routine includes indicia of field lengths and offsets for converting data from the original to the desired format.

22. The system of claim 1, wherein said original data format is a first date format and said desired data format is a second date format.

23. The system of claim 22, wherein one of the date formats is a 2-digit year format and the other is a 4-digit year format.

24. The system of claim 22, wherein one of the date formats is a 2-digit year format and the other is a 4-digit year packed decimal format. The system of claim 5, wherein one of the date formats is a 2-digit year •:*"format and the other is a 4-digit year format. 15 26. The system of claim 5, wherein one of the date formats is a 2-digit year format and the other is a 4-digit year packed decimal format.

27. The system of claim 8, wherein one of the date formats is a 2-digit year format and the other is a 4-digit year packed decimal format.

28. A system for transforming data from a data source having a first format into data having a second format for use by a program, said system comprising: e at least one conversion routine for mapping the first format into the second format in accordance with a data conversion definition; and a data adaptor operative to: 25 intercept read requests from said program and, utilising said conversion routine, substitute for data in said first format read from said data source, data having said second format, and (ii) intercept write requests from said program and, utilising said conversion routine, substitute in data written to said data source, data having said first format for data having said second format.

29. The system of claim 28 further comprising: a conversion routine generator operative to generate at least one conversion routine in accordance with the data conversion definition. The system according to claim 29, wherein said conversion routine generator comprises: a first program generator, said first program generator utilising said data conversion definition to generate a first program, said first program operative to determine the mapping of said date fields in said data source into the second format; and a second program generator, said second program generator processing output of said first program to generate said conversion routine.

31. The system of claim 29 wherein the data conversion definition comprises indicia of: record types in said data source; fields in said record type used to identify said record type and the relevant values of the identifying fields; and fields corresponding to data information to be virtualised by said system and the format corresponding thereto.

32. The system of claim 28, wherein the data conversion routine includes 15 indicia of field lengths and offsets for converting data from the original to the desired format. :33. The system of claim 28, wherein said first data format is a first date :format and said second data format is a second date format.

34. The system of claim 33, wherein one of the date formats is a 2-digit year format and the other is a 4-digit year format.

35. The system of claim 33, wherein one of the date formats is a 2-digit year format and the other is a 4-digit year packed decimal format. S36. The system according to claim 28, wherein said data source comprises a data file.

55.5 25 37. The system according to claim 28, wherein said data source comprises computer screen memory. 38. The system of claim 28 wherein the data conversion definition comprises indicia of: record types in said data source; fields in said record type used to identify said record type and the relevant values of the identifying fields; and fields corresponding to data information to be virtualised by said system and the format corresponding thereto. 39. The system of claim 1, wherein said data conversion definition comprises a header and a body; said header including indicia of file structure, and said body comprising indicia of record type, and indicia of data fields in said record type to be modified by the data conversion routine. The system of claim 39 wherein the header further includes indicia of a conversion protocol. 41. The system of claim 40 wherein the body further includes, for at least one of data fields to be modified by the data conversion routine, indicia of a conversion protocol. 42. The system of claim 39 wherein the body further includes, for at least one of data fields to be modified by the data conversion routine, indicia of a conversion protocol. 43. A data adapting system substantially as herein described and with reference to Figures 2 to 13 of the accompanying drawings. 44. A method of virtualising an arbitrary item of data substantially as herein described and with reference to Figures 2 to 13 of the accompanying 15 drawings. A system for transforming data substantially as herein described and with reference to Figures 2 to 13 of the accompanying drawings. a• Dated this fifth day of August 1999 S"VIASOFT, INC Patent Attorneys for the Applicant: ;Patent Attorneys for the Applicant: F B RICE CO