GB2293669A

GB2293669A - Software testing.

Info

Publication number: GB2293669A
Application number: GB9419748A
Authority: GB
Inventors: Timothy Ryan; Christopher Murphy; Anthony Foran
Original assignee: HOSKYNS TECHNOLOGIES IRELAND L
Current assignee: HOSKYNS TECHNOLOGIES IRELAND L
Priority date: 1994-09-30
Filing date: 1994-09-30
Publication date: 1996-04-03
Anticipated expiration: 2014-09-30
Also published as: GB9419748D0; BE1007160A6; GB2293669B

Abstract

A computer stores test instructions (10) for testing of a number of application software suites both during development on an in-house host system or running remote target systems. Target system codes and list files (5 and 6) allow automatic access to a sub-structure associated with a suite of application software. The sub-structure has levels which ensure access to the most recent program modifications and allow storage of programs for all target systems without the requirement for a very large storage capacity. Thus development, early and final stage testing, and support for new version or release testing of software on target systems can be carried out uniformly, quickly and correctly. <IMAGE>

Description

"A Software Testing Apparatus" The invention relates to an apparatus for the testing of application software.

In the specification, the term "testing" refers to the comprehensive testing before shipment of a piece or suite of software, both during development and at the final test environment stage and also to the on-going maintenance, development and support of the software after it has been shipped to the end user.

At present, many specific testing methods are known, one being described in European Patent Specification No. EP B1-0286361 (Westinghouse). However, one of the problems which is encountered by test engineers in testing of application software has been to ensure access to the most up-to-date version of the software running on a target system. Frequently, there are delays in accessing the correct programs and data files for a particular target system and errors are often caused by using an incorrect version of the software. This can lead to a large amount of time being wasted, loss of revenue and customer confidence, etc. This has particularly been the case for software development organisations where there are a large number of customer target systems running application software developed by the organisation.

One approach to this problem has been to gain on-line access via modem to the target system and to transmit fault diagnosis signals via the communications line.

While this is satisfactory to some extent, it does have limitations in the nature of the testing which can be carried out. For example, it is in some cases just not practical to carry out very comprehensive tests using the large volumes of data and programs used by the target systems. Further, this approach cannot provide quick access to the programs and data as they are being developed on the host system for situations where there are many different pieces of application software being developed.

The objects of the invention are to provide a software testing apparatus which: is capable of allowing near immediate access to all programs and relevant test data, both being developed in-house and remotely at on-site target systems; provides for correct access to the most up-to-date programs and data files in a simple manner with little programmer time input being required; provides for very comprehensive testing in all situations; and provides for efficiency in testing new application software versions for installation on a large number of target systems.

According to the invention, there is provided a software testing apparatus for testing of target system software developed on a host system, the apparatus comprising: a test controller connected to a read/write memory; a fixed disk array connected to a disk controller storing:- a file of security access control instructions; a set of software test instructions; and programs and data for software to be tested, an input interface connected to the test controller for reception of test commands; and an output interface, wherein the disk array stores a sub-structure for each suite of application software, the sub-structure comprising: a target-originating level containing a plurality of program sets, each set being associated with a target system and containing programs developed or modified on said target system; a host-originating level containing a plurality of program sets, each set being associated with a target system and containing programs for said target system and developed or modified on a host system for software development; a base level containing a single core set of programs common to all target systems; and a data level containing a plurality of test data files, each file being associated with a target system, wherein the test controller comprises means for: automatically determining the correct sub structure and, within said sub-structure, the correct program sets in the target-originating and host-originating levels and the correct data file in the data level for each test session initiating signal inputted at the input interface; and automatically directing retrieval of said program sets, the base programs and said data files by accessing the levels in target-originating, host originating, base and data order, and writing the retrieved programs and data to the read/write memory, retrieving test instructions and directing testing of retrieved programs and data in the read/write memory.

In one embodiment, the fixed disk array stores a file of target system codes, each code being associated with a testing session and the test controller comprises means for retrieving a target system code according to the received testing session initiation signals and for automatically providing access to and retrieving the relevant programs and data and the test instructions by decoding the target system code.

Preferably, the disk array stores a file of lists, each list being associated with a particular target system code and including the addresses for the relevant sub-structure programs and associated data files.

In another embodiment, the file of target system codes and the file of lists are dynamically updated by the test controller in real time.

In a still further embodiment, the test data files include sample test data for the associated target system, and the test controller comprises means for automatically updating a data file with live data from the remote target system whenever required for testing.

The invention will be more clearly understood from the following description of some embodiments thereof, given by way of example only with reference to the accompanying drawing which is a diagram illustrating a software testing apparatus of the invention.

Referring to the drawing, there is shown a software testing apparatus of the invention, indicated generally by the reference numeral 1. The apparatus 1 comprises a test controller 2 which is a micro-processor control circuit in an IBM AS400 Model 9406F.50 having 12.5Gb fixed disk and 96Mb of RAM.

The apparatus 1 is for testing of application software for target systems and developed on a host system. Testing of the software is during development, at the final stages of development, and after release for on-going support and development of new versions for the target systems.

The test controller 2 is connected to an input interface 3 which allows a test engineer to input instructions governing the testing operations which are to be carried out. The controller 2 reads a security access control file 4 which is stored on disk and which includes security control instructions to govern access by people to the stored data and programs. The file 4 is generated by the operating system of the apparatus 1 and includes access control instructions. These access control instructions are stored as a profile for each user of the apparatus 1 wherein each user has an indexed entry to a user definition type file also stored on disk. In this way a request to use the system is first checked to establish an authorised user identification code. When the user identity has been validated the indexed list of profiles is addressed to determine the access rights of that particular user. However, in addition, there is a further set of access control instructions particular to the testing level, to provide a level of security which is higher then that set by the operating system. This is achieved by defining a dedicated set of user definition types in addition to the operating systems definitions.

The access rights established in these definition types may for example preclude access to certain releases of software and/or development tools thereby ensuring that access to the correct development or testing environment is set by a project manager or security officer. By determining access in this way the possibility of accessing incorrect software is taken away from the test engineer.

Codes for objects (programs and data) of target systems are stored in a file 5 on a fixed disk array which is accessed by the test controller 2. These codes are directly associated with security access instructions of the file 4 so that each individual having access to the software testing apparatus 1 is permitted to access only those objects indicated by the codes of the file 5. The codes of the file 5 may in practice relate to objects such as a particular testing phase of a certain software development project. The format may be: Target Svstem Code Object TSC 0073 Programs and data for application software &num;05, testing phase &num;3.

TSC 0135 Programs and data for support of application software &num;7, installed on target system &num;105 and testing for support service.

A very important aspect is that the test controller 2 dynamically maintains the file 5 in real time to provide security access to the appropriate versions of the software.

There is a file 6 storing library lists for target system codes. The objects (programs and data) are stored on disk testing apparatus 1 in sets or libraries and the sequence for accessing libraries for any particular task is governed by a list. In the apparatus 1 there is a list for each target system code of the file 5 in a one-to-one relationship. For each code of the file 5 there is a library list in the file 6 governing the sequence in which objects are accessed for the particular operation. There is a disk controller 7 which allows access of the controller 2 to the objects which are stored on a fixed disk array.

In the drawing, the flow of signals to access objects stored on the disk array 8 is indicated by the set of arrows 20 (having interrupted lines). In operation, when a test engineer wishes to access objects for testing, he or she simply inputs the correct details required by the test controller, in response to the access control instructions retrieved from the file 4. Once this has been done, the test controller 2 automatically determines the correct target system code, the associated library list and subsequently retrieves the appropriate objects.

By doing this, the test controller 2 automatically creates the correct test environment for the test engineer. This is very important where there are many different possible testing operations with a potentially confusing large number of target systems to be supported. A practical advantage is that by use of the files 4, 5 and 6 and by real time and dynamic updating of the target system code file 5 and the list file 6, the test engineer will always be provided with the correct test environment and time wastage and errors due to working on out-of-date versions of software for example, are avoided.

For each suite of application software, the fixed disk array 8 has a sub-structure 8(a) as shown in the drawing.

For clarity, the sub-structure B(a) relates to only four target systems, however, in practice there would generally be much more than this. In addition, while each target system is shown as having only one library, it will be appreciated that many more are possible and indeed likely.

The sub-structure 8(a) has TO (target-originating), HO (host-originating), B (base) and D (data) levels.

The TO level contains a library for each target system running (or intended to run) the particular suite of application software associated with the sub-structure 8(a). Each library is associated with a target system and programs which are specific to the particular target system, and which originated on the target system.

The HO level also contains a library for each target system running (or intended to run) the suite of application software. Each library is associated with a target system, and has programs which are specific to the target system. However, in this case the programs are host-originating.

The B level has a single (large) library associated with the suite of application software, but is common to all target systems running (or intended to run) that software.

This library only has program objects.

Finally, the D level has a set of data file libraries, each associated with a particular target system. The data files in many cases are sample test data which may be received off-line or may be updated in instances on-line from the target system, depending on the nature of testing required. The association of these files to the associated programs is dynamically updated using a near relational database of access tables. The data files of the D libraries may also contain parameter values for the target system operation.

In operation, programs and data for a test session are retrieved by the disk controller 7 automatically accessing the correct libraries for the session by determining the correct sub-structure and the library addresses from the list retrieved from the file 6. Access to the libraries is in the sequence TO, HO, B and D, even if a library in TO or HO is empty. Retrieval of the programs and associated data to the RAM 9 is indicated by the arrows 30.

It will be appreciated immediately that the software testing apparatus 1 stores all of the programs and associated test data for all of the target systems for which application software is being developed and has been developed by the host system of the apparatus 1. Because the target-specific programs are separate, the apparatus 1 can store a single level library for each piece of application software. Further, sub-division of the target-specific into the HO and TO levels provides further structure and improves accessability. Because, on average, the base library B stores over 80% of all programs located on the various target systems each having its own application software, the levels TO and HO can be quite small and of course the base library need only be stored once.This allows the apparatus 1 to store all of the programs for all target systems without the need for a very large memory capacity and the resultant slow response times.

Effectively, this means that the testing environment for pre-release development testing, post-release support testing and also version and release update testing is created in the same manner. This provides uniformity and consistency for testing at all stages. Flexibility is also provided. For example, an experienced test engineer and a trainee will have different library lists for the same testing session. This may be pre-coded in the file 6 so that the test environment for the trainee includes dummy libraries mirroring the true test environment. The programs may thus be used by the trainee and any inadvertent changes or damage caused will have no detrimental effect on the true test environment. Status of such programs and data in the trainees list may be monitored to determine training progress.

Another major advantage of the storage of programs and data in this manner is that the lists in the file 6 may be pre-set for each target system code and may control the manner in which the programs and data are accessed in a pre-defined way with little or no potential for error.

For example, when testing of project 1, phase 1, there will be a particular target system code in the file 5 and for this there will be a library list in the file 6. This library list will govern the manner in which the disk controller 7 allows access by the test controller 2 to the stored programs and data. Thus, there is an overall set structure to ensure that any target-originating programs are accessed firstly and that host-originating programs are accessed secondly, both before the equivalent program in the base library. This ensures that there is always access to the most up-to-date programs for each target system.

The next most important signal flow is the retrieval by the test controller 2 of test instructions from a file 10.

These are transmitted (as indicated by the arrow 40) directly to the RAM 9 so that the data and programs which are written may be tested. All of the testing is carried out in the RAM 9. Testing output signals are transmitted to an output screen 12 as indicated by the arrow 50.

As new data relating to a particular test environment or new test data relating to a project is received by the test file 10 the database of testing conditions is overwritten or appended to, as appropriate.

On receipt of a request for a test or data signal from the test controller 2 the test or data files associated with the unit under test are retrieved in accordance with the associated links held by the database stored on the disk and stored in RAM 9. These data values or test parameters are fed to the unit under test serially or in parallel fashion across the system bus. The way in which they are received is determined by the unit under test.

The test file 10 is structured to respond to the unit under test with test instructions configured at program level, module level and at system level. The test instructions generate test valid signals or test fail signals, either of which may be passed to a test signal decoder (not shown) and may indicate to the test engineer how interactive testing should proceed. Alternatively, they may provide for automatic regression testing by accessing further test parameters also stored on the disk.

This depends on the appropriate target system code.

It will be appreciated that the apparatus 1 provides for increased productivity of test engineers as the correct objects are very quickly and automatically determined for each particular project. It will also be appreciated that because of the structure of the storage device 8, the most up-to-date versions and releases of programs and data are always correctly accessed. Further, access to the data and programs is very quick even where programs and data for a large number of target systems must be stored. This avoids in many circumstances the need for on-line access to the data files of a particular target system and allows comprehensive and rapid access to all of the most up-todate programs and data for each target system.The structure provides for storage of only a single base library for each piece of application software and thus it is possible to store code for all target systems.

A still further aspect of the invention is the manner in which testing operations may be extended to provide improved lines of code for a target system. Where such a new program is developed it is a very simple operation to carry out a search through the appropriate library list or lists for each target system to determine if there is a target-specific version of that program so that a comparison may be made and a decision as to whether or not changes should be made. In other words, the features of the testing apparatus 1 provide for efficient upgrading of programs on a large number and variety of target systems.

The invention is not limited to the embodiments hereinbefore described, but may be varied in construction and detail.

Claims

1. A software testing apparatus for testing of target system software developed on a host system, the apparatus comprising: a test controller connected to a read/write memory; a fixed disk array connected to a disk controller storing:- a file of security access control instructions; a set of software test instructions; and programs and data for software to be tested, an input interface connected to the test controller for reception of test commands; and an output interface, wherein the disk array stores a sub-structure for each suite of application software, the sub-structure comprising: a target-originating level containing a plurality of program sets, each set being associated with a target system and containing programs developed or modified on said target system; a host-originating level containing a plurality of program sets, each set being associated with a target system and containing programs for said target system and developed or modified on a host system for software development; a base level containing a single core set of programs common to all target systems; and a data level containing a plurality of test data files, each file being associated with a target system, wherein the test controller comprises means for: automatically determining the correct sub structure and, within said sub-structure, the correct program sets in the target-originating and host-originating levels and the correct data file in the data level for each test session initiating signal inputted at the input interface; and automatically directing retrieval of said program sets, the base programs and said data files by accessing the levels in target-originating, host originating, base and data order, and writing the retrieved programs and data to the read/write memory, retrieving test instructions and directing testing of retrieved programs and data in the read/write memory.

2. An apparatus as claimed in claim 1, wherein the fixed disk array stores a file of target system codes, each code being associated with a testing session and the test controller comprises means for retrieving a target system code according to the received testing session initiation signals and for automatically providing access to and retrieving the relevant programs and data and the test instructions by decoding the target system code.

3. An apparatus as claimed in claim 2, wherein the disk array stores a file of lists, each list being associated with a particular target system code and including the addresses for the relevant sub-structure programs and associated data files.

4. An apparatus as claimed in claim 3, wherein the file of target system codes and the file of lists are dynamically updated by the test controller in real time.

5. An apparatus substantially as hereinbefore described with reference to and as illustrated in the accompanying drawings.

5. An apparatus as claimed in any preceding claim, wherein the test data files include sample test data for the associated target system, and the test controller comprises means for automatically updating a data file with live data from the remote target system whenever required for testing.

6. An apparatus substantially as hereinbefore described with reference to and as illustrated in the accompanying drawings.

Amendments to the claims have been filed as follows 1. A software testing apparatus for testing of target system software developed on a host system, the apparatus comprising: a test controller connected to a readlwrite memory, an input interface connected to the test controller for reception of test commands, and a fixed disk array connected to a disk controller and storing: program and data objects of target system software to be tested; a file of security access control instructions; a set of target software test instructions; target system codes, each code associated with a test environment, and lists of objects, each list being associated in a one-to-one relationship with one of said codes, each list having addresses of the objects which may be retrieved for the associated test environment; and an output interface, wherein the disk array stores a sub-structure for each suite of application software program and data objects, the sub-structure comprising: a target-originating level containing a plurality of program sets, each set being associated with a target system and containing programs developed or modified on said target system; a host-originating level containing a plurality of program sets, each set being associated with a target system and containing programs for said target system and developed or modified on the host system; a base level containing a single core set of programs common to all target systems; and a data level containing a plurality of test data files, each file being associated with a target system, wherein the test controller comprises means for: automatically determining the disk array sub-structure associated with the application software suite identified in received test commands; automatically determining a code associated with the test environment requested in the test commands and using said code to automatically retrieve the associated list, the code being determined by:: directing a two-stage access control process in which firstly the user is identified by use of an authorised user identification code; and secondly, in the file of security access control instructions, determining testing access rights by references to dedicated user definition data in said file, the code being directly associated with the determined access rights; automatically retrieving the associated program and data objects addressed in said list by accessing the disk array sub-structure levels in the sequence of target-originating, host-originating, base, and data levels; writing the retrieved objects to the read/write memory to create the test environment; retrieving test instructions indicated in the received test commands; and directing testing of the retrieved objects in the read/write memory using said test instructions.

2. An apparatus as claimed in claim 1, wherein the test controller comprises means for dynamically updating the codes and the lists in real time.

3. An apparatus as claimed in any preceding claim, wherein the data objects include sample test data for the associated target system, and the test controller comprises means for automatically updating a data object with live data from the remote target system whenever required for testing.

4. An apparatus as claimed in any preceding claim, wherein the test controller comprises means for automatically determining the code according to received access data and access control instructions received from the file of security access control instructions.