RU2817185C1 - Method of confirming tests of embedded software of electronic devices - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: invention relates to computer engineering. When writing each functional test, the number of controlled points is determined and the test is checked using the tested software of the electronic device, i.e. triggering each control point in the test by simulating an error in data from the software of the electronic device. Error is simulated by reading data from the tested software of the electronic device and substituting data therein. When performing data substitution, during the test, the control equipment performs rewriting of data from the tested software of the electronic device in its own random access memory in accordance with the specified error, thus, detection of error in test control point is worked out. If an error is detected by a control point, the next control point of this test is checked, by imitating a new error in the software data of the electronic device, and upon completion of the test, the next functional test is written.

EFFECT: high reliability of the electronic device due to high reliability and quality of testing its software.

1 cl, 2 dwg

Description

The invention relates to computer technology, namely to a method for confirming tests of embedded software of electronic devices.

The invention is known: “A method for processing the amount of data used during the debugging phase of the functional software of a system installed on board an aircraft, and a device for its implementation” (Patent RU 2451990 C2), according to which the method includes the following stages: a) path breakdown executing the specified work program at functional intervals by setting waypoints in each program function; b) establishing control points associated with each waypoint; c) normal program execution, which involves: recording the program execution state at the location of each waypoint; wherein writing the execution state to memory causes the execution state previously recorded for the specified waypoint to be erased; when an error is detected, the following is carried out: search for a waypoint corresponding to the disrupted function; search for the initial state of program execution; restoring this original execution state; correcting the error in the broken function and re-executing the program.

The path from label to label is used as a test interval, which determines the functional interval of the software. Such markup can only be done by a programmer who knows the white-box implementation of the software. Knowledge of the implementation can reduce the validity of testing by preventing objective testing of a functional requirement, regardless of how it is implemented.

To increase the reliability of testing, it is better to involve a specialist who knows the requirements for the software and does not have a complete understanding of its implementation. As such a specialist, you can choose a software requirements developer who perceives software at the level of requirements for it, without taking into account the implementation. In this case, the requirements will be used as functional intervals, for testing of which it is necessary to write separate tests with error detection checkpoints installed in them. Thus, this testing will be carried out using a gray box method.

The quality of the written tests is essential and must be verified and validated before testing with normal software execution. Otherwise, situations are possible when the test does not detect an error, and it subsequently leads to abnormal situations in the operation of the electronic device and the system in which it is installed. Identifying and eliminating such an error will increase the test time by repeating the completed stages. Thus, in order to be able to talk about the reliability of testing, it is necessary not only to arrange control points, but also to check their performance, i.e. run tests.

Testing for error detection ensures unitary testing of the software of an electronic device, and therefore eliminates such possible errors even before testing with normal software execution, and, therefore, reduces test time.

The invention “Method for functional testing of software of electronic devices” (Patent RU 2780458 C1) is known, in which, in order to use completely reliable tests during their development, confirmation of error detection is carried out using the software of an electronic device. It is carried out by simulating an error through the technological exchange channel of an electronic device and confirming the detection of the introduced error by test control points. Introducing an error into the software can disrupt its execution and affect the results of subsequent tests, therefore, to eliminate such situations, additional algorithms for parrying introduced errors are required after checking the test for which they were introduced. This imposes additional labor intensity.

The disadvantages of this invention include:

- possible disruption of the computing process of an electronic device or its unpredictable behavior due to the introduction of an error into the software during testing.

- an error introduced to check one control point must be excluded to control other control points, which imposes additional costs that may be associated with eliminating the introduced error by restarting the computing process or performing an additional algorithm for parrying this error;

- if the introduced error for one control point is not completely eliminated, it may influence it when checking other control points.

The disadvantages of this invention can be eliminated by eliminating the introduction of an error into the software of the electronic device, and simulating it on the control equipment, by replacing the data read from the software of the electronic device, which will not disrupt the computing process of the software of the electronic device and ensure full testing.

The invention “Method for functional testing of software of electronic devices” (patent RU 2780458 C1) was selected as a prototype.

For the claimed invention, the following essential features in common with the prototype have been identified: a method for confirming tests of embedded software of electronic devices, which consists in the fact that one test checks one functional block and for each test at least one check point is established; introduce an error and run the test; checking the test checkpoint by executing it using the software under test; when an error is detected by a checkpoint, they proceed to checking the next checkpoint of this test by entering a new error; if the checkpoint does not detect the introduced error, the test adjusts the error detection logic at the checkpoint and repeats the test until the checkpoint detects the error; after checking all the control points of the next test, the control points of the next test are checked, and so on for all tests.

The technical problem is to increase the reliability of an electronic device by increasing the reliability and quality of testing of its software.

The technical problem is solved by validating the tests by simulating an error in the data from the electronic device software to verify each test point. An error is simulated by reading data from the electronic device software under test and replacing the data in it. When performing data substitution, during the test, the monitoring equipment rewrites the data from the electronic device software under test in its own RAM in accordance with the specified error, thereby identifying the error at the test checkpoint.

Substitution of data from the software of an electronic device to confirm tests allows you to increase the reliability of testing due to the fact that testing occurs without disrupting the computing process of the software of the electronic device (there is no real introduction of errors) and improves the quality of testing due to guaranteed testing of test checkpoints with the exception of influence working out each on the others, which will ultimately increase the overall reliability of the electronic device.

To implement the proposed method, a testing complex is used, containing control equipment 1 and electronic device 2, shown in Fig. 1.

Control equipment 1 contains an environment for developing and executing tests 1.1, means of direct memory access 1.2, simulators of external devices 1.3, hardware exchange modules 1.4.

The test development and execution environment 1.1 is implemented as a single user interface and allows you to develop tests. The tests contain a sequence of operations to control a computing device and set control points. Control points can be operations of monitoring certain data (the appearance of a telemetric sign in data words, the formation of a trigger flag according to a condition or a certain value of a mathematical transformation, etc.), registration of the triggering of an event in the software of an electronic device (issuing a certain control command to the simulator , changing the state of a telemetry information parameter, etc.), calculating the execution time of functional blocks or the time between events, etc.

Direct memory access tools 1.2 are tools that implement an exchange protocol over the diagnostic (technological) exchange channel 3 and provide direct memory access to an electronic device 2. Direct access to the memory of an electronic device 2 allows you to modify the data in it and perform its reprogramming by software electronic device.

Models of functional devices include exchange modules 1.4, which simulate the logic of operation of standard exchange channels 4, and simulators of external devices 1.3, which simulate the operation of external devices of an electronic device.

Electronic device 2 is connected to exchange modules 1.4 of control equipment 1 via standard exchange channels 4 and to means of direct access to memory 1.2 of control equipment 1 via diagnostic (technological) exchange channel 3. Software is programmed into electronic device 2 via diagnostic (technological) exchange channel 3 software for which testing will be carried out.

The electronic device software is developed based on the requirements document. It describes all the customer’s requirements for the software being developed for the electronic device. Such a document can be a specification, technical assignment, etc.

The method is implemented as follows:

The implementation of the method is shown in Fig. 2.

The tester, based on the requirements document for the software of the electronic device, breaks down the software of the electronic device into functional blocks. Such functional blocks can be either individual requirements for the software of an electronic device or a combination of them, interconnected by a common part, if they are presented to perform the same function.

An example of a function as a single requirement: providing a certain time between control commands to external devices, enabling a telemetry parameter for a specific command in the software of an electronic device, etc.

An example of a function as a multiplicity of requirements is issuing a control command to an external device (includes: issuing a command from control equipment, generating a receipt for control equipment after a certain time, issuing a control command to an external device simulator and monitoring telemetric information by control equipment about the status of issuing a command during external device), etc.

After dividing the electronic device software into functional blocks, the tester, in the test development and execution environment, creates functional tests with the installation of at least one checkpoint for each test. A checkpoint can be a single operation or a set of operations designed to monitor the state of data or arrays that characterize the operation of the tested function of the software under test.

Confirmation of tests consists of working out the guaranteed operation of checkpoints of all tests.

To confirm the tests, the tester on the control equipment in the test before the control point requests data from the electronic device software, sets substitution data in the read data from the electronic device software to check the operation of the control point. The tester performs a checkpoint test. When performing data substitution, during the test, the monitoring equipment rewrites data from the software of the electronic device under test in its own RAM in accordance with the specified error. If an error is not detected by the control point, it is corrected (updated data or control parameters are entered for it) and its operation is checked again, and so on until it detects the entered error. If an error is detected by a test point, the next test point in the test is checked in the same way. After checking all the control points in the test, the tester moves on to the next test and checks its control points in the same way, and so on until the control points for all tests are checked.

Thus, during test validation, the checkpointing conditions are inverse, that is, the checkpoint is successful when invalid data is detected and test execution continues. Once the tests are confirmed, testing is performed with the electronic device software running normally.

When testing with normal execution of electronic device software, data substitution operations in tests are not performed and the conditions for checking control points are normal, i.e. tests are performed normally. In this mode, passing the test point indicates that the data of the software function being tested is normal. Stopping the progress of tests at a checkpoint indicates the presence of an error in the controlled data received from the electronic device software under test.

Performing test confirmation allows you to guarantee the detection of errors in the software of an electronic device when testing it with normal execution, due to the practiced operation of control points in the tests.

The method was tested on a multifunctional workstation of a ground-based debugging complex for control instrument software, which is made in the PXI standard, consisting of:

- a computing module with software acting as an electronic device;

- monitoring equipment based on the NI PXI-1045 crate with a PXI-8110 bus controller:

- test development and execution environment that manages the hardware components of the control equipment;

- models of functional devices developed on the basis of exchange modules (from National Instruments), such as PXI-C1553M-EF-4 (MKO module), PXI-8431/4 (RS-485/422 module), PXI-7813R (digital input module - output from FPGA) etc., modeling the logic of operation of standard exchange channels and modeling the logic of operation of functional devices;

- means of access via the diagnostic (technological) exchange channel.

Using this method, testing of the built-in software of the computing device of control units and interface blocks of the on-board control complex of modern and promising spacecraft produced by ISS JSC was carried out.

The implementation of this technical solution allows you to obtain the necessary technical result:

- increasing the reliability of testing due to the fact that testing occurs without disrupting the computing process of the electronic device software;

- improving the quality of testing due to guaranteed testing of test checkpoints, and eliminating the influence of testing each on the others.

From the patent information materials known to the applicant, no features similar to the set of features of the proposed method were found.

Claims (1)

A method for confirming tests of embedded software of electronic devices, which consists in the fact that one test checks one functional block and for each test at least one test point is set; introduce an error and run the test; checking the test checkpoint by executing it using the software under test; when an error is detected by a checkpoint, they proceed to checking the next checkpoint of this test by entering a new error; if the checkpoint does not detect the introduced error, the test adjusts the error detection logic at the checkpoint and repeats the test until the checkpoint detects the error; after checking all the checkpoints of the next test, the checkpoints of the next test are checked, and so on for all tests, characterized in that before an error is introduced into the test, data is read from the software of the electronic device under test; at the error entry stage, data is replaced in the read data from the electronic device software under test; During the test, when performing data substitution, the monitoring equipment rewrites data from the electronic device software under test in its own RAM in accordance with the specified error.