RU2729210C1 - Electronic devices software testing system - Google Patents

Abstract

FIELD: computer equipment.

SUBSTANCE: invention relates to computer engineering. Technical result is achieved due to the fact that a single software environment comprises a synchronization timer associated with information analysis and processing means and with writing, editing and executing tests; functional device models comprise an event setting unit associated with writing, editing and executing tests, and an event generation unit associated with the synchronization timer.

EFFECT: technical result consists in improvement of quality of testing and reliability of tested software due to timely response and monitoring from side of complex on events from tested software and dynamic effect on computing process of tested software with simulation of emergency situations to confirm its failure stability.

1 cl, 1 dwg

Description

The invention relates to computer technology, namely to complexes for testing software (SW) of electronic devices (EI).

There is a known invention "Debugging Complex", which consists in the fact that the Program Debugging Complex contains a control computer, a control panel and a general-purpose computer, a group of input information simulators connected by inputs to a general-purpose computer, and outputs to a control computer, synchronizing output which is connected to the synchronizing input of the control panel, which is connected through the first bus bus to the control computer, and through the second to a general-purpose computer, the signal input of which is combined with the control input of the control computer and connected to the output of the control panel (RU # 2448363).

In this invention, the complex is based on hardware components that cannot always be easily and efficiently modified, and the presence of a physical exchange connection between simulators of functional modules limits the possibilities for their interaction, i.e. there is no possibility of expanding the functions of their interaction when new requirements for such a complex arise. Also, in this invention, the emphasis is on the interaction of hardware, without describing the interaction and building the software component of the debugging complex, which will ultimately complicate its implementation and, as a result, increase both labor costs and the timing of tests.

In the description of the operation of the complex it is said that the control of intermediate results of the program operation is provided by reading information from the memory of the control computer (CFM) at certain points after the message with the signal "Stop" - the end of the test, through the appropriate channels between the hardware components of the complex. When testing on this complex, control codes are entered into the registers of the address and data codes of the control panel, when they coincide with the address of the command or data during the execution of the UVM program, the corresponding comparison circuits are triggered, as a result of which the trigger is turned on, which generates the "Stop" signal. Having received the "Stop" signal, the computer reads the state of the internal memory of the UVM and outputs it to public display facilities according to a previously prepared program or according to the instructions of the operator. Then the operator sets other points and removes "Stop" by repeating the steps. Such provision of synchronization means only allows to work in the "start-stop" mode, i.e. set the state, and the program under test reported the end of testing. Thus, the described means allow you to register the end of the test, and there is no means of responding to events that occur during the test execution before the "Stop" signal. With such an organization, in the process of test execution, it is not possible to provide multiple responses from the UVM program, which is necessary to ensure greater variability of tests and, as a consequence, to increase the completeness of testing.

Known invention "Method and software testing automation system", which consists in the fact that the software testing automation system includes at least one user and / or tester device, at least one database, at least one database tests and a software and hardware complex capable of: software testing according to the prepared test plans through the interactive interaction of the tester with the user interface of the software under test, recording the specified user actions in automatic mode in predetermined terms of the business logic level of the software under test, saving the results in the test database as test scenarios, recording changes in the database made during the execution of the specified test scenarios, in automatic mode in predetermined terms, assigning these changes the attribute "Reference changes "And saving the results in the test database, after changing the software for its regression testing, playing test scripts from the test database in automatic mode, recording changes in the database made during the execution of the specified test scripts, in automatic mode in predetermined terms, assigning to the data changes the sign "Actual changes" and saving the results in the test database, carrying out verification for each test scenario, comparing the "Reference changes" and "Actual changes" and saving the comparison results in the test database, forming a list of test scenarios that did not pass verification, while under changes that have not passed verification imply discrepancies between the "Reference changes" and "Actual changes" and is additionally made with the ability to: identify the main business processes performed using the software under test, draw up a detailed test plan for the identified businesses c-processes, fixing the options for the interactive interaction of the tester with the user interface of the software under test (patent application of the Russian Federation No. 2013126869/08).

In this invention:

• there are no uniform and clear requirements for software models (simulators) of interaction with the electronic device of the software under test, which will necessarily lead to the complication of the implementation of the software package and the limitation of the possibilities for its expansion;

• there are no visualization tools (as the main tool for providing test results) for operational and flexible control of the state of modeling tools, as well as their data and entities;

• there are no synchronization tools, which can lead to limited functionality of the complex and, as a consequence, to a decrease in the completeness of testing and the impossibility of timely response to the reactions of the software under test during testing.

The closest to the proposed technical essence is the invention "Complex of automation and visualization of testing embedded software of electronic devices", which consists in the fact that the software complex includes a single software environment containing tools for writing, editing and executing tests, automated analysis and information processing, visualization data and entities during testing and at the stage of preparation for it, storing test sets and connecting them to the complex at any stage of testing, dynamic reconfiguration and control at any time by the software complex and any components included in its composition, allocating a common buffer for interaction connected models and their arbitrary reconfiguration, both in manual mode and during the execution of tests, protection and recovery of the complex in the presence of unforeseen failures during its operation, dynamically connected models of functional devices, re alizable in the form of independent unified software modules with a predetermined unified set of methods and functions and means of visualization and presentation of the state of their internal data and entities, developed according to unified rules of construction and implementation based on a unified template and called from a unified software environment, a database for storing data representations and entities, means of automated filling of the database with information based on the analysis of the documentation for which the software under test is being developed (RU # 2678717).

This invention talks about the implementation of the functionality of registering events in the model, through an event waiting timer in the model: “waiting for an event - organizing an event waiting timer in the selected model, indicating the number of expected events. Allows you to synchronize the progress of the software under test with the course of the test, which ensures the most accurate interaction and impact on the software under test ”, i.e. in each model of functional devices a waiting timer is organized and then a message about its activation is transmitted. When organizing a timer in a model, a situation is possible when the model is waiting for an event, and the environment can continue to perform functions, which can cause desynchronization of events in the model from the software under test and control actions from a single software environment. The presence of such feedback allows you to provide additional control during testing, but will not allow using it as a timely main control and management in real time, which ultimately will reduce the completeness and quality of testing. Also, the organization of many timers for waiting for events in each model creates an additional load on the resources of the control equipment used by them.

Thus, the present invention is not without its drawbacks and can be improved by implementing setting and generating events in models of functional devices and a single common timer for their waiting in a single software environment. In terms of the totality of features, this invention is the closest to the claimed one and is chosen as a prototype.

The disadvantage of the prototype is:

• The impossibility of ensuring complete synchronization of the testing process with the response of the software under test in real time, which boils down to testing, which does not take into account all the features of the execution of the program under test and reduces the quality and completeness of testing.

The complex operates in the mode of semi-synchronous execution of the testing process and the computing process of the software under test, which does not allow their fully predictable interaction and ensure the accuracy of the impact on the computing process of the software under test.

The technical problem of the invention is to improve the completeness and quality of testing.

The technical problem is solved due to the fact that the software testing complex for electronic devices includes input / output equipment; test storage server; a unified software environment containing tools for writing, editing and executing tests, tools for analyzing and processing information, tools for visualizing data and entities, tools for managing the storage of test sets and their connection to the complex at any stage of testing, tools for dynamic reconfiguration and control, a common buffer in RAM for interaction of plug-in models and their arbitrary reconfiguration, means of protection and restoration of the complex; functional device models containing a configuration block, a block for interaction with an electronic device, a simulation block, a state visualization block; database; tools for automated analysis and database filling; a single software environment contains a synchronization timer associated with tools for analyzing and processing information and with tools for writing, editing and executing tests; functional device models contain an event setting block associated with means of writing, editing and executing tests, and an event generating block associated with a synchronization timer.

The drawing shows a block diagram of the complex.

The complex is implemented as follows:

The complex consists of input / output equipment 6, a test storage server 5, a unified software environment 1, models of functional devices 2, a database 3, tools for automated analysis and filling of a database 4.

The input / output equipment 6 exchanges data with the models of functional devices 2 through an interaction unit with an electronic device 2.3 and exchanges data with an electronic device with embedded software 7.

The test storage server 5 provides storage and access to tests from a single software environment 1 through the tools for managing the storage of test cases and their connection 1.1.

Models of functional devices 2 are implemented in the form of independent unified software modules with a predetermined single set of methods and functions and means of visualization and presentation of the state of their internal data and entities, developed according to uniform rules for construction and implementation, and controlled by the software environment for executing tests in real time on any testing stage. Models of functional devices 2 contain a configuration block 2.1, a simulation block 2.2, a block for interaction with an electronic device 2.3, a state visualization block 2.4, an event setting block 2.5 and an event generation block 2.6.

A unified software environment 1 is implemented in the form of a unified user interface, which contains controls for storing test sets and their connection 1.1 to the complex at any stage of testing, provide interaction with the test storage server 5; tools for writing, editing and executing tests 1.2, provide the setting of configuration parameters in the configuration block 2.1, exchange of information with the block of interaction with an electronic device 2.3, setting events in the block for setting events 2.5 and starting the synchronization timer 1.7; information analysis and processing tools 1.3 provide data exchange with the modeling unit 2.2 and receive information from the synchronization timer 1.7; means of dynamic reconfiguration and control 1.4 at any time of any components included in the unified software environment 1, and transfer data to the configuration unit 2.1; visualization tools for data and entities 1.5, both their own and models of functional devices at any stage of testing, and receive data from the state visualization unit 2.4; means of protection and recovery 1.6 of a unified software environment in the presence of unforeseen failures in the process of operation and a timer for synchronization 1.7 with the computing process of the tested software. A single software environment 1 provides data transfer to a buffer in RAM 8 for the configuration of models of functional devices 2 in the configuration block 2.1 and their inter-model interaction, with the possibility of expanding such interaction as new requirements for models come in or adding new models to the complex.

Database (DB) 3 contains information that includes the parameters of the software package and the information necessary for a single software environment 1. The database is implemented as a set of related tables in accordance with a specific type of purpose and use, with reference to certain types of tested software.

Means of automated analysis of information and filling out the database 4 analyze the documentation describing the device for interacting with the tested software and filling the database in the order of the data belonging to its type of purpose and use, and internal binding to each other.

Synchronization of the test process is implemented as follows: each functional device model includes an event setting block 2.5, which contains means of interaction with the user, in which the user sets the number of events, their belonging to the model and the total waiting time, and an event generation block 2.6, in which events are generated based on external influences from the software under test or according to the internal algorithms of the model, and a synchronization timer 1.7 is included in a single software environment 1, which controls the computational process by stopping the testing process while waiting for events and continuing it, when registering events from models, or by interrupting the course testing in the absence of events at the specified time.

The complex works as follows.

The database control program, which is part of the automated analysis and filling of the database 4, contains basic information on the tested software of the electronic device (device accessory and software identifier, which is unique for any software of an electronic device).

Based on the software identifier, a typed file is created in the Microsoft Excell format (* .xls) with tables for filling in the data, into which information from the documentation for the software of the electronic device (ID for software development) is entered in the corresponding tables of the typed file.

The database management program automatically fills in database 4 with information, binding it to the software identifier, as well as assigning identifiers for each record in database 4 with data binding to each other, in accordance with the construction of the database 4 tables.

The user launches the unified software environment (UPC) 1. When launched, the UPC 1 determines the availability of the test storage server 5 and connects to it in the read / write access mode (by default, the read-only access mode, for unauthorized changes to the data on it). Further, it configures internal structures and data, allocates a common buffer in RAM 8 for inter-model interaction and, based on the software identifier, connects the corresponding models of functional devices 2, with the transfer of each model of their configuration parameters (data of the previous configuration, the currently selected software identifier, buffer descriptor memory for inter-model interaction, and other parameters necessary for setting up the model and launching the thread (s) that simulates the work of the model).

In ENP 1, two modes are implemented: "Editor" and "Working", both modes are selected by the corresponding tabs.

In the "Editor" mode the user creates test sequences (cyclograms), edits them and saves them in the working directory. The working directory generates ENP 1 in accordance with the selected current configuration, i.e. creating appropriate catalogs by belonging to the type of electronic device and the index of the tested software.

In the "Working" mode, a tool for executing test cases is presented, in several modes: single run, endless looping, and step-by-step execution, as well as tools that perform the functions of breakpoints for executing a test case to a specified position.

ENP 1 implements the functions of managing the storage of test cases using the test storage server 5:

• copy / move the current test suite to the test storage server 5, while on the test storage server 5 a chain of directories will be created corresponding to the working one and another directory with the version number for which this suite was developed and worked out will be added;

• copy / move a test suite from the test storage server 5 while selecting from the list of available suites according to the software version;

• open the directory containing the test case in the explorer;

• view the complete list of tests both in the working directory and on the test storage server 5.

In the event of an unforeseen failure or fall of ENP 1, the next time it is launched, it restores the unsaved test sets and execution protocols, and the configuration parameters of ENP 1.

ENP 1 contains elements of access and control of models of functional devices 2, implemented in the form of a context menu that appears when you click on the corresponding icon of the model. This menu is available at any stage of work with ENP 1, including the test execution mode, to influence the testing progress in real time.

Through this context menu, the user can access the functional device model 2 to perform the following interaction functions:

• setting up internal parameters (stream frequency, device and data addresses required both for setting up the hardware part of interaction with the electronic device 7, and internal structures, data and model entities);

• model flow control. The model stream performs the functions of modeling the logic of a functional device;

• implementation of the function of interaction with the model (recording control actions both for issuing to the electronic device 7 and for managing the internal state of the data and model entities, reading the state);

• visualization of data and entities of the model (data is presented both in the form of codes directly, and in the form of various types of representation: pictograms, icons, graphics, images, etc.).

Also, each model of functional device 2 logs in its local buffer information for the entire period of operation of ENP 1 (receiving commands, signals, impacts both from the electronic device and from the ENP 1), which allows at any stage to control what happened with each model of functional device 2, for example, before the failure of the test was recorded, which makes it easier to analyze the state of the software under test during the test execution.

The testing visualization tools available to the user in the testing process are presented in the form of a set of graphical support, which includes basic tools: highlighting the types of operations, highlighting the progress of testing, highlighting non-passing operations, highlighting certain information that has not passed control, etc., and complex means of presenting information (for example, displaying the position of the Sun in space when simulating devices in the solar orientation mode of the spacecraft, relative to its vision through the device "sensor of the presence of the Sun", with fixing and displaying each point of such a path on the plane and displaying all orientation parameters in real time ).

Synchronization of the test process is carried out using a synchronization timer 1.7 and an event generating unit 2.6 in each of the functional device models 2. Events can be generated both according to the internal logic of the model and according to external influences received by the model from the tested software, and according to more complex algorithms information exchange, including inter-model communications and additional single signals.

To synchronize the test process, the user inserts a synchronization operation at a certain place in the test sequence diagram, for which he indicates the number of events, their type, model affiliation, the time required to wait, etc. When the test sequence is executed on the synchronization operation, EPU 1 starts the synchronization timer 1.7 into standby , in accordance with the specified parameters, i.e. he expects specific events occurring in a certain functional device model, which will occur according to certain actions from the software under test. Synchronization timer 1.7 provides stopping the testing process for the time during which the event should occur. If an event arrives, then the testing process continues, otherwise, when the timer is triggered, the testing process is interrupted due to the absence of an event.

The complex is implemented in JSC "ISS", based on the following software:

• a unified software environment ("Ground development complex for the software of the computational module of the main-module equipment", certificate of registration of computer programs No. 2013618885);

• models of functional devices (certificates of registration of computer programs: №2013618778, №2013660234, №2013660490, №2014610477, №2014611654, №2015616152, №2015616230, 2015618458, 2015618459, 2015660760, 20156607681, 2015660760, 2015660668160 and etc.);

• databases (certificate of state registration of databases No. 2011620642);

• service for automated filling of the database of onboard trunk-module equipment (certificate of state registration of databases No. 2015616153).

With the use of this complex, the embedded software of the control units of the onboard control complex and the interface units of the onboard control complex of advanced spacecraft manufactured by JSC "ISS" were tested.

Due to the implementation of this technical solution, the required technical result is achieved:

• Increasing the completeness of testing by implementing the functionality of the complex with means of synchronization of the test process, which include a synchronization timer and functions for generating events in models of functional devices of the environment of an electronic device, implemented as a result of the impact of the software under test.

• Improving the quality of testing and reliability of the software under test due to the timely response (response) and control from the complex to events from the software under test and dynamic impact on the computational process of the software under test with imitation of emergency situations to confirm its failure-stability.

Claims (1)

A complex for testing software for electronic devices, consisting in the fact that the complex includes input / output equipment; test storage server; a unified software environment containing tools for writing, editing and executing tests, tools for analyzing and processing information, tools for visualizing data and entities, tools for managing the storage of test sets and their connection to the complex at any stage of testing, tools for dynamic reconfiguration and control, a common buffer in RAM for interaction of plug-in models and their arbitrary reconfiguration, means of protection and restoration of the complex; functional device models containing a configuration block, a block for interaction with an electronic device, a simulation block, a state visualization block; database; tools for automated analysis and database filling; characterized in that the unified software environment contains a synchronization timer associated with tools for analyzing and processing information and with tools for writing, editing and executing tests; functional device models contain an event setting block associated with means of writing, editing and executing tests, and an event generating block associated with a synchronization timer.

Images