JP2020204847A - Test scenario automatic generation method of control software and device and automatic test method, and device and computer program - Google Patents

Abstract

To reduce costs by reducing the number of inspection man-hours as well as preventing inspection errors by automating creation of test scenarios and testing (debugging) of control software.SOLUTION: There are provided a test scenario automatic generation method of a control software that automatically generates a test scenario 170 by reading a design document (140, 150, 160), modifies the design document (140, 150, 160) using the error information output during scenario generation, and automatically generates the test scenario 170 again using the modified design document (140, 150, 160), and/or an automatic test method of a control software that debugs the control software by reading the test scenario 170, generates the test result report 180 by the debugging, and debugs the test scenario 170 again by making only the NG items in the test result report 180 as the test scenario 170.SELECTED DRAWING: Figure 10

Description

The present invention relates to a control software test scenario automatic generation method, an apparatus and an automatic test method, an apparatus and a computer program, and is particularly suitable for use in a screen input / output inspection in a central monitoring and control apparatus of a plant such as a waste incinerator. Regarding the control software test scenario automatic generation method, device and automatic test method, device and computer program.

In a plant such as a waste incinerator, it is necessary to inspect the control software such as the central monitoring and control device in the actual machine test environment at the time of shipment from the factory before delivery to the site.

At the time of shipment from the factory, since there is no local equipment (sensor, control panel, operation end) or wiring with those equipment, the input / output of a controller, for example, a programmable logic controller (PLC) is simulated and inspected.

Conventionally, in the screen input / output (I / O) inspection, it is difficult to automate the identification of the position of the object to be checked on the screen, and the identification / quality judgment is performed by human judgment. The screen I / O inspection is a product shipment inspection, and because of quality assurance, sampling (partial) inspection is not allowed and 100% confirmation is required. Therefore, conventionally, as illustrated in FIG. 1, the input worker 2 creates and inputs the test I / O list 4, and the confirmation worker 6 checks the numerical value and the color change on the confirmation target screen 8. Multiple workers carry out manual inspections one by one, which not only takes time and manpower, but also has the risk of human error in checking. Furthermore, it was difficult to find the cause when there was a check error.

As prior art related to the present application, Patent Documents 1 and 2 describe a technique for automatically generating and testing a pseudo signal or a simulated program for a communication interface between an HMI (Human Machine Interface) screen and a PLC. Has been done.

Further, Patent Document 3 describes a technique for creating a ladder program for PLC from a circuit drawing.

Further, Patent Document 4 describes a technique for enabling a single person to carry out a PLC function confirmation test in a plant monitoring and control device by voice output.

Further, Patent Document 5 describes a technique for automatically generating, displaying, and tracking a checklist according to assigned work items.

Further, Patent Document 6 describes a technique of creating a test code according to a test specification, testing using the test code, and totaling the test results to create a test result report.

Further, Patent Document 7 describes a technique of selecting and executing a registered test item, analyzing the test result after the test is completed, and creating a report.

Further, Patent Document 8 describes a simulation support tool that automatically generates a test input.

Further, Patent Document 9 describes a software automatic test system that determines the quality of a test screen generated by an inspection target device that stores a screen generation program by image collation.

JP-A-2017-129957 JP-A-2017-130139 Japanese Patent No. 5498446 Japanese Patent No. 3713150 Japanese Unexamined Patent Publication No. 2014-225232 Japanese Patent No. 4777740 Japanese Unexamined Patent Publication No. 2006-215907 Japanese Patent No. 4379687 Japanese Patent No. 5540413

However, conventionally, only fragmentary techniques have been described, and comprehensive techniques have not been described.

The present invention has been made to solve the above-mentioned conventional problems, and automates test scenario creation and test (debugging) of control software to reduce inspection man-hours, reduce costs, and prevent inspection errors. The challenge is to provide the technology that enables.

The present invention reads a design document, automatically generates a test scenario, checks error information output at the time of scenario generation (for example, the generated test scenario is used to check inconsistencies between the design documents, and outputs the result of the check. The problem is solved by the test scenario automatic generation method of the control software, which is characterized in that the design document is modified by using the error information) and the test scenario is automatically generated again by the modified design document. ..

Here, the design document is a design document for screen design, an input / output list describing input / output signals, a screen image showing coordinates corresponding to the input / output signals of the input / output list, and the input / output. The test scenario can be automatically generated based on the list and the software function specification that interpolates the screen image.

Further, in the test using the test scenario, an analog or digital input signal is simulated and input to check whether the display is displayed at an appropriate display location or whether the alarm message is displayed correctly. be able to.

In addition, when checking the screen using the test scenario, the correct image pattern of the input should be created, the place to be checked on the screen should be searched from the design document, the area should be narrowed down, and the correct image pattern should be checked within the narrowed down area. It is possible to identify the location by comparing the images and read the change in the color, numerical value, and image shape of the identified location to be checked to judge the quality.

Further, when confirming the signal output using the test scenario, the screen operation switch window can be opened and the screen operation can be performed to determine whether or not the operation signal is correctly output from the control device.

The present invention is also characterized in that the test scenario is read, the control software is debugged, the test result report is generated by the debugging, and the control software is re-debugged with only the NG item of the test result report as the test scenario. By providing an automatic test method for the control software to be used, the above-mentioned problems are similarly solved.

Here, the test scenario can be assumed to have been created by the test scenario automatic generation method.

The present invention also provides means for reading a design document and automatically generating a test scenario, means for checking inconsistencies between the design documents using the generated test scenario, and error information output as a result of the check. By providing a test scenario automatic generation device for control software, which comprises means for modifying the design document by using the design document and means for automatically generating a test scenario again by the modified design document. Similarly, the above problem is solved.

The present invention also re-debugs the control software by using only the means for reading the test scenario and debugging the control software, the means for generating the test result report by the debugging, and the NG item of the test result report as the test scenario. Similarly, the above problem is solved by providing an automatic test device for control software, which comprises means and means.

The present invention also provides a method for automatically generating a test scenario or a computer program for causing a computer to execute the automatic test method.

The present invention also provides the test scenario automatic generation device or a computer program for realizing the automatic test device in a computer.

According to the present invention, it is possible to automate test scenario creation and test (debugging) of control software to reduce inspection man-hours, reduce costs, and prevent inspection errors.

Conceptual diagram showing the state of conventional inspection The figure which shows the system configuration example of the debug test target plant which adopted the embodiment of this invention. The figure which shows the flow of the debug test in the said embodiment A diagram showing an example of one input / output list of a detailed design document also used in the debug test. Similarly, a diagram showing an example of a screen input / output list. Similarly, a diagram showing an example of software function specifications. A diagram also showing an example of a test scenario Similarly, a diagram showing an example of a test result report Similarly, a diagram showing an example of a test target screen Similarly, a diagram showing the overall flow A flow chart showing the procedure of automatic debugging by the test scenario that also checks the analog input / display. Similarly, a flow chart showing the procedure of automatic debugging by a test scenario that checks analog inputs / alarms. Flow chart showing the continuation of FIG. Flow diagram showing the procedure of automatic debugging by the test scenario for checking the digital input / display in the above embodiment. A flow chart showing the procedure of automatic debugging by the test scenario that also checks the digital input / alarm. Flow chart showing the continuation of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the contents described in the following embodiments and examples. In addition, the constituent requirements in the embodiments and examples described below include those that can be easily assumed by those skilled in the art, those that are substantially the same, that is, those in a so-called equal range. Further, the components disclosed in the embodiments and examples described below may be appropriately combined or appropriately selected and used.

FIG. 2 shows an example of the system configuration of the debug test target plant (for example, a waste incinerator) targeted by the present invention. This plant has a No. 1 incinerator and a No. 2 incinerator (not shown), which are control panels 11, 12, sensors 21, 22, operating ends 31, 32, wiring cables 41, 42, and programmable logic controller (not shown). PLC) 51 and 52 are provided. Further, the control panel 13, the sensor 23, the operation end 33, the wiring cable 43, and the PLC 53 are also provided for the common system of the No. 1 incinerator and the No. 2 incinerator. In addition, distributed control systems (DCS) 71, 72, 73 and monitor screens 91, 92, 93 are provided as a monitoring operation system for the entire system common to the No. 1 incinerator and the No. 2 incinerator.

Of these, the factory shipment inspection targets 100 are PLC51, 52, 53, DCS71, 72, 73, and monitor screens 91, 92, 93.

For the final local delivery system 200, at the time of shipment from the factory, the local equipment (sensors 21, 22, 23, control panels 11, 12, 13, operation ends 31, 32, 33) and the wiring cable 41 with those equipment , 42 and 43 do not exist, so it is necessary to simulate and inspect the input and output of PLCs 51, 52 and 53. As illustrated in FIG. 1, since the inspection was performed manually one by one in the past, a plurality of workers, an input worker 2 and a confirmation worker 6, are required, and the check is performed by a human. There was a possibility of a mistake.

Therefore, in the present embodiment, the automatic debugging tools 81, 82, and 83, which are software for generating simulated signals, are incorporated in the DCS 71, 72, and 73, respectively, and the devices are connected with a communication cable to create a factory inspection system configuration.

On the other hand, as shown in FIG. 3, the software test scenario creation tool 120 is installed on the designer computer 110, the detailed design documents 140, 150, and 160 described in detail later are automatically read, and the test scenario 170 is automatically read. Generate. Then, the detailed design document error report 122 showing the OK / NG determination result is generated, the designer confirms the NG portion, and corrects the detailed design documents 140, 150, 160. Then, the created test scenario 170 is copied to DCS (71 in the example of FIG. 3).

In this way, NG can be reduced by automatically reading the modified design document again and automatically regenerating the test scenario 170. As a result, the design documents are also checked, and the accuracy of the design documents is improved before the software is created, so that mistakes during software creation can be reduced.

On the other hand, the automatic debugging tool 81 is installed in the DCS 71, and the test scenario 170 generated by the designer computer 110 is automatically read to perform automatic debugging. Then, a test result report 180 including the OK / NG determination result is generated. Then, the designer confirms the NG portion and corrects the DCS software.

Further, the test result report 180 is automatically read instead of the test scenario 170, and re-debugging is repeated. At this time, the automatic debugging tool 81 can streamline the re-debugging test by skipping the OK scenario and re-debugging only the NG items.

FIG. 4 shows an example of one hard / communication input / output (I / O) list 140 of the detailed design documents 140, 150, and 160 for screen design used for generating the test scenario 170. The main items (there are other items to be described) as shown in FIG. 4 are defined and described, and test data is generated from these items. In the figure, AI is an analog input and DI is a digital input.

FIG. 5 also shows an example of the screen input / output (I / O) list 150, which is another example of the detailed design documents 140, 150, and 160 for screen design. This is a combination of a screen image showing the coordinates on the lower side of the figure and a list on the upper side of the figure. The main items (there are other items to be described) as shown in FIG. 5 are defined and described, and test data is generated from these items.

FIG. 6 also shows an example of the software function specification 160, which is the third example of the detailed design documents 140, 150, and 160 for screen design. In the figure, 162 represents an instrument face plate (FP) which is a control screen of an instrument (an example of the figure is a thermometer). This software function specification 160 interpolates the hardware / communication input / output list 140 and the screen input / output list 150, and reads face plate information (tags with face plates, alarm setting values, etc.) from the specification creation rules. , Generate test data.

As a method of automating screen confirmation, signal input is automatically performed based on a list. Specifically, the correct image pattern of the input is created, and the designer searches the place to be checked on the screen and narrows it down to a certain range. In the narrowed area, compare the correct image pattern with the image of the place to be checked to identify it. The quality is judged by reading the change in the color, numerical value, and image shape of the specified place to be checked. The verification target of the automatic test in the area is, for example, numerical value, unit, character color, device display symbol color, abnormality display label color, alarm display character and character color, FP display operation, display in FP screen (graph, graph, Numerical value, color).

Further, as a method of automating the signal output confirmation, the screen operation switch window is automatically opened and the screen operation is automatically performed. Then, it is determined whether or not the operation signal is correctly output from the PLC 51. Specifically, for example, the MV value is input from the FP screen and confirmed by the PLC data, or the result of the operation / stop operation is confirmed by the PLC data.

For example, FIG. 7 shows an example of test scenario 170 including four test scenarios 170A, 170B, 170C, and 170D. When the automatic debug test is executed, the test result (OK / NG) and the result details (supplementary explanation) are automatically embedded, and a test result report (inspection report) 180 as illustrated in FIG. 8 is generated.

An example of the test target screen is shown in FIG. The part surrounded by the broken line is an example of the check target with this tool (automatic input to evaluate the screen state change). It should be noted that not only the tool can automatically input to the PLC 51 to check the screen display, but also the tool can automatically operate the screen to check the output of the PLC 51 in the same manner. Specifically, the tool operates the FP162 or the like, opens a sub-window (window at the time of operation from the FP162) 164 which is a screen operation switch window, and operates the output from the sub-window 164.

The overall flow is shown in FIG. As shown in the figure, for example, a hardware / communication I / O list 140 composed of calculation software data (first in the detailed design document) and a screen I / O list composed of, for example, calculation software data and image data. Input 150 (second in the detailed design document) and software function specification 160 (third in the detailed design document) composed of data for image software into the test scenario generation tool 120, for example, composed of data for calculation software. Test scenario 170 is generated. Then, by looking at the scenario generation result, the errors in the detailed design documents 140, 150, and 160 are corrected.

In this way, NG can be reduced by automatically reading the modified design document again and automatically regenerating the test scenario 170. As a result, the design documents are also checked, and the accuracy of the design documents is improved before the software is created, so that mistakes during software creation can be reduced.

Further, the test scenario 170 is inserted into the automatic debugging tool 81, and a test result report 180 composed of, for example, calculation software data is output. Here, the software is modified by re-examining the test result report 180 as a scenario as it is. At this time, the automatic debugging tool 81 can streamline the re-debugging test by skipping the OK scenario and re-debugging only the NG items.

FIG. 11 shows an example of the procedure of automatic debugging by the test scenario 170A for checking the analog input / display (screen, FP).

First, in step 1000, the presence or absence of FP is checked. If it is determined that there is no FP, the process proceeds to step 1010 and the screen definition file is read.

Then, the process proceeds to step 1020, and a correct image (font, size, color) is generated in units of test values.

Then, the process proceeds to step 1030, and the test data is automatically set by the PLC address.

Next, the process proceeds to step 1040, and the screen display position area is scanned with the correct image to perform image matching.

Then, the process proceeds to step 1050 to determine the result.

If the determination result is OK, the process proceeds to step 1060 and OK is recorded. On the other hand, if the determination result of step 1050 is NG, the process proceeds to step 1070, and NG and the reason thereof are recorded in detail.

On the other hand, if it is determined in step 1000 that there is an FP, the process proceeds to step 1100 and the FP definition file is read.

Then, the process proceeds to step 1110, and a correct FP image (font, size, color) is generated in units of test values.

Then, the process proceeds to step 1120, and the test data is automatically set in the PLC address.

Then, the process proceeds to step 1130, and the inside of the FP is scanned with the correct FP image to perform image matching.

Then, the process proceeds to step 1140 to determine the result.

If the determination result is OK, the process proceeds to step 1150, and OK is recorded. On the other hand, if the determination result in step 1140 is NG, the process proceeds to step 1160, and NG and the reason thereof are recorded in detail.

The work from step 1010 to step 1060 or 1070 is always performed, and at the same time, if FP is included in the scenario item, that is also performed. That is, when there is an FP in the item of the scenario, two operations before and after the branch are automatically executed, and OK or NG is determined for each item.

Next, FIGS. 12 and 13 show an example of the procedure of automatic debugging by the test scenario 170B for checking the analog input / alarm (screen, FP).

In this procedure, if it is determined in the first step 2000 that there is no FP, the process proceeds to step 2010 to determine the presence or absence of an alarm message. Then, if there is no determination result, the process proceeds to step 2020 and the screen definition file is read.

Next, in step 2030, a correct image (alarm color) is generated in units of test values.

Hereinafter, steps 2040 to 2080 similar to steps 1030 to 1070 in FIG. 11 are executed.

On the other hand, when the judgment result of the second step 2010 is present and the alarm message is displayed, the process proceeds to step 2100 of FIG. 13, the image definition file is read, and then the alarm message area and the correct character image are displayed in step 2110. Generate (font, size, color).

Then, the process proceeds to step 2120, and the test data is automatically set in the PLC address.

Then, the process proceeds to step 2130, and the inside of the alarm message area is scanned with the correct image to match the images.

Then, the process proceeds to step 2140, the result is determined, and if it is OK, the process proceeds to step 2150, and OK is recorded. On the other hand, if the determination result of step 2140 is NG, the process proceeds to step 2160, and NG and the reason thereof are recorded in detail.

If it is determined in the first step 2000 that there is an FP, the process proceeds to step 2200 and the FP definition file is read.

Next, in step 2210, a correct FP image (alarm color) is generated in units of test values.

Hereinafter, steps 2220 to 2260 similar to steps 1120 to 1160 in FIG. 11 are executed.

The work from step 2020 to step 2070 or 2080 is always performed, and at the same time, if an FP or an alarm message is included in the scenario item, that is also performed. For example, when it is determined that there is an FP in the item of the scenario and there is an alarm message, three operations before and after the branch are automatically executed, and OK and NG are determined for each item.

FIG. 14 shows an example of the procedure of automatic debugging by the test scenario 170C for checking the digital input / display (screen, FP).

In step 2320, a correct answer image (run / stop: shape, color) is generated with the symbol type and label replacement display contents, and in step 2410, a correct answer FP image (character, color) is generated with the label replacement display contents. Except for the points, it is substantially the same as FIG. 11, and detailed description thereof will be omitted.

An example of the procedure of automatic debugging by the test scenario 170D for checking the digital input / alarm (screen, FP) is shown in FIGS. 15 and 16.

In step 2530, a correct answer image (run / stop: shape, color) is generated with the symbol type and label replacement display contents, and in step 2610, an alarm message area and correct character image (font, size, color) are generated. Since it is substantially the same as in FIGS. 12 and 13, except that the correct FP image (character, color) is generated from the label replacement display content in step 2710, detailed description thereof will be omitted.

In this embodiment, not only the effect at the time of screen I / O inspection but also the inconsistency check of the design document is automatically output / judged when the list for performing the automatic inspection is automatically generated. Therefore, the design documents are also checked, and the accuracy of the design documents is improved before the software is created, so that mistakes when creating the software can be reduced.

In addition, when debugging, when the software designer makes a software correction of the part that was NG based on the output test report card, the test report card is used as it is as a test scenario, and the item of inspection OK is displayed by the automatic test tool. By skipping and automatically testing only the NG part, the efficiency of re-inspection can be improved. For example, when the ratio of NG is 1/10, the test time is also 1/10.

In an actual waste incineration facility, when an automatic debug test was conducted according to this embodiment with the same design specifications for the conventional manual screen I / O inspection method, the automation rate was 80, with human inspection being about 1/5 of the conventional method. % Was achieved, and the false judgment that the error was the correct answer became 0 by the automatic inspection, and the cost reduction and inspection error could be prevented by reducing the inspection man-hours by 80%, and the effectiveness could be confirmed.

It is clear that the application of the present invention is not limited to waste incineration facilities and can be applied to plants in general as well. The control device is not limited to DCS and PLC.

Further, the number of furnaces, DCS, and PLC is not limited to 2 to 3 of the embodiment, and may be one, for example.

11, 12, 13 ... Control panel 21, 22, 23 ... Sensor 31, 32, 33 ... Operation end 41, 42, 43 ... Wiring cable 51, 52, 53 ... Programmable logic controller (PLC)
71, 72, 73 ... Distributed Control System (DCS)
81, 82, 83 ... Automatic debugging tool 91, 92, 93 ... Monitor screen 100 ... Factory inspection target 110 ... Designer computer 120 ... Test scenario generation tool 122 ... Error report 140 ... Hard / communication input / output (I / O) ) List (detailed design book)
150 ... Screen input / output (I / O) list (detailed design document)
160 ... Software function specifications (detailed design documents)
162 ... (Instrument) Face plate (FP)
164 ... Sub-window (screen operation switch window)
170, 170A, 170B, 170C, 170D ... Test scenario 180 ... Test result report 200 ... Local delivery system

Claims (16)

Read the design document, automatically generate the test scenario,
Correct the design document using the error information output when the scenario is generated.
A method for automatically generating a test scenario for control software, which comprises automatically generating a test scenario again based on a modified design document. The design document is a design document for screen design,
An input / output list that describes input / output signals and
A screen image showing the coordinates corresponding to the input / output signals of the input / output list and
Includes the input / output list and software function specifications that interpolate the screen image.
The method for automatically generating a test scenario of the control software according to claim 1, wherein the test scenario is automatically generated based on these. The test using the test scenario is to imitate an analog or digital input signal to check whether the display is displayed at an appropriate display location or whether the alarm message is displayed correctly. The method for automatically generating a test scenario of the control software according to claim 1 or 2. When checking the screen using the test scenario, create a correct image pattern for input, search the design document for the location to be checked on the screen, narrow down the area, and search for the correct image pattern and the location to be checked within the narrowed down area. The test scenario of the control software according to any one of claims 1 to 3, wherein the image is compared and specified, and the change in the color, numerical value, and image shape of the specified place to be checked is read to judge whether the image is good or bad. Automatic generation method. The first to third aspects of claim 1 to 3, wherein when the signal output is confirmed using the test scenario, the screen operation switch window is opened, the screen operation is performed, and it is determined whether or not the operation signal is correctly output from the control device. The control software test scenario automatic generation method described in either. Load the test scenario, debug the control software,
A test result report is generated by the debugging.
An automatic test method for control software, characterized in that the control software is re-debugged using only the NG item of the test result report as a test scenario. The automatic test method for control software according to claim 6, wherein the test scenario is created by the test scenario automatic generation method according to any one of claims 1 to 5. A means to read design documents and automatically generate test scenarios,
A means of checking for inconsistencies between the design documents using the generated test scenario,
A means for modifying the design document using the error information output as a result of the check, and
A means to automatically generate test scenarios again with modified design documents,
A test scenario automatic generator for control software, which is characterized by being equipped with. The design document is a design document for screen design,
An input / output list that describes input / output signals and
A screen image showing the coordinates corresponding to the input / output signals of the input / output list and
Includes the input / output list and software function specifications that interpolate the screen image.
The test scenario automatic generation device for control software according to claim 8, wherein the test scenario is automatically generated based on these. A test using the above test scenario is to imitate an analog or digital input signal to check whether the display is displayed at an appropriate display location or whether the alarm message is displayed correctly. The automatic test scenario generation device for the control software according to claim 8 or 9. When checking the screen using the test scenario, create a correct image pattern for input, search the design document for the location to be checked on the screen, narrow down the area, and search for the correct image pattern and the location to be checked within the narrowed down area. The control according to any one of claims 8 to 10, wherein the control is specified by comparing images, and the change in the color, numerical value, and image shape of the specified place to be checked is read to determine the quality. Software test scenario automatic generator for. When confirming the signal output using the test scenario, the claim is characterized in that the screen operation switch window is opened, the screen operation is performed, and it is determined whether or not the operation signal is correctly output from the control device. Item 4. The automatic test scenario generation device for the control software according to any one of Items 8 to 10. A way to read test scenarios and debug control software,
A means of generating a test result report by the debugging,
A means for re-debugging the control software using only the NG item of the test result report as a test scenario,
An automatic test device for control software that features. The automatic test apparatus for control software according to claim 13, wherein the test scenario is created by the automatic test scenario generator according to any one of claims 8 to 12. A computer program for causing a computer to execute the test scenario automatic generation method according to any one of claims 1 to 5 or the automatic test method according to claim 6 or 7. A computer program for realizing the test scenario automatic generation device according to any one of claims 8 to 12 or the automatic test device according to claim 13 or 14 on a computer.