JP7392821B2 - Automatic testing method and device for control software and computer program - Google Patents

Description

The present invention relates to an automatic test method and apparatus for control software, and a computer program, and in particular, an automatic test for control software suitable for use in screen input/output inspection in a central monitoring and control device of a plant such as a waste incinerator. METHODS AND APPARATUS AND COMPUTER PROGRAMS.

In plants such as waste incinerators, the control software for the central monitoring and control equipment must be tested in an actual test environment before being shipped from the factory before being delivered to the site.

When shipped from the factory, there are no local devices (sensors, control panels, operation terminals) or wiring to these devices, so the input and output of a controller, such as a programmable logic controller (PLC), is simulated and inspected.

Conventionally, in screen input/output (I/O) inspections, it has been difficult to automate the identification of the location of the object to be checked within the screen, and identification and pass/fail determination have been performed using human judgment. Screen I/O inspection is a product shipping inspection, and to ensure quality, sampling (partial) inspection is not allowed and must be confirmed in full. Conventionally, as shown in FIG. 1, an input worker 2 creates and inputs a test I/O list 4, and a confirmation worker 6 checks changes in numbers and colors on a confirmation screen 8. Multiple workers manually inspected each item one by one, which not only took time and labor, but also caused the risk of human checking errors. Furthermore, it was difficult to find the cause when a check error occurred.

Furthermore, as prior art related to the present application, Patent Documents 1 and 2 describe techniques for automatically generating and testing pseudo signals and simulation programs regarding a communication interface between an HMI (Human Machine Interface) screen and a PLC. has been done.

Further, Patent Document 3 describes a technique for creating a PLC ladder program from a circuit drawing.

Furthermore, Patent Document 4 describes a technique that allows one person to carry out a function confirmation test of a PLC in a plant monitoring and control device using voice output.

Furthermore, Patent Document 5 describes a technique for automatically generating, displaying, and tracking checklists according to assigned work items.

Further, Patent Document 6 describes a technique of creating a test code based on a test specification, performing a test using the test code, and creating a test result report by compiling the test results.

Further, Patent Document 7 describes a technique of selecting and executing registered test items, and after the test is completed, analyzing the test results and creating a report.

Further, Patent Document 8 describes a simulation support tool that automatically generates test inputs.

Further, Patent Document 9 describes a software automatic test system that determines the quality of a test screen generated by a device to be inspected that stores a screen generation program by image comparison.

JP 2017-129957 Publication JP 2017-130139 Publication Patent No. 5498446 Patent No. 3713150 JP2014-225232A Patent No. 4777740 Japanese Patent Application Publication No. 2006-215907 Patent No. 4379687 Patent No. 5540413

However, until now, only fragmentary techniques have been described, and comprehensive techniques have not been described.

The present invention has been made to solve the above-mentioned conventional problems, and is a technology that can automate testing (debugging) of control software to reduce costs by reducing inspection man-hours and prevent inspection errors. The challenge is to provide the following.

The present invention reads a test scenario to test control software, generates a test result report based on the test, allows a designer to check the NG part of the program and corrects the control software, and NG items in the test result report. In an automatic testing method for control software in which the control software modified by the designer is retested using only a test scenario, when checking the screen using the test scenario, a correct image pattern of the input is created and the screen is checked from the design document. Search for the location that should be checked, narrow down the areas that are divided into screens, compare the correct image pattern and the location that should be checked within the narrowed down area, identify the location that should be checked, and check the color, numerical value, and image shape of the identified location that should be checked. The above-mentioned problem is solved by providing an automatic testing method for control software, which is characterized by reading changes and determining pass/fail.

In the present invention, the control software is tested by reading a test scenario, a test result report is generated by the test, the designer checks the NG part of the program and corrects the control software, and the test result report is In an automatic test method for control software in which control software modified by the designer is retested using only NG items as a test scenario, when confirming signal output using the test scenario, open a screen operation switch window and perform screen operations. The object of the present invention is to similarly solve the above problem by providing an automatic test method for control software, which is characterized by determining whether the operation signal is correctly output from the control device.

Here , when operating the screen, the software automatically forcibly operates the output value using the screen operation switch window, and it is possible to judge whether the output signal is a correct value or not.

The present invention also provides a means for reading a test scenario and testing the control software, a means for generating a test result report by the test, and a means for a designer to check the NG part of the program and correct the control software. , means for retesting the control software modified by the designer using only the NG items in the test result report as a test scenario; in an automatic test device for control software, when checking the screen using the test scenario, Create a correct image pattern for the input, search for the places on the screen that should be checked from the design document, narrow down the areas into which the screen is divided, compare images with the correct image pattern and identify the places that should be checked within the narrowed down area, The above-mentioned problem is similarly solved by providing an automatic test device for control software, which is characterized by reading changes in color, numerical value, and image shape of a specified location to be checked to determine pass/fail. It is something to do.
The present invention also provides a means for reading a test scenario and testing the control software, a means for generating a test result report by the test, and a means for a designer to check the NG part of the program and correct the control software. , means for retesting the control software modified by the designer using only the NG items in the test result report as a test scenario; in an automatic test device for control software, when confirming a signal output using the test scenario; To provide an automatic test device for control software, characterized in that the device opens a screen operation switch window, performs screen operations, and determines whether or not operation signals are correctly output from a control device. This similarly solves the above problem.

The present invention also provides a computer program for causing a computer to execute the automatic test method.

The present invention also provides a computer program for implementing the automatic test device on a computer.

According to the present invention, it is possible to automate testing (debugging) of control software, thereby reducing costs by reducing the number of inspection steps and preventing inspection errors.

Conceptual diagram showing the state of conventional inspection A diagram showing an example of a system configuration of a debugging test target plant in which an embodiment of the present invention is adopted. A diagram showing the flow of a debug test in the embodiment A diagram showing an example of an input/output list of one of the detailed design documents also used in debugging tests. A diagram showing an example of the screen input/output list as well. Similarly, a diagram showing an example of a software functional specification Diagram also showing an example test scenario Similarly, a diagram showing an example of a test result report. A diagram showing an example of the screen to be tested as well. Diagram also showing the overall flow A flowchart showing the steps for automatic debugging using a test scenario that also checks analog inputs/displays. A flowchart showing the procedure for automatic debugging using a test scenario that also checks analog inputs/alarms. Flowchart showing a continuation of Figure 12 Flowchart showing the procedure of automatic debugging using a test scenario for checking digital inputs/displays in the embodiment. Flowchart showing the steps for automatic debugging with a test scenario that also checks digital inputs/alarms. Flowchart showing a continuation of Figure 15

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the contents described in the following embodiments and examples. Furthermore, the constituent elements in the embodiments and examples described below include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Further, the constituent elements disclosed in the embodiments and examples described below may be combined as appropriate, or may be appropriately selected and used.

FIG. 2 shows an example of the system configuration of a debug test target plant (for example, a garbage incinerator) targeted by the present invention. This plant has a No. 1 incinerator and a No. 2 incinerator (not shown), each with control panels 11 and 12, sensors 21 and 22, operating ends 31 and 32, wiring cables 41 and 42, and a programmable logic controller ( PLC) 51, 52 are provided. Further, a control panel 13, a sensor 23, an operating end 33, a wiring cable 43, and a PLC 53 are provided for the common system of the No. 1 incinerator and the No. 2 incinerator. Further, distributed control systems (DCS) 71, 72, 73 and monitor screens 91, 92, 93 are provided as a monitoring and operation system for the No. 1 incinerator, No. 2 incinerator, and the entire common system.

Among these, the factory shipment inspection targets 100 are the PLCs 51, 52, 53, the DCSs 71, 72, 73, and the monitor screens 91, 92, 93.

For the final form of the local delivery system 200, when shipped from the factory, there are local devices (sensors 21, 22, 23, control panels 11, 12, 13, control ends 31, 32, 33) and wiring cables 41 for connecting these devices. , 42, and 43 do not exist, it is necessary to simulate and inspect the input/output of PLCs 51, 52, and 53. As illustrated in Fig. 1, conventionally each point was manually inspected, which required multiple workers, including an input worker 2 and a confirmation worker 6. Furthermore, since the check was performed by humans, There was a possibility of a mistake.

Therefore, in this embodiment, automatic debugging tools 81, 82, and 83, which are software for generating simulated signals, are installed in the DCSs 71, 72, and 73, respectively, and the devices are connected to each other via communication cables to create a factory inspection system configuration.

On the other hand, as shown in FIG. 3, a software test scenario creation tool 120 is installed on the designer's computer 110, and detailed design documents 140, 150, and 160, which will be described in detail later, are automatically read and a test scenario 170 is automatically created. generate. Then, a detailed design document error report 122 indicating the OK/NG determination result is generated, and the designer checks the NG portions and corrects the detailed design documents 140, 150, and 160. Then, the created test scenario 170 is copied to the DCS (71 in the example of FIG. 3).

In this way, by automatically reading the modified design document again and automatically regenerating the test scenario 170, it is possible to reduce the number of NGs. This allows the design documents to be checked and improves the accuracy of the design documents before creating the software, reducing errors when creating the software.

On the other hand, an automatic debugging tool 81 is installed in the DCS 71 to automatically read the test scenario 170 generated by the designer computer 110 and perform automatic debugging. Then, a test result report 180 including the OK/NG determination results is generated. Then, the designer checks the NG points and corrects the DCS software.

Furthermore, the test result report 180 is automatically read in place of the test scenario 170 and debugging is repeated. At this time, the automatic debugging tool 81 can improve the efficiency of the re-debugging test by skipping the scenarios that passed OK and re-debugging only the NG items.

FIG. 4 shows an example of a hardware/communication input/output (I/O) list 140 of one of the detailed design documents 140, 150, 160 for screen design used to generate the test scenario 170. The main items as shown in Figure 4 (there are other items) are defined and described, and test data is generated from these items. In the figure, AI is an analog input and DI is a digital input.

Further, FIG. 5 shows an example of a screen input/output (I/O) list 150, which is another example of the detailed design documents 140, 150, and 160 for screen design. This is a combination of the screen image showing the coordinates at the bottom of the figure and the list at the top of the figure. The main items (there are other items) as shown in Figure 5 are defined and described, and test data is generated from these items.

FIG. 6 also shows an example of a software function specification 160, which is a third example of the detailed design documents 140, 150, and 160 for screen design. In the figure, 162 represents an instrument face plate (FP) that is a control screen for an instrument (the example in the figure is a thermometer). This software function specification 160 interpolates the hardware/communication input/output list 140 and the screen input/output list 150, and reads faceplate information (tags with faceplates, alarm settings, etc.) from the specification creation rules. , generate test data.

A method for automating screen confirmation is to automatically input signals based on a list. Specifically, the designer creates a correct image pattern for the input, searches for areas on the screen that should be checked, and narrows it down to a certain range. Within the narrowed-down area, the correct image pattern and the location to be checked are compared and identified. It reads the changes in color, numerical value, and image shape of the identified areas to be checked to determine pass/fail. The verification targets for automatic testing in the area include, for example, numerical values, units, font colors, equipment display symbol colors, error display label colors, alarm display characters and font colors, FP display operations, and displays on the FP screen (graphs, (number, color).

Further, as a method for automating signal output confirmation, a screen operation switch window is automatically opened and screen operations are automatically performed. Then, it is determined whether the operation signal is correctly output from the PLC 51 or not. Specifically, for example, the MV value is input from the FP screen and checked with PLC data, or the results of running and stopping operations are checked with PLC data.

An example of a test scenario 170 including, for example, four test scenarios 170A, 170B, 170C, and 170D is shown in FIG. When an automatic debug test is executed, the test result (OK/NG) and result details (supplementary explanation) are automatically embedded, and a test result report (inspection report) 180 as illustrated in FIG. 8 is generated.

An example of the screen to be tested is shown in FIG. The area surrounded by broken lines is an example of what this tool checks (automatically inputs and evaluates screen state changes). In addition to automatically inputting information into the PLC 51 from the tool and checking the screen display, it is also possible to check the output of the PLC 51 by automatically operating the screen from the tool. Specifically, the tool operates the FP 162 or the like, opens a sub-window (operation window from the FP 162) 164, which is a screen operation switch window, and operates the output from this sub-window 164.

The overall flow is shown in FIG. As shown in the figure, there is a hardware/communication I/O list 140 (first part of the detailed design document) made up of data for calculation software, for example, and a screen I/O list made up of data for calculation software and image data, for example. 150 (second detailed design document) and software function specification 160 (third detailed design document) consisting of data for image software are input into the test scenario generation tool 120, and the A test scenario 170 is generated. Then, by looking at the scenario generation results, errors in the detailed design documents 140, 150, and 160 are corrected.

In this way, by automatically reading the modified design document again and automatically regenerating the test scenario 170, it is possible to reduce the number of NGs. This allows the design documents to be checked and improves the accuracy of the design documents before creating the software, reducing errors when creating the software.

Further, the test scenario 170 is inserted into the automatic debugging tool 81, and a test result report 180 composed of, for example, data for calculation software is output. Here, the software is corrected by reexamining the test result report 180 as it is as a scenario. At this time, the automatic debugging tool 81 can improve the efficiency of the re-debugging test by skipping the scenarios that passed OK and re-debugging only the NG items.

FIG. 11 shows an example of an automatic debugging procedure using a test scenario 170A that checks analog input/display (screen, FP).

First, in step 1000, the presence or absence of an FP is checked. If it is determined that there is no FP, the process advances to step 1010 and a screen definition file is read.

Next, the process proceeds to step 1020, where a correct image (font, size, color) is generated using the test numerical value and unit.

Next, the process proceeds to step 1030, where test data is automatically set using the PLC address.

Next, the process proceeds to step 1040, where the screen display position area is scanned with the correct image to perform image matching.

Next, the process proceeds to step 1050, where the result is determined.

If the determination result is OK, the process advances to step 1060 and OK is recorded. On the other hand, if the determination result in step 1050 is NG, the process proceeds to step 1070, and the NG and the reason thereof are recorded in detail.

On the other hand, if it is determined in step 1000 that there is an FP, the process proceeds to step 1100, and an FP definition file is read.

Next, the process proceeds to step 1110, where a correct FP image (font, size, color) is generated using the test numerical value and unit.

Next, the process proceeds to step 1120, where test data is automatically set to the PLC address.

Next, the process proceeds to step 1130, where the inside of the FP is scanned with the correct FP image and image matching is performed.

Next, the process proceeds to step 1140, where the result is determined.

If the determination result is OK, the process advances to step 1150 and OK is recorded. On the other hand, if the determination result in step 1140 is NG, the process proceeds to step 1160, and the NG and the reason thereof are recorded in detail.

Note that the operations from step 1010 to step 1060 or 1070 are always performed, and at the same time, if FP is included in the scenario item, they are also performed. That is, if there is an FP in the scenario item, two tasks before and after branching are automatically executed, and OK or NG is determined for each item.

Next, an example of an automatic debugging procedure using a test scenario 170B for checking analog input/alarms (screen, FP) is shown in FIGS. 12 and 13.

In this procedure, if it is determined in the first step 2000 that there is no FP, the process proceeds to step 2010, and it is determined whether or not there is an alarm message. If there is no determination result, the process advances to step 2020 and the screen definition file is read.

Next, in step 2030, a correct image (alarm color) is generated using the test numerical value and unit.

Thereafter, steps 2040 to 2080 similar to steps 1030 to 1070 in FIG. 11 are executed.

On the other hand, if there is a judgment result in the second step 2010 and a warning message is to be displayed, the process advances to step 2100 in FIG. (font, size, color).

Next, the process proceeds to step 2120, where test data is automatically set to the PLC address.

Next, the process proceeds to step 2130, in which the correct image is scanned within the alarm message area and image matching is performed.

The process then proceeds to step 2140, where the result is determined, and if OK, the process proceeds to step 2150, where OK is recorded. On the other hand, if the determination result in step 2140 is NG, the process proceeds to step 2160, and the NG and the reason thereof are recorded in detail.

Further, if it is determined in the first step 2000 that there is an FP, the process advances to step 2200 and reads the FP definition file.

Next, in step 2210, a correct FP image (alarm color) is generated using the test numerical value and unit.

Thereafter, steps 2220 to 2260 similar to steps 1120 to 1160 in FIG. 11 are executed.

Note that the operations from step 2020 to step 2070 or 2080 are always performed, and at the same time, if FP and warning messages are included in the scenario items, they are also performed. For example, if it is determined that there is an FP in the scenario item and there is a warning message, three tasks before and after branching are automatically executed, and OK or NG is determined for each item.

FIG. 14 shows an example of an automatic debugging procedure using a test scenario 170C that checks digital input/display (screen, FP).

In step 2320, a correct image (run/stop: shape, color) is generated based on the symbol type and label replacement display content, and in step 2410, a correct FP image (text, color) is generated based on the label replacement display content. Since this is substantially the same as FIG. 11 except for this point, detailed explanation will be omitted.

Examples of automatic debugging procedures using a test scenario 170D for checking digital inputs/alarms (screen, FP) are shown in FIGS. 15 and 16.

In step 2530, a correct image (run/stop: shape, color) is generated based on the symbol type and label replacement display contents, and in step 2610, an alarm message area and correct character image (font, size, color) are generated. 12 and 13, except that in step 2710, a correct FP image (text, color) is generated based on the label replacement display contents, and detailed explanation will be omitted.

In this embodiment, in addition to the effects of screen I/O inspection, when automatically generating a list for automatic inspection, the inconsistency check of design documents is automatically output and determined as a report. This allows the company to check the design documents and improve the accuracy of the design documents before creating the software, reducing errors when creating the software.

In addition, when debugging, when the software designer modifies the parts that were rejected based on the output inspection report, the inspection report can be used as it is as a test scenario, and the items that are OK to be inspected can be checked using an automatic test tool. By skipping and automatically testing only the NG parts, re-examination can be made more efficient. For example, if the NG ratio is 1/10, the test time will also be 1/10.

At an actual waste incineration facility, we conducted an automatic debugging test using this embodiment with the same design specifications as compared to the conventional manual screen I/O inspection method, and found that the automation rate was 80, with human inspection being about 1/5 of the conventional method. %, and the number of erroneous judgments in which errors are considered correct was reduced to 0 through automatic inspection, reducing costs by reducing inspection man-hours by 80%, preventing inspection errors, and confirming effectiveness.

Note that the present invention is not limited to waste incineration facilities, but is obviously applicable to plants in general. The control device is also not limited to DCS or PLC.

Furthermore, the number of furnaces, DCSs, and PLCs is not limited to two or three in the embodiments, and may be one, for example.

11, 12, 13... Control panel 21, 22, 23... Sensor 31, 32, 33... Operating end 41, 42, 43... Wiring cable 51, 52, 53... Programmable logic controller (PLC)
71, 72, 73...Distributed control system (DCS)
81, 82, 83...Automatic debugging tool 91, 92, 93...Monitor screen 100...Factory inspection target 110...Designer's computer 120...Test scenario generation tool 122...Error report 140...Hardware/communication input/output (I/O ) list (detailed design documents)
150...Screen input/output (I/O) list (detailed design document)
160...Software functional specifications (detailed design documents)
162...(Instrument) Face plate (FP)
164...Sub window (screen operation switch window)
170, 170A, 170B, 170C, 170D...Test scenario 180...Test result report 200...Local delivery system

Claims (8)

Load the test scenario to test the control software,
generating a test result report by the test;
The designer checks the NG part of the program and corrects the control software,
In an automatic test method for control software, the method re-tests the control software modified by the designer using only the NG items in the test result report as a test scenario,
When checking the screen using the above test scenario, create a correct image pattern for the input, search the design document for the location on the screen that should be checked, narrow down the areas in which the screen is divided, and check the correct image pattern within the narrowed down area. 1. An automatic test method for control software, characterized by identifying a location to be checked by comparing images, and determining pass/fail by reading changes in color, numerical value, and image shape of the identified location to be checked. Load the test scenario to test the control software,
generating a test result report by the test;
The designer checks the NG part of the program and corrects the control software,
In an automatic test method for control software, the method re-tests the control software modified by the designer using only the NG items in the test result report as a test scenario,
When checking the signal output using the test scenario, the control software automatically opens a screen operation switch window, performs screen operation, and determines whether the operation signal is correctly output from the control device. Test method. 3. The control software according to claim 2 , wherein when the screen is operated, the software automatically forcibly operates the output value using the screen operation switch window to determine whether the output signal is a correct value or not. Automated testing methods. a means for testing the control software by loading test scenarios;
means for generating a test result report from the test;
means for a designer to check the NG part of the program and correct the control software;
means for retesting the control software modified by the designer using only the NG items in the test result report as a test scenario;
In automatic test equipment for control software equipped with
When checking the screen using the above test scenario, create a correct image pattern for the input, search the design document for the location on the screen that should be checked, narrow down the areas in which the screen is divided, and check the correct image pattern within the narrowed down area. An automatic test device for control software, characterized in that a location to be checked is identified by comparing images, and a pass/fail judgment is made by reading changes in color, numerical value, and image shape of the identified location to be checked. a means for testing the control software by loading test scenarios;
means for generating a test result report from the test;
means for a designer to check the NG part of the program and correct the control software;
means for retesting the control software modified by the designer using only the NG items in the test result report as a test scenario;
In automatic test equipment for control software equipped with
When confirming the signal output using the test scenario, a screen operation switch window is opened, the screen is operated, and it is determined whether the operation signal is correctly output from the control device or not. Automatic test equipment for control software. 6. When the screen is operated, software automatically forcibly operates the output value using the screen operation switch window to determine whether the output signal is a correct value or not. Automatic test equipment for control software. A computer program for causing a computer to execute the automatic test method according to any one of claims 1 to 3 . A computer program for causing a computer to implement the automatic test device according to any one of claims 4 to 6 .

Images