KR20230114499A - Apparatus and method for testing software - Google Patents

Abstract

The present invention relates to a software testing apparatus and method, and more particularly, to a software testing apparatus and method for checking errors in software installed in a device under test.
A software testing apparatus according to an embodiment of the present invention includes a script determining unit for determining a script defining a format of a message executable in software installed in a device under test; a scenario determining unit for determining a scenario to test the software with a message having a defined format; a message generator for generating a message to be transmitted to the device under test according to the script and scenario; a message transmission unit for transmitting the generated message to the device under test; and an analyzer configured to analyze test result data in which the transmitted message is executed in the software.

Description

Software testing apparatus and method {APPARATUS AND METHOD FOR TESTING SOFTWARE}

The present invention relates to a software testing apparatus and method, and more particularly, to a software testing apparatus and method for checking errors in software installed in a device under test.

Embedded software is installed in various military or civilian devices such as radar systems and weapon systems in order to operate or control the devices.

In this way, before the software installed device is used in a real environment, it is first checked whether the software operates normally in a test environment, that is, whether there is an error in the installed software. Here, the confirmation of software error is performed by connecting the software test device to the device on which the software is installed, that is, the device under test, and transmitting a test message from the software test device to the device under test.

In the conventional software test, whenever a software developer develops software, he or she develops a software test apparatus suitable for the developed software and tests the software. That is, a software developer manually writes and compiles source codes for testing the developed software, and then statically inspects the software by executing it in a software test device.

However, such a conventional software test method is only for testing specific software and cannot test devices with other software installed thereon. That is, in the conventional software test method, a test program suitable for each developed software must be rewritten as a whole, so there is a problem in that an additional program for the test is created and additional time and cost are required accordingly.

KR 10-2021466 B1

The present invention provides a software testing apparatus and method capable of performing a general-purpose software test without being dependent on a specific software test.

A software testing apparatus according to an embodiment of the present invention includes a script determining unit for determining a script defining a format of a message executable in software installed in a device under test; a scenario determining unit for determining a scenario to test the software with a message having a defined format; a message generator for generating a message to be transmitted to the device under test according to the script and scenario; a message transmission unit for transmitting the generated message to the device under test; and an analyzer configured to analyze test result data in which the transmitted message is executed in the software.

The script determination unit may include: a script creation unit for creating a script defining a format of a message executable in the software; and a script storage unit for storing at least one script including the script created by the script creation unit.

The scenario determination unit may include: a scenario data creation unit for creating scenario data including a transmission method and content of a message having a defined format; and a scenario data storage unit configured to store at least one scenario data including the scenario data prepared by the scenario data creation unit.

It may further include a communication environment setting unit for setting a communication environment with the device under test.

A display unit for displaying the test result data may be further included.

In addition, a software test method according to an embodiment of the present invention is a software test method for checking errors in software installed in a device under test, including: generating a script to define a format of a message executable in the software; generating a scenario to test the software with a message having a defined format; generating a message according to the script and scenario and transmitting the message to the device under test; and analyzing test result data in which the transmitted message is executed in the software to confirm an error in the software.

The process of generating the script may include checking a format of a message executable in the software; The process of writing a script to define the format of acknowledged messages; and a process of storing the prepared script.

In the process of saving the written script, the written script may be saved as an XML (eXtensible Markup Language) file or a JSON (JavaScript Object Notation) file.

The process of generating the scenario may include determining a transmission method and contents of a message having a defined format; Creating scenario data including the determined transmission method and content; and storing the created scenario data.

In the process of storing the created scenario data, the created scenario data may be saved as a CSV (Comma Separated Value) file.

In addition, a software test method according to an embodiment of the present invention is a software test method for checking errors in software installed in a device under test, and includes selecting a script for defining a format of a message executable in the software; selecting a scenario to test the software with a message having a defined format; generating a message according to the script and scenario and transmitting the message to the device under test; and analyzing the transmitted message and test result data executed by the software to confirm an error in the software.

In the process of selecting the script, a file in which a script defining a format of a message executable in the software is stored may be selected from among a plurality of files in which different scripts are stored.

In the process of selecting the scenario, a file in which scenario data to test the software is stored may be selected from among a plurality of files in which different scenario data are stored.

Prior to the process of transmitting the message to the device under test, a step of setting a communication environment for transmitting a message to the device under test may be further included.

The process of checking the error of the software may include: storing the transmitted message and test result data executed in the software as a log file; and analyzing the stored log file to determine whether or not the software has an error.

According to an embodiment of the present invention, only the format of a message executable in software is defined as a script, and a message having a defined format is used to determine a scenario for testing the software, and the message is generated and transmitted, thereby minimizing time and cost. Software errors can be checked.

1 is a diagram schematically illustrating a software testing apparatus according to an embodiment of the present invention;
2 is a diagram showing scenario data created according to an embodiment of the present invention.
3 is a view showing how to check a stored log file according to an embodiment of the present invention;
4 is a diagram schematically illustrating a software testing method according to an embodiment of the present invention;

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention will not be limited to the embodiments disclosed below, but will be implemented in various different forms, only the embodiments of the present invention will make the disclosure of the present invention complete, and will make the scope of the invention clear to those skilled in the art. It is provided to fully inform you. In order to explain the invention in detail, the drawings may be exaggerated, and like reference numerals refer to like elements in the drawings.

1 is a diagram schematically illustrating a software testing apparatus according to an embodiment of the present invention. 2 is a diagram showing scenario data created according to an embodiment of the present invention, and FIG. 3 is a diagram showing how to check a stored log file according to an embodiment of the present invention.

1 to 3, the software testing apparatus 100 according to an embodiment of the present invention is for determining a script defining a format of a message executable in software S installed in a device under test T. Script determining unit 110, a scenario determining unit 120 for determining a scenario to test the software S with a message having a defined format, and a message to be transmitted to the test target device T according to the script and the scenario A message generating unit 130 for generating a message, a message transmitting unit 140 for transmitting the generated message to the device under test (T), and the transmitted message analyzing the test result data executed by the software (S). It includes an analysis unit 150 for

The software testing apparatus 100 according to an embodiment of the present invention may check an error in the software S installed in the device under test T through a specification-based test, a so-called black box test. The black box test does not open the program code of the software (S) to be tested, that is, the source code, to check for errors, but sets the expected output value for the input value to the software (S) and returns the result as it is. By checking whether or not there is an error in the software (S), it is confirmed. Such a black box test checks whether or not the software S has an error through a syntax technique, an equal division technique, a boundary value analysis technique, and the like.

In order to test the software (S) through the black box test, a test message in which the input value entered into the software (S) is recorded is transmitted to the test target device (T) in which the corresponding software is installed, and from the test target device (T) It is necessary to check the output value recorded in the transmitted response message. However, in order for the software (S) installed in the device under test (T) to check the input value from the test message, the test message must have a specific message format. That is, in order for the software (S) to check the input value from the test message, the test message must have the format of an executable message, and only when the test message satisfies the corresponding format, the software (S) sends the input recorded in the test message. You can provide an output value for a value.

Accordingly, the script determining unit 110 determines a script defining a format of a test message executable in the software S installed in the test target device T. Here, the script may include a message specification document defining the format of the test message. The script determining unit 110 is configured to store at least one script including a script creating unit for creating a script defining a format of a test message executable in the software S and a script created by the script creating unit. A script storage unit may be included.

The script preparation unit is provided to create a script defining the format of the test message executable in the software (S). That is, the user can first check the format of the message executable in the software S installed in the device under test T, and write a script to define the format of the checked message through the script writer.

The script storage unit stores at least one script including the script prepared by the script creation unit. That is, the script storage unit may store a script prepared to define the format of a message executable in the software (S) installed in the device under test (T), and define the format of a message executable in other software (S). Scripts prepared to do so may also be stored. Here, a script prepared to define a format of a message executable in other software may be a script saved when a test has been previously performed on other software. In this way, the prepared script may be stored as an eXtensible Markup Language (XML) file or a JavaScript Object Notation (JSON) file.

For example, when a user defines the format of a “TEST_MESSAGE_A” message (refer to the message name in FIG. 2), the corresponding script can be saved as an XML file as follows.

[script-1]

[script-2]

Here, [Script-1] and [Script-2] are split scripts for defining the format of the "TEST_MESSAGE_A" message, and in [Script-1], from the line starting with <define> to </ The line ending with define> defines the format of the message header, and the lines starting with <define> in [Script-2] and ending with </define> define the format of the message body.

The scenario determining unit 120 determines a scenario to test the software S with a test message having a format defined by the script determining unit 110 . That is, the scenario determining unit 120 determines the transmission method and contents of the test message transmitted to the test target device T in order to test the software S. For example, the scenario determining unit 120 determines in what order and what kind of test message contents to be recorded and transmitted in order to test the software (S). To this end, the scenario determination unit 120 includes a scenario data creation unit for creating scenario data including a transmission method and contents of a message having a defined format, and at least one scenario data including the scenario data created by the scenario data creation unit. A scenario data storage unit for storing scenario data may be included.

The scenario data preparation unit is provided to create scenario data including a transmission method and contents of a message having a defined format. That is, the user determines the transmission method and content of the test message according to the format of the message executable in the software (S) installed in the test device (T), and scenario data including the transmission method and content determined through the scenario data preparation unit can be written.

The scenario data storage unit stores at least one scenario data including the scenario data prepared by the scenario data creation unit. That is, the scenario data storage unit may store scenario data including the transmission method and contents of the message determined for the test of the software (S) installed in the device under test (T), and in addition to the test of other software (S). Scenario data including the transmission method and content of the message determined for this purpose may also be stored. Here, the scenario data including the transmission method and contents of the message determined for the test of the other software (S) may be scenario data stored when the test of the other software was previously performed. In this way, the prepared scenario data may be saved as a CSV (Comma Separated Value) file.

2 shows "TEST_MESSAGE_A", "TEST_MESSAGE_B", "TEST_MESSAGE_C", "TEST_MESSAGE_D", "TEST_MESSAGE_E", and "TEST_MESSAGE_F" 1.1 seconds after the software testing device 100 according to an embodiment of the present invention is connected to the device under test. " messages simultaneously, and after 4.1 seconds, "TEST_MESSAGE_A", "TEST_MESSAGE_B", "TEST_MESSAGE_C", "TEST_MESSAGE_D", "TEST_MESSAGE_E", and "TEST_MESSAGE_F" messages are simultaneously transmitted. Each message may be transmitted to the device under test (T) in a set transmission method including the content of the message header and message body included in the scenario data.

The message generating unit 130 generates a test message to be transmitted to the test target device T according to the script determined by the script determining unit 110 and the scenario determined by the scenario determining unit 120 . That is, the message generating unit 130 generates a test message according to the format defined by the script determining unit 110 and the content included in the scenario data created by the scenario determining unit 120, and the generated test message is included in the scenario data. It is controlled to be transmitted to the device under test (T) by the included transmission method. The message generator 130 may automatically generate a test message by loading an XML file or JSON file stored in the script storage unit and a CSV file stored in the scenario data storage unit.

The software test apparatus 100 according to an embodiment of the present invention may further include a communication environment setting unit for setting a communication environment with the test target device T. In the communication environment setting unit, the IP (Internet Protocol) address and communication port of the software test apparatus 100 and the IP address and communication port of the test target device (T) can be set, and the software test apparatus 100 and the test target device ( It goes without saying that a communication protocol between T) may be set. Meanwhile, the message transmission unit 140 transmits the generated test message to the test target device T in the communication environment set in the communication environment setting unit.

The analysis unit 150 analyzes the test result data of the test message transmitted to the device under test (T) executed by the software (S). The analysis unit 150 analyzes test result data including an output value output by the software S installed in the test target device T with respect to the input value recorded in the test message. Here, the analysis unit 150 determines the grammar and tests whether the expected output value according to the suitable input value or the inappropriate input value comes out properly, or sets the expected output value for a plurality of areas that the input value can have in advance, and inputs the input value. In each region to which the value belongs, a random value is used as an input value to check whether the actual result is equal to the expected output value, or a value at the boundary of each region is used as an input value to check whether the actual result is equal to the expected output value. It is possible to check whether the software (S) installed in the target device (T) has an error. Test result data including such an output value may be included in a response message transmitted from the device under test (T) to the software testing device 100, and the test result data included in the response message may be included in the software testing device 100. It can be saved as a log file and analyzed.

Meanwhile, the software test apparatus 100 according to an embodiment of the present invention may further include a display unit for displaying test result data. As shown in FIG. 3, the user can check the log files stored in the software test device 100 through the display unit, and the display unit includes a plurality of scripts stored in the script storage unit and a plurality of scenario data stored in the scenario data in addition to the test result data. Of course, a screen for the user to check and select desired script and scenario data may be provided.

Hereinafter, a software test method according to an embodiment of the present invention will be described. The software test method according to an embodiment of the present invention may be a method of testing software installed in a test target device using a software test device, and the above information in relation to the software test device may be applied as it is. Description is omitted.

4 is a diagram schematically illustrating a software testing method according to an embodiment of the present invention.

Referring to FIG. 4 , a software test method according to an embodiment of the present invention is a software test method for checking errors in software (S) installed in a device (T) under test, which can be executed in the software (S). The process of generating a script to define the format of the message (S100), the process of generating a scenario to test the software (S) with a message having a defined format (S300), generating a message according to the script and scenario The process of transmitting the transmitted message to the test target device (T) (S500) and the process of analyzing the test result data executed in the software (S) and checking the error of the software (S) (S600). .

Here, the software test method according to an embodiment of the present invention includes a script for defining the format of a test message executable in the software (S) and a scenario to test the software (S) in advance with a message having a defined format. It can be done if not prepared.

On the other hand, a software test method according to another embodiment of the present invention is a software test method for checking errors in software (S) installed in a device under test (T), and a format of a message executable in the software (S) is used. A process of selecting a script to define (S200), a process of selecting a scenario to test the software (S) with a message having a defined format (S400), a message is generated according to the script and the scenario, and the device under test (T) may include a process of transmitting (S500) and a process of checking errors in the software (S) by analyzing test result data in which the transmitted message is executed in the software (S) (S600).

Here, in the software test method according to another embodiment of the present invention, a script for defining the format of a test message executable in the corresponding software (S) and a scenario for testing the corresponding software (S) with a message having a defined format are provided in advance. It can be performed when prepared and stored. However, even if the scripts and scenarios are not prepared in advance, after creating and saving the scripts and scenarios, the software (S) is tested by selecting the scripts and scenarios for the test of the software (S) among the saved scripts and scenarios. Since it can be performed, hereinafter, a method of generating, saving, and selecting scripts and scenarios to perform a test of the software (S) will be described as an example in an integrated manner.

The process of creating a script (S100) creates a script for defining a format of a message executable in the software (S). Here, the process of generating the script (S100) includes the process of checking the format of the message executable in the software (S), the process of creating a script to define the format of the confirmed message, and the process of storing the created script. can include

In the process of checking the format of the message, the user first checks the format of the message executable in the software (S) installed on the device under test (T). When the format of the message executable in the corresponding software (S) is confirmed, the user creates a script to define the format of the confirmed message in the process of preparing the script. The prepared script is stored in the script storage unit, and the script prepared to define the format of the message executable in other software (S) may also be stored in the script storage unit as described above. In this way, the prepared script may be saved as an XML file or JSON file.

In the process of selecting a script (S200), a format of a message executable in the software (S) is selected. As described above, in the script storage unit, a script prepared to define a format of a message executable in the software (S) installed in the device under test (T) may be stored as a file, and in addition, a message executable in other software (S) may be stored. The script written to define the format of can also be saved as a file. Accordingly, the user may select a file in which a script defining a format of a test message executable in the software S installed in the test target device T is stored among a plurality of files in which different scripts are stored. In this way, the process of selecting a script can be performed by displaying a plurality of files in which different scripts are stored through the display unit, and selecting a file in which a desired script is stored among them.

The process of creating a scenario (S300) creates a scenario to test the software (S) with a message having a format defined by the script. Here, the process of creating a scenario (S300) includes the process of determining the transmission method and contents of a message having a defined format, the process of creating scenario data including the determined transmission method and contents, and the process of storing the created scenario data. can include

In the process of determining the transmission method and contents of the message, the user decides in what order and what kind of test message to record and transmit in order to test the software (S). When the transmission method and contents of the message for testing the corresponding software (S) are determined, the user creates scenario data including the transmission method and contents of the determined message. As described above, the prepared scenario data is stored in the scenario data storage unit, and scenario data including transmission methods and contents of messages determined for testing other software (S) may also be stored in the scenario data storage unit. Scenario data created in this way may be saved as a CSV file.

The process of selecting a scenario (S400) selects a scenario to test the software (S) with a message having a defined format. As described above, in the scenario data storage unit, scenario data including the transmission method and contents of the message determined for the test of the software (S) installed in the device under test (T) may be stored as a file, and other software ( Scenario data including the transmission method and content of the message determined for the test of S) can also be saved as a file. Accordingly, the user may select a file in which scenario data for testing the software (S) installed in the test target device (T) is stored among a plurality of files in which different scenario data are stored. In this way, the process of selecting scenario data can be performed by displaying a plurality of files in which different scenario data are stored through the display unit, and the user selects a file in which desired scenario data is stored.

In the process of transmitting to the test target device T (S500), a test message is generated according to the created or selected script and scenario, and the generated test message is transmitted to the test target device T. Here, in the software test method according to an embodiment of the present invention, prior to transmitting the generated test message to the device under test T, the process of setting a communication environment for transmitting the test message to the device under test T is further performed. can include In the process of setting the communication environment, the IP (Internet Protocol) address and communication port of the software test apparatus 100, and the IP address and communication port of the test target device (T) may be set, and the software test apparatus 100 and the test subject It goes without saying that a communication protocol between the devices T may be set. Meanwhile, in the process of transmitting to the test target device (T) (S500), the test message is transmitted to the test target device (T) under the communication environment set in the process of setting the communication environment.

In the process of checking the error of the software (S) (S600), the test message transmitted to the test target device (T) analyzes the test result data executed by the software (S). In the process of checking the error of the software S (S600), test result data including an output value output by the software S installed in the test target device T with respect to the input value recorded in the test message is analyzed. Here, the process of checking the error of the software (S) (S600) is to test whether the expected output value according to the proper input value or the inappropriate input value comes out properly after determining the grammar, or to predict a plurality of areas that the input value can have. After pre-determining the output value, use a random value as an input value in each area to which the input value belongs to check whether the actual result is the same as the expected output value, or use the value at the boundary of each area as an input value so that the actual result is It is possible to check whether the software (S) installed in the device under test (T) has an error by checking whether it is the same as the expected output value.

The process of checking the error of the software (S) (S600) is the process of saving the transmitted test message and the test result data executed in the corresponding software (S) as a log file and analyzing the stored log file to determine the quality of the software (S). It may include a process of checking for errors. That is, the test result data including the above-described output value may be included in a response message transmitted from the device under test T to the software test device 100, and the test result data included in the response message may be included in the software test device 100. ) is saved as a log file and analyzed, and if the output value is the same as the expected output value, it is determined that there is no error in the software (S), and if the output value is different from the expected output value, it is determined that the software (S) has an error. can

In this way, according to an embodiment of the present invention, by defining only the format of a message executable in software as a script, determining a scenario to test the software with a message having a defined format, and generating and transmitting the message, the time and You can check for errors in various software at a cost.

In the above, although preferred embodiments of the present invention have been described and illustrated using specific terms, such terms are only intended to clearly explain the present invention, and the embodiments and described terms of the present invention are the technical spirit of the following claims. And it is obvious that various changes and changes can be made without departing from the scope. Such modified embodiments should not be individually understood from the spirit and scope of the present invention, and should be said to fall within the scope of the claims of the present invention.

110: script decision unit 120: scenario decision unit
130: message generation unit 140: message transmission unit
150: analysis unit

Claims (15)

a script determining unit for determining a script defining a format of a message executable in software installed in a device under test;
a scenario determining unit for determining a scenario to test the software with a message having a defined format;
a message generator for generating a message to be transmitted to the device under test according to the script and scenario;
a message transmission unit for transmitting the generated message to the device under test; and
A software test apparatus comprising: an analyzer configured to analyze test result data in which the transmitted message is executed in the software. The method of claim 1,
The script determining unit,
a script writing unit for creating a script defining a format of a message executable in the software; and
A software test apparatus comprising a; script storage unit for storing at least one script including the script created by the script creation unit. The method of claim 1,
The scenario decision unit,
a scenario data creation unit for creating scenario data including a transmission method and content of a message having a defined format; and
A software test apparatus comprising: a scenario data storage unit configured to store at least one scenario data including the scenario data prepared by the scenario data creation unit. In any one of claims 1 to 3,
A software test apparatus further comprising a communication environment setting unit for setting a communication environment with the device under test. In any one of claims 1 to 3,
A software test apparatus further comprising a display unit for displaying the test result data. As a software test method for checking errors in software installed on a device under test,
generating a script for defining a format of a message executable in the software;
generating a scenario to test the software with a message having a defined format;
generating a message according to the script and scenario and transmitting the message to the device under test; and
A software test method comprising: analyzing test result data of a transmitted message executed in the software to confirm an error in the software. The method of claim 6,
The process of creating the script is,
checking a format of a message executable by the software;
The process of writing a script to define the format of acknowledged messages; and
A process of saving the written script; software test method including. The method of claim 7,
The process of saving the written script,
A software test method that saves written scripts as XML (eXtensible Markup Language) files or JSON (JavaScript Object Notation) files. The method of claim 6,
The process of generating the scenario is,
The process of determining the transmission method and content of a message having a defined format;
Creating scenario data including the determined transmission method and content; and
A software test method comprising: saving written scenario data. The method of claim 9,
The process of storing the prepared scenario data,
A software test method that saves written scenario data as a CSV (Comma Separated Value) file. As a software test method for checking errors in software installed on a device under test,
selecting a script for defining a format of a message executable in the software;
selecting a scenario to test the software with a message having a defined format;
generating a message according to the script and scenario and transmitting the message to the device under test; and
A software test method comprising: analyzing test result data of a transmitted message executed in the software to confirm an error in the software. The method of claim 11,
The process of selecting the script,
A software test method of selecting a file in which a script that defines a format of a message executable in the software is stored among a plurality of files in which different scripts are stored. The method of claim 11,
The process of selecting the scenario is,
A software testing method of selecting a file storing scenario data to test the software from among a plurality of files storing different scenario data. The method according to any one of claims 6 to 13,
Before the process of transmitting to the device under test,
The software test method further comprising setting a communication environment for transmitting a message to the device under test. The method according to any one of claims 6 to 13,
The process of checking the error of the software,
saving the transmitted message and test result data executed in the software as a log file; and
A software test method comprising: analyzing stored log files to determine whether or not the software has errors.

Images