JP2020087106A - Test support method, test execution method, test support device and computer program - Google Patents

Abstract

To provide a test support method, a test execution method, a test support device and a computer program that realize efficiently selecting a method of a test.SOLUTION: A test support method comprises: storing inquiries from a different user including any of a predetermined plurality of keywords corresponding to a software program; extracting any of the predetermined plurality of keywords included in the inquiries from the user; calculating a value indicating a priority for the extracted keyword on the basis of an appearance frequency of the extracted keyword or importance of a user of an inquiry source; and outputting the predetermined plurality of keywords and the value indicating the calculated priority by associating them with each other.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a test support method, a test execution method, a test support device, and a computer program that realize the efficiency of a test process of a software program.

In the development of many software programs, a quality improvement method called continuous integration (Continuous Integration) is adopted to realize quality control of the software program in a short period of time, and build (test), deploy, and deploy are cyclically repeated. Has been. Non-Patent Document 1 discloses a history in which a certain program is maintained/updated by many users.

In the test process of continuous improvement, a regression test is performed to test whether there is any effect on other parts in response to bug fixes after the release of the program, or partial changes due to the addition of functions. Be done. Regression testing is important for continuous improvement, but it is time consuming and costly, accounting for 50-66% of the program's lifetime management costs (Non-Patent Document 2).

Many regression tests are created by testers under the assumption that the software program in question is used, and executed according to the situation. However, it requires man-hours because it is manually created. Regarding regression tests, many innovations have been made in the past to improve efficiency, such as creating scripts for automatic testing and narrowing down to tests that handle data related to corrections. Among them, Non-Patent Document 3 discloses a method in which the dependency of a function or variable included in a software program is modeled as an effective graph, the importance is determined for an arbitrary input group, and the test priority is determined. Has been done. It has been proposed to efficiently select and execute tests by such a method.

A so-called CAE (Computer Aided Engineering) program that realizes analysis of physical phenomena in an object on a computer is used in various fields such as structural analysis, welding strain analysis, thermo-fluid analysis, and electromagnetic field analysis. .. The software program for physical phenomenon analysis takes a long time to perform the simulation in order to execute the simulation, and reads the file for setting the shape (structure), material, condition, etc. of the object that is the prerequisite of the analysis as a specification. , There is one that outputs the analysis result as a file (Patent Document 1, etc.). At this time, the input/output file is a reserved word (keyword) corresponding to a parameter name, a variable name, a function name, a predetermined value, an error number, etc. set for each software program for physical phenomenon analysis, and a numerical value, It is described by associating logical values and the like (Non-Patent Document 4, etc.).

International publication 2013/084306

"Debian Continuous Integration", [online], Debian Project, [Search August 10, 2018], Internet <URL:https://ci.debian.net/> R. Pressman, “Software Engineering: A Practitioner's Approach”, McGraw-Hill Education, USA, 2000 Swarnendu Biswas et al., “Regression Test Selection Techniques: A Survey”, Informatica 35, 2011, pp.289-321, Internet <URL:https://users.ices.utexas.edu/~sbiswas/files/papers /informaticasurvey.pdf> "LS-DYNA Manual R10.0-Vol I", [online], October 16, 2017, Livermore Software Technology Corporation, [search on August 10, 2018], internet <URL:https://www.dynasupport .com/manuals/ls-dyna-manuals/ls-dyna-manual-r10.0-vol-i>

This kind of software program for physical phenomenon analysis has various shapes of objects, and there are various combinations with materials and boundary conditions. For this reason, as disclosed in Non-Patent Document 1, there are many types of reserved words used to describe the input/output file, and the number of combinations of reserved words for describing the analysis target is enormous. Therefore, the number of test cases for the software program for physical phenomenon analysis is basically larger for each reserved word.

By applying the test selection method described above to the software program for physical phenomenon analysis, it is possible to narrow down from a huge number of test cases. However, even if the regression test can be narrowed down around the data related to the change, it does not always correspond to the needs of the user, and it is not always possible to improve the efficiency of the test of the part with high needs. Further, when the configuration is such that a software program provided from another is incorporated and the configuration is unknown or complicated, it is not always possible to select a test based on the design information of the software program.

An object of the present disclosure is to update a test priority of a software program to reduce a test man-hour, and a test support method and a test execution for realizing a test automatic operation for continuous improvement in response to a user's need. A method, a test support device, and a computer program are provided.

A test support method according to an aspect of the present disclosure stores an inquiry from a different user including any of a plurality of predetermined keywords corresponding to a software program, and the predetermined plurality of items included in the inquiry from the user. Of the extracted keywords, and calculates the value indicating the priority of the extracted keywords based on the appearance frequency of the extracted keywords or the importance of the user who made the inquiry, and calculates the predetermined plurality of keywords. The value indicating the priority thus set is output in association with the value.

In a test execution method according to an aspect of the present disclosure, when update completion of a software program is detected, a priority calculated based on an inquiry from a user with respect to a plurality of predetermined keywords corresponding to the software program. The computer refers to the test case extracted by, and the computer executes the referenced test case.

A test support apparatus according to an aspect of the present disclosure includes a storage unit that stores an inquiry from a different user including any of a plurality of predetermined keywords corresponding to a software program, and the storage unit included in the inquiry from the user. An extraction unit that extracts any of a plurality of predetermined keywords, and a calculation that calculates a value indicating the priority for the extracted keyword based on the appearance frequency of the extracted keyword or the importance of the user who made the inquiry. And an output unit that outputs the predetermined plurality of keywords in association with the calculated value indicating the priority.

A computer program according to an aspect of the present disclosure records, in a computer, an inquiry from a different user including any of a plurality of predetermined keywords corresponding to a software program, and the predetermined information included in the inquiry from the user. Any of the plurality of keywords is extracted, and based on the appearance frequency of the extracted keyword or the importance of the user who made the inquiry, a value indicating the priority for the extracted keyword is calculated, and the predetermined plurality of The process of outputting the keyword and the value indicating the calculated priority in association with each other is executed.

According to one aspect of the present disclosure, any one or a plurality of keywords (reserved words) corresponding to parameter names, variable names, function names, predetermined values, error numbers, etc. set for a predetermined software program are used. Inquiries from different users are analyzed, and a value indicating the priority of the keyword is calculated by reflecting the importance of the user of the inquiry source as a customer. By calculating the value indicating the priority of the keyword, a test case including more keywords having higher priority or having higher correlation is selected.

In one aspect of the present disclosure, the value indicating the priority is calculated by adding the appearance frequency or the importance calculated for each different period with a lower influence of those values in the older period.

In one aspect of the present disclosure, the value indicating the priority may be calculated by using any one or all of the number of visits regarding the target software program, the number of contract licenses, and the sales amount as an element. Different weights may be given to each element for calculation.

According to an aspect of the present disclosure, a learning model for determining whether or not a keyword included in an inquiry is important in the context of the inquiry is used, and a value indicating a priority is calculated using the importance. To be done.

According to the test support method, test execution method, test support apparatus, and computer program of the present disclosure, it is possible to efficiently select a test method.

It is a figure which shows the outline of the test support apparatus in this Embodiment. It is a block diagram which shows the structure of a test support apparatus. It is a flow chart which shows an example of the processing procedure of selection of a test case. It is a flow chart which shows an example of the automatic execution processing procedure of a test. 6 is a flowchart showing an example of a priority calculation method. It is explanatory drawing which shows the example of a test case. 9 is a flowchart showing a method of calculating a priority in Modification 1. 9 is a flowchart showing a method of calculating a priority in Modification 2. It is a figure which shows the example of the content of a learning model. 9 is a flowchart showing a method of calculating a priority in Modification 3.

Specific examples of the test support device and the computer program according to the embodiment of the present disclosure will be described below with reference to the drawings. It should be noted that the present invention is not limited to these exemplifications, and is shown by the scope of the claims, and is intended to include meanings equivalent to the scope of the claims and all modifications within the scope.

FIG. 1 is a diagram showing an outline of a test support device 1 according to the present embodiment. The test support device 1 cooperates with a support service that realizes support to the user regarding the use of the analysis program 1P and a service that manages sales of the analysis program 1P, and supports the development and improvement of the analysis program 1P. The analysis program 1P includes an engine that executes a simulation calculation for physical phenomenon analysis. In simulation calculation, a huge number of steps are executed, physical phenomena are various, boundary conditions used in the calculation are also various, and an error is output from the analysis program 1P in the analysis computer that executes the analysis program 1P. It may be terminated or forced to terminate. When the calculation by the analysis program 1P does not go as intended by the user, the user makes an inquiry to the support service provider. The support service provider accepts the inquiry, records the received inquiry data, and provides the user with support on how to use the analysis program 1P based on the data. The test support device 1 efficiently and automatically executes the test of the analysis program 1P based on the inquiry to the support service provider to identify bugs or improve functions.

The analysis program 1P targeted by the test support device 1 of the present embodiment includes a physical phenomenon analysis engine. In this physical phenomenon analysis engine, reserved words (keywords) corresponding to preset parameter names, variable names, function names, predetermined values, error numbers, etc. are described by associating numerical values, logical values, etc. The input file IF is read and the processing is executed. The number of reserved words in the physical phenomenon analysis engine is enormous, and even a simple combination of reserved words is tens of thousands. Considering variations in parameter values (numerical value, logical value, etc.) for the reserved words, the total number is 100%. Testing is impossible. Therefore, the test support apparatus 1 according to the present embodiment selects the input file IF corresponding to the test case to be preferentially executed according to the priority of the reserved word determined based on the information regarding the support to the user. The test support device 1 executes the automatic script corresponding to the test case using the selected input file IF, and outputs the execution result to the storage device 3.

The test support device 1 is connected to a server device 2 and a storage device 3 managed by a support service provider. The test support device 1, the server device 2, and the storage device 3 are connected to the same local network LN. The server device 2 can send and receive information to and from the communication terminal device 5 used by the user via the local network LN and the public network N.

The server device 2 includes a function of a mail server that sends and receives a mail with the administrator of the support service as a sender or a recipient, and functions as an interface for inquiries from users. The server device 2 can store an inquiry mail about the analysis program 1P targeted by the support service in the storage device 3. The email includes time information such as the transmission time or the reception time. In addition, the server device 2 has a function of a web server and receives an inquiry about the analysis program 1P on the web page. The server device 2 stores the content of the inquiry for each user in the storage device 3 in association with the time information when the inquiry is received. Further, the server device 2 stores information about support regarding the analysis program 1P in the storage device 3. In the description of the first embodiment, the server device 2 will be described as a single server computer having the functions of a mail server and a Web server. The server device 2 may perform processing by being distributed to a plurality of server computers and storage media, and is one of a plurality of server computers (instances) virtually generated in one large computer. May be.

The storage device 3 is shared by support service providers. The storage device 3 stores the reserved words of the analysis program 1P. The reserved word is a keyword corresponding to a parameter name, variable name, function name, predetermined value, error number, etc. set for the analysis program 1P. The storage device 3 stores a plurality of patterns of a reserved word and a test input file IF in which numerical values, logical values, and the like of the reserved word are described in correspondence with the test case of the analysis program 1P according to the function of the analysis program 1P. .. The test input file IF may be automatically created by the test support device 1 or another device, or may be created by an operator using the test support device 1. ..

The test support device 1 stores a target analysis program 1P, and can automatically execute a test based on the analysis program 1P. The test support device 1 is used by an operator of a support service or a developer of the analysis program 1P, and as described above, using the information stored in the storage device 3, a test to be given priority (input corresponding to a test case). Information for selecting the file IF) is created. The test support device 1 automatically tests the analysis program 1P by using the created information with the test script corresponding to the test case. The process of creating information for selecting a test and the process of executing the test of the analysis program 1P may be performed by different devices.

The network N includes a public network. The public network is the so-called Internet. The public network may include an access point AP. The communication terminal device 5 used by the user can transmit information to the server device 2 via the network N.

FIG. 2 is a block diagram showing the configuration of the test support device 1. The server device 2 to which the test support device 1 is connected has the functions of the mail server 21 and the web server 22. The server device 2 stores the mail data received from the information terminal device used by the user by the mail server 21 and the content of the inquiry obtained by the Web server 22 in the storage device 3 as inquiry data.

The storage device 3 referred to by the test support device 1 stores information shared by support service providers using a large-capacity storage medium such as an SSD or a hard disk. Although the storage device 3 will be described as one storage device 3 for ease of description, the storage media of the plurality of storage devices 3 are virtually treated as one large-capacity storage medium. Good. The storage device 3 is readable and writable by the test support device 1 and the server device 2, which are communicatively connectable to the local network LN.

In the storage device 3, as described above, the reserved words set for the analysis program 1P, one or a plurality of reserved words, and the input file IF corresponding to the plurality of test cases used according to the design of the analysis program 1P are stored. Remembered The storage device 3 further stores a test execution history including the success or failure of the test executed for any of the plurality of stored test cases.

The storage device 3 stores an inquiry mail from the user regarding the analysis program 1P by the server device 2 or inquiry data including text describing the inquiry content received by the Web server 22. The storage device 3 stores a history of correspondence (visit, visit) for each user by the support service. The storage device 3 also stores contract information (number of contracts, contract history, contract version, etc.) for each user regarding the analysis program 1P obtained from the sales management system regarding the analysis program 1P.

The test support device 1 includes a control unit 10, a storage unit 11, a display unit 12, an operation unit 13, and a communication unit 14. Note that the test support apparatus 1 will be described below as one computer device for ease of description, but it may be a virtual machine using hardware resources of one or a plurality of computer devices.

The control unit 10 is a processor using a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit), and includes a built-in volatile memory, a clock, and the like. The control unit 10 can execute each process based on the analysis program 1P stored in the storage unit 11. Therefore, a processor whose specifications can withstand the computational load is used. Then, the control unit 10 executes a process of giving a priority to a reserved word to be prioritized, based on the test selection support program 100P stored in the storage unit 11.

The storage unit 11 stores the analysis program 1P using a nonvolatile storage medium such as an SSD (Solid State Drive) or a hard disk. The storage unit 11 stores a test selection support program 100P. The storage unit 11 stores the test script 1S corresponding to the test case of the analysis program 1P. The test script 1S is stored together with the test case in the storage device 3 in association with the test case identification information, and may be duplicated in the storage unit 11 by the control unit 10. The storage unit 11 stores data referenced by the control unit 10.

The display unit 12 includes a display device such as a liquid crystal panel or an organic EL display. The operation unit 13 is an interface that receives a user operation, and includes an input/output interface such as a keyboard and a mouse. The operation unit 13 may include a touch panel device with a built-in display.

The communication unit 14 is a communication module that realizes connection to the server device 2 and the storage device 3 via the local network LN. The communication unit 14 is specifically a network card, for example, and performs wired communication via the network LN.

The process of selecting the test case of the analysis program 1P by the test support device 1 in this way will be described with a specific example. The operator uses the operation unit 13 of the test support apparatus 1 to set a test selection support program 100P and an automatic test execution schedule. For example, the control unit 10 executes the following processing at a regular timing such as once a day, once a week, once a month, or at a timing when the operator gives an execution instruction. For each execution timing, the inquiry data, the correspondence history, and the test execution history read from the storage device 3 are limited to the inquiry data received during the target period from the previous execution time to the current execution time, Perform processing.

FIG. 3 is a flowchart showing an example of a processing procedure for selecting a test case. The control unit 10 of the test support apparatus 1 reads out the inquiry data of the target period from the storage device 3 (step S1), and uses the reserved word included in the inquiry data as user identification information (user identification information for identifying the user who sent the email or Web inquiry). It is extracted in association with the account) (step S2). The control unit 10 reads the correspondence history and the contract information from the storage device 3 for each user identification information (step S3).

The control unit 10 calculates, for each of the reserved words extracted in step S2, a value indicating the priority in which the importance of the email or the user who made the inquiry is reflected in the appearance frequency in the inquiry of the reserved word (step S4). ). The method of calculating the value indicating the priority will be described later.

The control unit 10 uses the value of the priority calculated for each reserved word, and calculates the comprehensive evaluation value for each of the test input files IF (test cases) described using the reserved word up to now. The calculation is performed in consideration of the test execution history (step S5). In step S5, the control unit 10 changes the comprehensive evaluation value depending on the success or failure of the test. For example, the control unit 10 may change the total evaluation value of the unexecuted test cases to be high. Since the regression test is so important that the test that succeeded the last time fails, the control unit 10 evaluates the comprehensive evaluation value of the test that the execution result of the previous time was successful to be high. To fluctuate.

The control unit 10 extracts the input files IF (test cases) by a predetermined number that can be implemented for each period in the descending order of the total evaluation value calculated in step S5 (step S6). The control unit 10 outputs test case identification information for identifying each of the extracted input files IF (step S7), and ends the processing. The output in step S7 may be output to a file or output to the display unit 12.

It is preferable that the test support apparatus 1 read the test script 1S based on the output test case identification information and automatically execute it, following the selection procedure shown in the flowchart of FIG. FIG. 4 is a flowchart showing an example of the automatic test execution procedure. The test support apparatus 1 may immediately execute the following processing after the selection processing shown in the flowchart of FIG. Preferably, when it is detected that the update (build) in the continuous improvement of the analysis program 1P is completed, the following process may be executed to execute the regression test.

The control unit 10 refers to the extracted test case identification information output by the procedure shown in the flowchart of FIG. 3 in order of priority (step S11). The control unit 10 uses the input file IF corresponding to the referred test case identification information and executes the test based on the corresponding test script 1S (step S12).

The control unit 10 outputs the success or failure of the test, which is the test execution result, as the test execution history (step S13). The test execution history may be stored as a file in the storage device 3 or may be output as a screen on the display unit 12 of the test support device 1.

The control unit 10 sequentially executes the processes of steps S11 to S13 for all or some of the extracted test cases.

Next, a method of calculating the priority for the test case will be described in detail. In the priority calculation method according to the present embodiment, items of elements that contribute to the priority are preset and stored in the storage unit 11. Further, a weight for the priority is set for each element and is similarly stored in the storage unit 11. Based on these items, the control unit 10 of the test support apparatus 1 reads the element from the storage device 3 for each reserved word or each user, assigns the set weight, and calculates the value indicating the priority. In the example described below, the control unit 10 sets the “occurrence frequency” of the reserved word, “the number of times of correspondence (the number of visits and visits)”, “the number of times of visits and visits” as the importance of the user of the sender of the email or inquiry using the reserved word Use the number of contract licenses. In addition, "contract version (whether premium or not)", "number of emails or inquiries", and the like may be used for the importance of the user.

FIG. 5 is a flowchart showing an example of a priority calculation method. The control unit 10 calculates the number of appearances of each of the reserved words, and extracts the reserved words of which the number of appearances is a predetermined number (at least "1") or more (step S401). The control unit 10 identifies, for each of the extracted reserved words, the user identification information of the user who is the sender of the mail or inquiry including the reserved word (step S402).

Next, the control unit 10 selects one piece of user identification information from all the contracted users (step S403). The control unit 10 reads, from the storage device 3, the “number of contract licenses” for the corresponding user or the organization to which the user belongs, for the selected user identification information (step S404). The control unit 10 reads the “correspondence number” in the target period for the selected user identification information from the “correspondence history” of the storage device 3 (step S405).

The control unit 10 determines whether or not the user identification information of all users under contract has been selected (step S406), and when it is determined that all users have not been selected (S406: NO), the process is performed. The process returns to S403 and the next user identification information is selected. When it is determined in step S406 that the user identification information of all users has been selected (S406: YES), the control unit 10 advances the process to the next step S407.

The control unit 10 selects one of the reserved words extracted in step S401 (step S407), and calculates the appearance frequency of the selected reserved word (the ratio of the number of appearances to the number of appearances of all reserved words) (step S408). ). The control unit 10 calculates the frequency calculated in step S408, the “number of contract licenses” read in step S404 for the user identification information specified in step S402 for the selected reserved word, and the “corresponding number of times” calculated in step S405. A value indicating the priority is calculated based on the ratio of “” to the whole and the weight value set for each (step S409). As a result, the priority is calculated.

The control unit 10 determines whether or not all the reserved words extracted in step S401 have been selected and executed (step S410), and when it is determined that all have not been executed (S410: NO). The control unit 10 returns the process to step S407.

When it is determined in step S410 that the processing has been executed for all the reserved words (S410: YES), the calculation of the value indicating the priority is terminated, and the control unit 10 advances the processing to step S5 shown in the flowchart of FIG. return.

As described above, the flowchart of FIG. 5 is only an example of calculating the priority, and if the test case including the important reserved word can be preferentially automatically executed in the inquiry from the customer who should prioritize, the other cases can be performed. Any method may be used. In a simple example, a hash table or the like is used to store information in the storage device 3 by a data structure that adds a user who has made an inquiry using a reserved word to a hash list using the reserved word as a key. In this case, the user's search (S402) or processing such as selection of a reserved word and search is unnecessary.

The calculation method shown in the flowchart of FIG. 5 will be described with a specific example. In the specific example, the three reserved words "*KEYWORD1", "*KEYWORD2", and "*KEYWORD3" that appear more than the specified number of times "1" in the target period are "USER1", "USER2", and "USER3". The priority that reflects the importance of the user having the user identification information (user name) is calculated.

The number of appearances of each reserved word is as follows, and the weight for "appearance frequency" is 0.5.
"*KEYWORD1": 3 times (once from "USER1", twice from "USER2")
"*KEYWORD2": Once (from "USER3" once)
"*KEYWORD3": 7 times (2 times from "USER1", 5 times from "USER3")

The “correspondence count” of each user is as follows, and the weight for the ratio of “correspondence count” is 0.25.
"USER1": 1 time, "USER2": 5 times, "USER3": 0 times

The “number of contract licenses” is as follows, and the weight for the ratio of “number of contract licenses” is 0.25.
"USER1": 1, "USER2": 1, "USER3": 2

Here, the weight for each element is designed so that the total is "1".

The control unit 10 calculates the appearance frequency of the first selected "*KEYWORD1" as follows (S408).
"*KEYWORD1": 3/(3+1+7)=3/11

In step S410, the control unit 10 calculates the ratio of the "corresponding number of times" for "USER1" and "USER2" for which an email including the selected "*KEYWORD1" or an inquiry is made as follows:
"USER1": 1/(1+5+0)=1/6
"USER2": 5/(1+5+0)=5/6
The ratio of the "number of contract licenses" is calculated as follows.
"USER1": 1/(1+1+2)=1/4
"USER2": 1/(1+1+2)=1/4

In step S410, the control unit 10 calculates a value indicating the priority for the selected "*KEYWORD1" as to "USER1" and "USER2" specified as using the "*KEYWORD1" as follows.
Value indicating priority = {Ratio of "number of responses" for "USER1" * 0.25
+ Ratio of "number of contract licenses" *0.25
+ "Appearance frequency" * 0.5}
+{Percentage of "corresponding frequency" for "USER2" *0.25
+ Ratio of "number of contract licenses" *0.25
+ "Appearance frequency" * 0.5}
= {(1/6)*0.25+(1/4)*0.25+(3/11)*0.5}
+{(5/6)*0.25+(1/4)*0.25+(3/11)*0.5}
= {0.0416+0.0625+0.13636}
+{0.2083+0.0625+0.13636}
= 0.64762

Next, the control unit 10 calculates the appearance frequency of the second selected "*KEYWORD2" as follows (S408).
"*KEYWORD2": 1/(3+1+7)=1/11

In step S410, the control unit 10 calculates, for the selected "*KEYWORD2", a value indicating the priority level for "USER3" specified as having been used as follows.
Value indicating priority = {Percentage of "response times" for "USER3" *0.25
+ Ratio of "number of contract licenses" *0.25
+ "Appearance frequency" * 0.5}
= {(0)*0.25+(2/4)*0.25+(1/11)*0.5}
= {0+0.125+0.04545}
= 0.17045

Next, the control unit 10 calculates the appearance frequency of the third selected "*KEYWORD3" as follows (S408).
"*KEYWORD1": 7/(3+1+7)=7/11
In step S410, the control unit 10 calculates a value indicating the priority for the selected "*KEYWORD3" as to "USER1" and "USER3" specified as having been used as follows.
Value indicating priority = {Ratio of "number of responses" for "USER1" * 0.25
+ Ratio of "number of contract licenses" *0.25
+ "Appearance frequency" * 0.5}
+{Percentage of "corresponding frequency" for "USER3" *0.25
+ Ratio of "number of contract licenses" *0.25
+ "Appearance frequency" * 0.5}
= {(1/6)*0.25+(1/4)*0.25+(7/11)*0.5}
+{(0)*0.25+(2/4)*0.25+(7/11)*0.5}
= {0.0416+0.0625+0.31818}
+{0+0.125+0.31818}
= 0.86546

Therefore, in step S4 in the flowchart of FIG. 3, a value indicating the priority is calculated for each reserved word as follows.
"*KEYWORD1": 0.64762
"*KEYWORD2": 0.17045
"*KEYWORD3": 0.86546

FIG. 6 is an explanatory diagram showing an example of a test case. As shown in FIG. 6, the input file IF of the test case "001" includes "*KEYWORD1" twice and "*KEYWORD2" once, and the input file IF of "002" indicates "*KEYWORD3" is included once. In step S5 in the flowchart of FIG. 3, the control unit 10 calculates respective comprehensive evaluation values for the input files IF of "001" and "002" as follows (S5). In this example, the control unit 10 adds the total evaluation value only for the types of reserved words included in the input file IF, and does not add the same for overlapping. Also, it is assumed that the test execution history has the same conditions.
Total evaluation value of "001" = value indicating the priority of "*KEYWORD1"
+ Value indicating the priority of "*KEYWORD2"
=0.64762+0.17045
= 0.81807
"002" comprehensive evaluation value = "*KEYWORD3" value indicating the priority
=0.64762+0.17045
= 0.86546

In this way, the control unit 10 calculates the overall evaluation value higher for the test case “002” than for the test case “001” (S5).

In this case, when only one test case is selected, the control unit 10 extracts the test case “002” (S6).

Note that the comprehensive evaluation value is calculated by the sum of the values indicating the priority in the above example. However, not limited to this, the correlation between the distribution of reserved words included in the test case and the reserved word group with high priority is calculated, and the value indicating the calculated correlation is used as the overall evaluation value. The test case may be selected according to.

As shown in the above example, in the present embodiment, the test case does not always preferentially select the function related to the changed portion and the test case using the reserved word. In the present embodiment, the test case regarding the reserved word that is frequently inquired by the user who should be prioritized is preferentially executed. As a result, rather than selecting and executing a test case based on the design or implementation, a test that benefits the user can be performed, and the quality improvement that satisfies the user can be realized. Further, in the case of a program incorporating another engine, such as the analysis program 1P in the present embodiment, the test case can be selected regardless of the mounted or changed portion, and therefore the engine design and mounted status are unknown. Even so, a valid test can be performed. In this way, the test support apparatus 1 according to the present disclosure can realize continuous improvement in accordance with the actual use of the software program.

(Modification 1)
The method of calculating the value indicating the priority of the reserved word is not limited to the procedure shown in the flowchart of FIG. 5, and various methods can be used by using the elements that contribute to the priority. FIG. 7 is a flowchart showing the method of calculating the priority in the first modification. The control unit 10 selects one piece of user identification information from all contracted users (step S421). The control unit 10 extracts, for the selected user identification information, the reserved word included in the mail or inquiry from the user (step S422). The control unit 10 calculates the appearance frequency in the mail or inquiry from the user selected for the extracted reserved word (step S423).

Next, with respect to the selected user identification information, the control unit 10 reads the “number of contract licenses” for the corresponding user or the organization such as the company to which the user belongs from the storage device 3 (step S424). The control unit 10 reads the “correspondence number” in the target period for the selected user identification information from the “correspondence history” of the storage device 3 (step S425). Needless to say, the information regarding the importance of the user used for the calculation is not limited to these.

The control unit 10 sets the appearance frequency calculated in step S423 for each of the reserved words used by the user extracted in step S422, the "number of contract licenses" read in step S424, and the "corresponding number" calculated in step S425. , A value relating to the priority of each reserved word for each user is calculated based on the weight value set for each (step S426). The control unit 10 temporarily stores the value regarding the priority of each reserved word for each user in association with the user identification information (step S427).

The control unit 10 determines whether or not the user identification information of all the users under contract has been selected (step S428), and when it is determined that all the users have not been selected (S428: NO), the process is performed. The process returns to S421 to select the next user identification information. When it is determined in step S428 that the user identification information of all users has been selected (S428: YES), the control unit 10 advances the process to the next step S429.

The control unit 10 selects one reserved word from the reserved words extracted in step S421 for all users, that is, a reserved word group that is extracted at least once out of all reserved words and has a predetermined number of appearances or more. One is selected (step S429). In step S429, the reserved words may be narrowed down by the number of appearances that is more than the predetermined number of times.

With respect to the reserved word selected in step S429, the user who sent the mail or the inquiry including the reserved word is specified (step S430), and the reserved word for each user temporarily stored in step S427 is specified for the specified user. The values relating to the priority are summed up, and the value indicating the priority is calculated (step S431).

The control unit 10 determines whether or not all the reserved words extracted for all users have been selected (step S432), and when it is determined that all the reserved words have not been selected (S432: NO), the process To step S429.

When it is determined in step S432 that all reserved words have been selected (S432: YES), the control unit 10 returns the process to step S5 shown in the flowchart of FIG.

The calculation example of the priority shown in the flowchart of FIG. 7 will be specifically described. The appearance conditions for the three reserved words "*KEYWORD1", "*KEYWORD2", and "*KEYWORD3" are the same as those in the above-mentioned example.

In the example shown in the flowchart of FIG. 7, the value indicating the priority for each reserved word can be expressed as follows.
Value indicating priority of reserved word = Sum of values of respective elements indicating priority calculated for each user who uses the reserved word

For example, regarding the value related to the priority of "KEYWORD1", the values related to the priority calculated for each user are totaled as follows,
"KEYWORD1" priority value = "USER1" {[appearance frequency] * 0.5 + "correspondence count" ratio * 0.25
+ Ratio of "number of contract licenses" *0.25}
+ For “USER2” {Ratio of “appearance frequency” * 0.5 + “number of times of correspondence” * 0.25
+ Ratio of "number of contract licenses" *0.25}
={(1/3)*0.5+(1/6)*0.25+(1/4)*0.25}
+{(2/2)*0.5+(5/6)*0.25+(1/4)*0.25}
=0.27077+0.7708
= 1.04157

Here, the "appearance frequency" is the appearance frequency during mail or inquiry for each user. For example, for "USER1", "*KEYWORD1" is once and "*KEYWORD3" is twice. Is.
"*KEYWORD1": 1/(1+0+2)=1/3
"*KEYWORD3": 2/(1+0+2)=2/3
Is.

Similarly, the value regarding the priority of "KEYWORD2" = "USER3" {{appearance frequency * 0.5 + proportion of "corresponding number of times" * 0.25
+ Ratio of "number of contract licenses" *0.25}
={(1/6)*0.5+(0)*0.25+(2/4)*0.25}
={0.0833+0+0.125}
= 0.2083

Similarly, the value of the priority of "KEYWORD3" = "USER1" {{appearance frequency * 0.5 + "correspondence count" ratio * 0.25
+ Ratio of "number of contract licenses" *0.25}
+ For “USER3” {“Appearance frequency” * 0.5 + “Count of correspondence” ratio * 0.25
+ Ratio of "number of contract licenses" *0.25}
={(2/3)*0.5+(1/6)*0.25+(1/4)*0.25}
+{(5/6)*0.5+(0)*0.25+(2/4)*0.25}
=
={“appearance frequency”*0.5+ratio of “corresponding frequency”*0.25
+ Ratio of "number of contract licenses" *0.25}
={(5/6)*0.5+(0)*0.25+(2/4)*0.25}
={0.43743+0.54167}
= 0.9791

When calculated by the processing procedure shown in the flowchart of FIG. 7, the priority of “KEYWORD1” is evaluated higher than that of “KEYWORD3”.

(Modification 2)
In the above-described example, the method of calculating the priority by using the mail, the inquiry, or the correspondence history in the target period is described below, but the history of the other target period is also used in the following calculation. FIG. 8 is a flowchart showing a method of calculating the priority in the second modification. In the processing procedure shown in the flowchart of FIG. 8, the same processing as the processing shown in the flowchart of FIG. 5 according to the first embodiment is denoted by the same step number, and detailed description thereof will be omitted.

After executing the processing from step S401 to S410, the control unit 10 stores the value indicating the priority at the time point t in the storage unit 11 in association with the time information (step S411). The control unit 10 reads the value stored in the storage unit 11 and indicating the priority for each reserved word calculated up to the previous time (step S412).

The control unit 10 multiplies the discount rate η by using the value indicating the priority stored in step S411 and the value indicating the priority calculated up to the previous time read in step S412, and the priority in consideration of the past value is calculated. A value indicating the degree is calculated (step S413). The control unit 10 returns the process to step S5 in the flowchart of FIG. 3 using the value calculated in step S413.

If the value indicating the priority at the time point t is Wt, the value M indicating the priority calculated in step S413 is calculated by the following discount sum reward.
M = Wt + ηWt-1 + η 2 Wt-2 + ... + η k Wt-k + ...
Note that Wt-k is a value W indicating the priority calculated k times before.

As a result, it becomes possible to calculate the priority for the reserved word that reflects the evaluation in the past, and it becomes possible to select the test case more efficiently.

(Modification 3)
The test support device 1 of the present embodiment basically inquires about reserved words and calculates the priority based on the frequency included in emails and the like. However, even if the reserved word is included frequently, the way of using the reserved word in the inquiry and the mail is different. For example, depending on the context, there should be differences such as being positive (thanks to the support service) or being negative (reporting a defect). In the third embodiment, the weight based on the context of the reserved word is reflected in the appearance frequency of the reserved word.

As for the weight based on the context, a learning model of natural language processing for the reserved words included in the mail data and the inquiry data stored in the storage device 3 is created in advance. The created learning model is stored in the storage device 3, and when the target period newly arrives, new mail data and inquiry data during the target period are given to the learning model and derived.

FIG. 9 is a diagram showing an example of the contents of the learning model 3L. The learning model 3L includes an input layer 301 for inputting text data which is mail data or inquiry data, and an output layer 302 for outputting a set of reserved words included in the text data and importance based on the context for the reserved words. And an intermediate layer 303. The intermediate layer 303 learns with the teacher data (data set) given to the reserved word included in the input text data whether the content is positive, negative, or intermediate. Already done. The output layer 302 may output only the importance, not the reserved word. As shown in FIG. 9, the node of the output layer 302 corresponds to each reserved word, and the weight for the reserved word corresponds to the importance.

FIG. 10 is a flowchart showing the method of calculating the priority in the third modification. Of the processing procedure shown in the flowchart of FIG. 10, steps common to the processing procedure shown in the flowchart of FIG. 7 are denoted by the same step numbers, and detailed description thereof will be omitted.

The control unit 10 sequentially reads the text data of the mail or inquiry from the user for the user identification information selected in step S421 from the storage device 3 (step S441) and gives it to the input layer 301 of the learning model 3L (step S442). .. The control unit 10 temporarily stores a set of the reserved word output from the output layer 302 of the learning model 3L and the importance of the reserved word (step S443). The processes of steps S442 and S443 are executed for the mail or inquiry from the selected user in the target period, and the reserved word included in the mail or inquiry of the user of the selected user identification information is extracted.

For each of the extracted reserved words, the control unit 10 uses the importance level output by the learning model 3L for each mail or inquiry, and calculates a value indicating the priority level based on the accumulation, average, etc. of the importance levels (step S444). ).

After that, the control unit 10 reads the "number of contract licenses" of the selected user (S424), reads the "corresponding number of times" from the storage device 3 (S425), and the like, and determines the importance of the user selected from these pieces of information. Is calculated and stored (step S445).

The control unit 10 identifies the user who used the reserved word for each selected reserved word (S430), and integrates the value indicating the priority calculated in step S444 with respect to the reserved word for each specified user. In this way, the comprehensive evaluation value is calculated (step S446). In step S446, the control unit 10 adds the weight of the importance of the user to the priority calculated for each user when integrating the comprehensive evaluation values.

As described above, even when the importance and the severity are different depending on the usage (context) of the reserved word, it is possible to reflect these pieces of information and select the test case that should be prioritized appropriately.

In the above-described present embodiment, the test case is selected by using the reserved word (keyword) set for the physical phenomenon analysis engine included in the analysis program 1P. However, the target program of the test selection support program 100P shown in the present embodiment is not limited to the analysis program 1P for physical phenomenon analysis. Since the test selection support program 100P determines the priority order for the reserved words related to the test to be executed with priority, the test case (corresponding input file IF) should be extracted by using the determined priority order. Kika can be changed depending on the design.

1 Test Support Device 10 Control Unit 1P Analysis Program 100P Test Selection Support Program 3 Storage Device 3L Learning Model

Claims (12)

Stores queries from different users, including any of a plurality of predetermined keywords corresponding to the software program,
Extracting any one of the predetermined plurality of keywords included in the inquiry from the user,
Based on the appearance frequency of the extracted keyword or the importance of the user who made the inquiry, a value indicating the priority for the extracted keyword is calculated,
A test support method for outputting the predetermined plurality of keywords in association with a value indicating the calculated priority. For each test case for the software program containing any of the predetermined plurality of keywords, calculate an evaluation value using a value indicating the priority calculated for the plurality of keywords,
The test support method according to claim 1, wherein the test cases are extracted in descending order of evaluation based on the calculated evaluation value. Execution history is stored in association with the test case,
The test support method according to claim 2, wherein the evaluation value is calculated as a high evaluation value for an unexecuted test case based on the execution history. The inquiry is stored in association with time information,
The value indicating the priority is calculated by multiplying the appearance frequency or importance for each different period by multiplying a component of an older period by a low coefficient, and adding the multiplied components. The test support method described in. The number of visits for each of the different users regarding the software program is stored,
The test support method according to claim 1, wherein the degree of importance of the user is calculated using the number of visits as an element. The number of contract licenses of the software program is stored for each of the different users,
The test support method according to claim 1, wherein the importance of the user is calculated by using the number of contract licenses as an element. The sales amount based on the contract of the software program is stored for each of the different users,
The test support method according to claim 1, wherein the degree of importance of the user is calculated using the sales amount as an element. The test support method according to claim 1, wherein the value indicating the priority is calculated by different weighting for each of the appearance frequency and a plurality of items that contribute to the importance of the user. .. An input layer that receives an inquiry from a user, an output layer that outputs the degree of importance for any of the predetermined plurality of keywords included in the inquiry, the degree of importance specified from the inquiry and the context of the inquiry Using a learning model including an intermediate layer trained based on teacher data including
When a new inquiry is received from the user, any one of the predetermined plurality of keywords included in the inquiry from the user is extracted,
Giving the query to the input layer of the learning model,
The value indicating the priority for the extracted keyword is calculated by using the importance for any of the predetermined plurality of keywords output from the output layer in response to the inquiry. The test support method according to any one of items. When the update completion of the software program is detected, the computer refers to the test cases extracted by the priority calculated for the plurality of keywords corresponding to the software program based on the inquiry from the user,
The computer is a test execution method for executing the referenced test case. A storage unit for storing inquiries from different users including any of a plurality of predetermined keywords corresponding to the software program;
An extraction unit that extracts any one of the predetermined plurality of keywords included in the inquiry from the user,
A calculation unit that calculates a value indicating the priority of the extracted keyword based on the appearance frequency of the extracted keyword or the importance of the user who made the inquiry.
A test support apparatus comprising: an output unit that outputs the plurality of predetermined keywords and a value indicating the calculated priority in association with each other. On the computer,
Record inquiries from different users, including any of the predefined keywords corresponding to the software program,
Extracting any one of the predetermined plurality of keywords included in the inquiry from the user,
Based on the appearance frequency of the extracted keyword or the importance of the user who made the inquiry, a value indicating the priority for the extracted keyword is calculated,
A computer program that executes a process of outputting the plurality of predetermined keywords and a value indicating the calculated priority in association with each other.

Images