CN112232071A - Multi-round dialogue script test method, device, equipment and storage medium - Google Patents
Multi-round dialogue script test method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112232071A CN112232071A CN202011141117.3A CN202011141117A CN112232071A CN 112232071 A CN112232071 A CN 112232071A CN 202011141117 A CN202011141117 A CN 202011141117A CN 112232071 A CN112232071 A CN 112232071A
- Authority
- CN
- China
- Prior art keywords
- script
- node
- request
- question
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the field of research and development management and discloses a method, a device, equipment and a storage medium for testing a multi-turn dialog script. The method comprises the following steps: generating an opening white request set based on a preset single request and analyzing a multi-turn dialog script to be tested to obtain a plurality of script nodes; taking out a new opening request, and initiating a multi-round dialogue script test by taking the opening request as a question; reading node information of a script node corresponding to the current question, if the script node corresponding to the current question is a non-ending node, constructing a question corresponding to a next-hop script node, and initiating a next multi-turn dialog script test based on the currently constructed question; and if the script node corresponding to the current question is the ending node, a new opening white request is taken out again, the opening white request is taken as the question, and a multi-round dialogue script test is initiated. The invention greatly reduces the complexity of script test, shortens the test period and reduces the test cost.
Description
Technical Field
The invention relates to the field of research and development management, in particular to a method, a device, equipment and a storage medium for testing a multi-turn dialog script.
Background
With the rapid development of artificial intelligence technology, the human-computer conversation demand in customer service business is higher and higher, and the human-computer conversation can not only reduce the labor cost, but also is not limited by the working time, thereby greatly improving the customer service level and quality of enterprises.
The existing man-machine conversation generally belongs to a multi-round conversation scene, the type scene generally has the characteristics of more scripts and more script nodes, meanwhile, the preparation of the script is complex, and the script skipping involves keyword analysis and the like, so that the script test of the multi-round conversation scene becomes complex. The scenario test of a multi-round session scene mainly comprises scenario node and jump branch tests, and the tests need to spend a lot of time to manually analyze jump conditions and construct requests, and then the tests are carried out branch by branch, so the test period is long, the cost is high, and the test risk is high.
Disclosure of Invention
The invention mainly aims to provide a multi-turn dialog script test method, a multi-turn dialog script test device, equipment and a storage medium, and aims to solve the technical problem that the script test of the existing multi-turn dialog scene is complex.
The invention provides a multi-round dialogue transcript testing method, which comprises the following steps:
s1, acquiring a preset single request, generating a field opening white request set based on the single request, and storing the field opening white request set into a field opening white request list;
s2, analyzing the multi-turn dialogue script to be tested to obtain a plurality of script nodes in the multi-turn dialogue script;
s3, taking a new opening request from the opening request list, and initiating a multi-turn dialog script test by taking the opening request as a question;
s4, reading the node information of the script node corresponding to the current question, and judging whether the script node corresponding to the current question is an end node;
s5, if the script node corresponding to the current question is a non-ending node, constructing a question corresponding to the next-hop script node according to the node information, initiating a next multi-turn dialog script test based on the currently constructed question, and jumping to the step S4 until the script node corresponding to the current question is an ending node;
and S6, if the script node corresponding to the current question is the ending node, jumping to the step S3 until the starting white request list is empty, and ending the multi-turn dialog script test.
Optionally, in a first implementation manner of the first aspect of the present invention, a data type of a request field in the single request includes: binary type, enumerated type, string type.
Optionally, in a second implementation manner of the first aspect of the present invention, the generating an open-field white request set based on the single request includes:
if the data type of the request field in the single request is a binary type, generating a multi-bit binary anti-gray code in a recursive mode, and taking the generated binary anti-gray code as an open field white request set;
if the data type of the request field in the single request is an enumeration type, traversing the cache corresponding to the request field by using a pseudo-random traversal strategy, randomly selecting a data to return after all data in the cache are traversed, and taking the minimum set of returned data after each traversal as an open field white request set;
and if the data type of the request field in the single request is the character string type, configuring scenario node jump logic corresponding to the request field, and taking the configured scenario node jump logic as an opening white request set.
Optionally, in a third implementation manner of the first aspect of the present invention, the analyzing a multi-turn dialog scenario to be tested to obtain a plurality of scenario nodes in the multi-turn dialog scenario includes:
analyzing a plurality of rounds of the opposite voice files in the YML format to be tested based on YAML grammar;
and reading all script nodes in the multi-round dialogue script file, and storing all script nodes into a script node list.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the reading node information of the scenario node corresponding to the current question, and determining whether the scenario node corresponding to the current question is an end node includes:
reading node information of a scenario node corresponding to the current question from the scenario node list, wherein the node information comprises node identification information, a next hop list and a hop condition;
and judging whether the script node corresponding to the current question is an ending node or not according to the node identification information.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the taking a new opening request from the opening request list, and initiating a multi-turn dialog test with the opening request as a question includes:
judging whether the field opening white request list is empty or not;
if the opening white request list is not empty, taking a new opening white request from the opening white request list;
generating a first question of a multi-turn dialogue according to a request keyword in the opening request;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and determining a script node corresponding to the current question according to the answer;
and if the opening white request list is empty, quitting the multi-turn dialogue script test.
Optionally, in a sixth implementation manner of the first aspect of the present invention, if the scenario node corresponding to the current question is a non-end node, the constructing a question corresponding to a next-hop scenario node according to the node information, and initiating a next multi-turn dialog scenario test based on the currently constructed question includes:
if the scenario node corresponding to the current question is a non-ending node, establishing a question corresponding to the scenario node from the current scenario node to the next hop according to the next hop list and the hop condition of the current scenario node;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and updating the node of the current script to complete a multi-turn dialog script test;
reading a next hop list and a hop condition of the current script node, and judging whether the next hop list of the current script node is empty or not;
if the next jump list of the current script node is not empty, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-round dialog script test according to the next jump list and the jump condition of the current script node;
and if the next jump list of the current script node is empty, returning to the script node corresponding to the previous multi-turn dialog script test, updating the current script node, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-turn dialog script test.
A second aspect of the present invention provides a multi-round dialog scenario testing apparatus, comprising:
the system comprises a field opening white generation module, a field opening white request list generation module and a field opening white generation module, wherein the field opening white generation module is used for acquiring a preset single request, generating a field opening white request set based on the single request and storing the field opening white request set into a field opening white request list;
the script analysis module is used for analyzing the multi-round dialogue scripts to be tested to obtain a plurality of script nodes in the multi-round dialogue scripts;
the first testing module is used for taking out a new opening request from the opening request list and initiating a multi-turn dialogue script test by taking the opening request as a question;
the ending node judging module is used for reading the node information of the script node corresponding to the current question and judging whether the script node corresponding to the current question is an ending node or not;
the second testing module is used for constructing a question corresponding to a next-hop script node according to the node information if the script node corresponding to the current question is a non-ending node, initiating a next multi-turn dialog script test based on the currently constructed question, and jumping to the ending node judging module until the script node corresponding to the current question is an ending node;
and the third testing module is used for jumping to the first testing module if the script node corresponding to the current question is the ending node, and ending the multi-turn dialogue script test until the starting white request list is empty.
Optionally, in a first implementation manner of the second aspect of the present invention, the data type of the request field in the single request includes: binary type, enumerated type, string type.
Optionally, in a second implementation manner of the second aspect of the present invention, the open-field white generating module is further configured to:
if the data type of the request field in the single request is a binary type, generating a multi-bit binary anti-gray code in a recursive mode, and taking the generated binary anti-gray code as an open field white request set;
if the data type of the request field in the single request is an enumeration type, traversing the cache corresponding to the request field by using a pseudo-random traversal strategy, randomly selecting a data to return after all data in the cache are traversed, and taking the minimum set of returned data after each traversal as an open field white request set;
and if the data type of the request field in the single request is the character string type, configuring scenario node jump logic corresponding to the request field, and taking the configured scenario node jump logic as an opening white request set.
Optionally, in a third implementation manner of the second aspect of the present invention, the script parsing module is specifically configured to:
analyzing a plurality of rounds of the opposite voice files in the YML format to be tested based on YAML grammar;
and reading all script nodes in the multi-round dialogue script file, and storing all script nodes into a script node list.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the end node determining module is configured to:
reading node information of a scenario node corresponding to the current question from the scenario node list, wherein the node information comprises node identification information, a next hop list and a hop condition;
and judging whether the script node corresponding to the current question is an ending node or not according to the node identification information.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the first test module is specifically configured to:
judging whether the field opening white request list is empty or not;
if the opening white request list is not empty, taking a new opening white request from the opening white request list;
generating a first question of a multi-turn dialogue according to a request keyword in the opening request;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and determining a script node corresponding to the current question according to the answer;
and if the opening white request list is empty, quitting the multi-turn dialogue script test.
Optionally, in a sixth implementation manner of the second aspect of the present invention, if the second test module is specifically configured to:
if the scenario node corresponding to the current question is a non-ending node, establishing a question corresponding to the scenario node from the current scenario node to the next hop according to the next hop list and the hop condition of the current scenario node;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and updating the node of the current script to complete a multi-turn dialog script test;
reading a next hop list and a hop condition of the current script node, and judging whether the next hop list of the current script node is empty or not;
if the next jump list of the current script node is not empty, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-round dialog script test according to the next jump list and the jump condition of the current script node;
and if the next jump list of the current script node is empty, returning to the script node corresponding to the previous multi-turn dialog script test, updating the current script node, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-turn dialog script test.
A third aspect of the present invention provides a multi-round dialog scenario testing apparatus, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the multi-turn dialog script testing apparatus to perform the multi-turn dialog script testing method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to perform the above-described multi-round dialog scenario testing method.
In the technical scheme provided by the invention, all nodes in the man-machine dialog script are generally in an incidence relation, and the test of the multi-turn man-machine dialog script mainly comprises the setting of the opening time and the setting of the jump request among all nodes. In this embodiment, the script request structure is based on script file parsing, so that the script test complexity is greatly reduced, the test period is shortened, and the test cost is reduced.
Drawings
FIG. 1 is a diagram of a first embodiment of a multi-turn dialog script testing method in an embodiment of the invention;
FIG. 2 is a diagram of a second embodiment of a multi-turn dialog script testing method in accordance with an embodiment of the present invention;
FIG. 3 is a diagram of a third embodiment of a multi-turn dialog script testing method in an embodiment of the invention;
FIG. 4 is a schematic diagram of an embodiment of a multi-turn dialog script testing apparatus according to an embodiment of the invention;
fig. 5 is a schematic diagram of an embodiment of a multi-turn dialog script testing apparatus in an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a multi-turn dialogue script test method, a multi-turn dialogue script test device, equipment and a storage medium. The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, a specific flow of an embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of a method for testing a multi-turn dialog script according to an embodiment of the present invention includes:
101. acquiring a preset single request, generating a field opening white request set based on the single request, and storing the field opening white request set into a field opening white request list;
it is to be understood that the execution subject of the present invention may be a multi-turn dialog script testing apparatus, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
In this embodiment, any of the scenarios of the human-computer interaction is triggered by an opening request, and multiple rounds of human-computer interaction are realized by circularly adopting a question-answer form. Different opening titles can enter different script nodes to carry out man-machine conversation, so that in order to realize comprehensive coverage test of multiple turns of dialogue scripts, a large number of different opening titles requests need to be constructed and tested one by one, the workload is high, and test omission exists. Therefore, the embodiment introduces a single-request self-variation mode and constructs a plurality of open-field white requests.
In this embodiment, a single request refers to one open-white request, that is, one open-white request is given in advance, and then a plurality of open-white requests are evolved based on the given open-white request. In this embodiment, different open-field white requests are constructed specifically by analyzing the data type of the request field. For example "do you ask XX how open the device? "the request field in the opening request is" XX device "and" how to open ", and therefore," XX device "may be replaced by another device, and" how to open "may be replaced by another function operation, such as" how to close "," how to adjust sound ", and the like.
Further, for the convenience of testing, the opening request set is stored in the opening request list, so that different opening requests are sequentially acquired to perform multiple rounds of dialog script testing.
Optionally, in a specific embodiment, the data type of the request field in the single request includes: binary type, enumerated type, string type.
(1) Binary type, e.g. identifying a certain attribute combination field of the user, 1011 for identifying the presence of 1, 3, 4 attributes but not 2 attributes of the user; 1001 is used to identify that the user has a 1, 4 attribute but not a 2, 3 attribute.
In this embodiment, for a request field represented by a binary system, it is preferable to generate an n-bit binary inverse gray code in a recursive manner, and construct request field data covering all scenes. For example, the request field is 1011, and 1 bit, 2 bits, 3 bits, and 4 bits can be constructed. . . The n-bit binary is used as a new request field.
And if the data type of the request field in the single request is a binary type, generating a multi-bit binary inverse gray code in a recursive mode, and taking the generated binary inverse gray code as an open field white request set.
(2) Enumerated types, such as those identifying user professional fields, the professional choices may be 0-7, only 1, such as professional field data of 1, or 2, or 3, etc.
In this embodiment, for a request field adopting an enumeration type, preferably, a pseudo-random traversal strategy is used to traverse a cache corresponding to the request field, and after all data in the cache is traversed, one data is randomly selected to be returned.
If the data type of the request field in the single request is an enumeration type, traversing the cache corresponding to the request field by using a pseudo-random traversal strategy, randomly selecting a data to return after all data in the cache are traversed, and taking the minimum set of the returned data after each traversal as an open field white request set.
(3) String type, such as user response field.
In this embodiment, a read configuration policy is used for the string field, and different values are set according to the conditions of branches that may be actually affected, such as FAQ standard questions, bibliographic branch training dialogues, and the like. The branch processing logic is specifically configured for customizing a project, and the branch jumping logic is configured with configuration contents comprising configuration files and partial codes.
And if the data type of the request field in the single request is the character string type, configuring script node skip logic corresponding to the request field, and taking the configured script node skip logic as an opening white request set.
102. Analyzing a multi-round dialogue script to be tested to obtain a plurality of script nodes in the multi-round dialogue script;
in this embodiment, before initiating a multi-turn dialog script test, the multi-turn dialog script needs to be analyzed first, so as to obtain a script node in the script file. The multi-turn dialog in the present embodiment preferably adopts the YML file format.
The YML file adopts an intuitive data serialization format which is written by YAML Language (YAML air Markup Language) and can be recognized by a computer, and is easy to read by human beings and interact with a script Language. All script nodes in the script file and node information of each script node can be obtained by analyzing the YML file.
103. Taking a new opening request from the opening request list, and initiating a multi-turn dialogue script test by taking the opening request as a question;
in this embodiment, a multi-turn dialog script test is initiated by taking an opening white request as a script test starting point and by means of questioning. A multi-turn dialog script typically has a plurality of script nodes, with each script node being associated with a jump condition. In addition, a multi-turn dialog typically has an end node as an identifier for the end dialog.
The multi-round dialogue script test mainly tests the jumping condition and the node covering condition of each node in the script. Inputting the current question into a man-machine dialogue question-answer model, determining a script node corresponding to the current question according to the question and an answer, and further reading node information corresponding to the script node from the parsed script node, wherein the node information comprises node identification information and can be used for identifying an ending node.
104. Reading node information of a script node corresponding to the current question, and judging whether the script node corresponding to the current question is an ending node or not;
in this embodiment, the first multi-round dialog test specifically uses the read field white request as a question, inputs the question into a human-computer dialog question-answer model, searches for an answer matching the field white request field by analyzing the field white request field, and outputs the answer, thereby completing the one multi-round dialog test. After all script nodes of the script file are analyzed, the script node corresponding to the current question can be determined according to the answer output by the man-machine dialogue question-answer model, and the node information of the script node is read.
In this embodiment, after completing a human-computer conversation test, the scenario node that makes the answer is further determined to determine whether the scenario node is an end node. And if the node of the next jump script has the end identifier, the multi-turn dialog is ended.
105. If the scenario node corresponding to the current question is a non-ending node, constructing a question corresponding to a next-jump scenario node according to the node information, initiating a next multi-turn dialog scenario test based on the currently constructed question, and jumping to the step 104 until the scenario node corresponding to the current question is an ending node;
in this embodiment, if the scenario node corresponding to the current question is not the end node, the multi-turn dialog may continue, but a question of the next dialog needs to be generated.
In this embodiment, the scenario file includes node information, such as a next hop list corresponding to a current scenario node and a corresponding hop condition. For example, the scenario node a may jump to B, C, D, the jump condition from the scenario node a to B is ab, the jump condition from the scenario node a to C is AC, and the jump condition from the scenario node a to C is AC, so that the scenario node may jump as long as the corresponding jump conditions ab, AC, and ad are satisfied, that is, one dialog may jump to the next dialog.
In this embodiment, in order to implement automatic jumping between scenario nodes and further implement the next man-machine conversation, a question corresponding to the next-hop scenario node needs to be constructed. In this embodiment, a question corresponding to the next-hop scenario node is specifically constructed according to the jump condition field corresponding to the next-hop scenario node. And after the questions are successfully constructed, initiating a next multi-turn dialogue script test based on the constructed questions, and then continuing to execute the step 104 until the script node corresponding to the current question is the ending node, and ending the multi-turn dialogue script test.
106. And if the scenario node corresponding to the current question is the ending node, jumping to step 103 until the starting white request list is empty to end the multi-turn dialogue scenario test.
In this embodiment, after the one-round multi-round dialog scenario test is finished, a new opening white is taken out from the opening white list again, and a new one-round multi-round dialog scenario test is initiated.
The nodes in the human-computer dialog script are usually in an incidence relation, and the test of the multi-turn human-computer dialog script mainly comprises the setting of the opening time and the setting of the jumping request between the nodes. In the embodiment, the script request structure is based on script file analysis, the script can be directly executed after the script is modified to generate a new script request, the script test complexity is greatly reduced, the test period is shortened, and the test cost is reduced.
Referring to fig. 2, a second embodiment of the method for testing a multi-turn dialog scenario according to the embodiment of the present invention includes:
201. acquiring a preset single request, generating a field opening white request set based on the single request, and storing the field opening white request set into a field opening white request list;
202. analyzing a plurality of rounds of the opposite voice files in the YML format to be tested based on YAML grammar;
203. reading all script nodes in the multi-round dialogue script file, and storing all script nodes into a script node list;
in this embodiment, the multi-turn dialog script preferably adopts a YML file format. The YML file adopts an intuitive data serialization format which is written by YAML Language (YAML air Markup Language) and can be recognized by a computer, and is easy to read by human beings and interact with a script Language.
In this embodiment, based on YAML syntax, all scenario nodes in the scenario file and node information of each scenario node may be obtained by parsing the scenario file in the YML file format. Since there are many scenario nodes and many jump branches of scenario nodes, it is preferable to store all scenario nodes in the scenario node list so as to be traversed circularly.
204. Taking a new opening request from the opening request list, and initiating a multi-turn dialogue script test by taking the opening request as a question;
205. reading node information of a scenario node corresponding to the current question from the scenario node list, wherein the node information comprises node identification information, a next hop list and a hop condition;
206. judging whether a script node corresponding to the current question is an ending node or not according to the node identification information;
in this embodiment, the node identification information is used to identify nodes so as to distinguish different nodes, the next hop list is used to have a next hop node set corresponding to the current node, for example, the scenario node a may jump to the scenario node B, C, D, the scenario node stored in the next hop list corresponding to the scenario node a is B, C, D, and the jump condition is a condition that the current scenario node jumps to the next hop scenario node. For example, if the jumping condition of the scenario node a to B is ab, the jumping condition of the scenario node a to C is AC, and the jumping condition of the scenario node a to C is AC, the scenario node jumps as long as the corresponding jumping conditions ab, AC, and ad are satisfied, that is, the scenario node jumps from one dialog to the next dialog is realized.
In this embodiment, the next jump list and the jump condition of the scenario node are preset and stored in the scenario file as the attribute of the scenario node, and the scenario node is analyzed to obtain the next jump list and the jump condition. In addition, an ending node is usually set in the scenario for ending a multi-turn dialog, so that the scenario node jump requires that the scenario node is a non-ending node and can be confirmed by judging the node identification information.
207. If the scenario node corresponding to the current question is a non-ending node, establishing a question corresponding to the scenario node from the current scenario node to the next hop according to the next hop list and the hop condition of the current scenario node;
208. inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and updating the node of the current script to complete a multi-turn dialog script test;
209. reading a next hop list and a hop condition of the current script node, and judging whether the next hop list of the current script node is empty or not;
210. if the next jump list of the current script node is not empty, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-round dialog script test according to the next jump list and the jump condition of the current script node;
in this embodiment, each scenario node corresponds to a next hop list, and therefore, before traversing scenario nodes in the scenario node list, the next hop list of each scenario node needs to be traversed. The one multi-round dialog scenario test initiated in this step is directed to the node traversal in the next hop list of the current scenario node, and step 208 needs to be continuously executed.
211. If the next hop list of the current script node is empty, returning to the script node corresponding to the previous multi-turn dialog script test, updating the current script node, continuously constructing a question corresponding to the node from the current script node to the next hop script node, initiating a multi-turn dialog script test, and jumping to the step 205 until the script node corresponding to the current question is an end node;
the one multi-round dialog scenario test initiated in this step is directed to the scenario node corresponding to the previous multi-round dialog scenario test of the current scenario node, that is, the traversal of the scenario node in the scenario node list, and step 208 needs to be continuously executed.
212. And if the scenario node corresponding to the current question is the ending node, jumping to step 204 until the starting white request list is empty to end the multi-turn dialogue scenario test.
In this embodiment, an iterative traversal mode is adopted to perform regression testing on each scenario node in the scenario file, where a scenario node from a current scenario node to a next jump scenario node is a jump branch of a human-computer conversation in the scenario, which may be regarded as an edge of a directed graph, and the scenario node may be regarded as a node of the graph, so that the scenario testing process may be regarded as an iterative traversal process of the directed graph.
In this embodiment, when performing a multi-round dialog test, the scenario nodes and the skip branches are further converted into corresponding graph structures, the skip branches are converted into a Start- > End format, and the converted graph structures are recorded so as to be used as a basis for judging whether a response meets an expected response and a basis for judging whether a test is sufficient, where a test target requires that all the nodes and branches be covered.
In this embodiment, the scenario test is converted into the iterative traversal of the graph, so that the comprehensive coverage test of each jump branch of the scenario node is realized. The script request construction is based on script file analysis, the script can be directly executed after the script is modified to generate a new script request, the script test complexity is greatly reduced, the test period is shortened, and the test cost is reduced.
Referring to fig. 3, a third embodiment of the method for testing a multi-turn dialog scenario according to the embodiment of the present invention includes:
301. acquiring a preset single request, generating a field opening white request set based on the single request, and storing the field opening white request set into a field opening white request list;
302. analyzing a multi-round dialogue script to be tested to obtain a plurality of script nodes in the multi-round dialogue script;
303. judging whether the field opening white request list is empty or not;
304. if the opening white request list is not empty, taking a new opening white request from the opening white request list; if the opening white request list is empty, quitting the multi-turn dialogue script test;
305. generating a first question of a multi-turn dialogue according to a request keyword in the opening request;
in this embodiment, the request keyword in the open-form request is a question keyword that the user needs to ask a question during the man-machine conversation, so that a question can be constructed by using the request keyword as the question keyword, for example, a question grammar such as a question sentence pattern and a question sentence pattern, and a question can be constructed by adding the question keyword. The question is the first question of a multi-turn conversation, as it is constructed by an open-field request.
306. Inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and determining a script node corresponding to the current question according to the answer;
in this embodiment, after constructing the first question of the multi-turn dialog, the current question is input into a preset question-answer model for recognition, where the question-answer model is also an FAQ model and needs to be trained in advance. After the questions are input into the question-answer model, the answers corresponding to the questions can be output through processing such as retrieval and matching, and the answers are associated with script nodes, so that the script nodes corresponding to the current questions can be determined according to the answers.
The scenario test needs to start with questioning, in the embodiment, a plurality of open space white requests are generated through self variation on the basis of a single request, then, questions with multiple rounds of conversations are generated according to the open space white requests, and the scenario test is initiated according to the questions, so that corresponding scenario nodes are determined, one-time test from questioning to answering, namely coverage test from questioning to scenario nodes, is completed, automation of the scenario test is realized, and the test efficiency is improved.
307. Reading node information of a script node corresponding to the current question, and judging whether the script node corresponding to the current question is an ending node or not;
308. if the scenario node corresponding to the current question is a non-ending node, constructing a question corresponding to a next-jump scenario node according to the node information, initiating a next multi-turn dialog scenario test based on the currently constructed question, and jumping to the step 307 until the scenario node corresponding to the current question is an ending node;
309. and if the scenario node corresponding to the current question is the ending node, jumping to step 303 until the starting white request list is empty, and ending the multi-turn dialogue scenario test.
In the above description of the method for testing a multi-turn dialog scenario in the embodiment of the present invention, referring to fig. 4, a device for testing a multi-turn dialog scenario in the embodiment of the present invention is described below, where one embodiment of the device for testing a multi-turn dialog scenario in the embodiment of the present invention includes:
the system comprises an open field white generation module 401, a open field white request list generation module and a closed field white generation module, wherein the open field white generation module 401 is used for acquiring a preset single request, generating an open field white request set based on the single request, and storing the open field white request set into an open field white request list;
a scenario analysis module 402, configured to analyze a multi-turn dialog scenario to be tested, to obtain a plurality of scenario nodes in the multi-turn dialog scenario;
a first testing module 403, configured to take a new opening request from the opening request list, and initiate a multi-turn dialog script test with the opening request as a question;
an end node determining module 404, configured to read node information of a scenario node corresponding to the current question, and determine whether the scenario node corresponding to the current question is an end node;
a second testing module 405, configured to construct a question corresponding to a next-hop script node according to the node information if the script node corresponding to the current question is a non-end node, initiate a next multi-turn dialog script test based on the currently constructed question, and jump to the end node determining module until the script node corresponding to the current question is an end node;
and the third testing module 406 is configured to jump to the first testing module if the scenario node corresponding to the current question is an end node, and end the multi-turn dialog scenario test until the opening white request list is empty.
Optionally, in a specific embodiment, the data type of the request field in the single request includes: binary type, enumerated type, string type.
Optionally, in a specific embodiment, the open-field white generating module 401 is further configured to:
if the data type of the request field in the single request is a binary type, generating a multi-bit binary anti-gray code in a recursive mode, and taking the generated binary anti-gray code as an open field white request set;
if the data type of the request field in the single request is an enumeration type, traversing the cache corresponding to the request field by using a pseudo-random traversal strategy, randomly selecting a data to return after all data in the cache are traversed, and taking the minimum set of returned data after each traversal as an open field white request set;
and if the data type of the request field in the single request is the character string type, configuring scenario node jump logic corresponding to the request field, and taking the configured scenario node jump logic as an opening white request set.
Optionally, in a specific embodiment, the script parsing module 402 is specifically configured to:
analyzing a plurality of rounds of the opposite voice files in the YML format to be tested based on YAML grammar;
and reading all script nodes in the multi-round dialogue script file, and storing all script nodes into a script node list.
Optionally, in a specific embodiment, the end node determining module 404 is configured to:
reading node information of a scenario node corresponding to the current question from the scenario node list, wherein the node information comprises node identification information, a next hop list and a hop condition;
and judging whether the script node corresponding to the current question is an ending node or not according to the node identification information.
Optionally, in a specific embodiment, the first testing module 403 is specifically configured to:
judging whether the field opening white request list is empty or not;
if the opening white request list is not empty, taking a new opening white request from the opening white request list;
generating a first question of a multi-turn dialogue according to a request keyword in the opening request;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and determining a script node corresponding to the current question according to the answer;
and if the opening white request list is empty, quitting the multi-turn dialogue script test.
Optionally, in an embodiment, if the second testing module 405 is specifically configured to:
if the scenario node corresponding to the current question is a non-ending node, establishing a question corresponding to the scenario node from the current scenario node to the next hop according to the next hop list and the hop condition of the current scenario node;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and updating the node of the current script to complete a multi-turn dialog script test;
reading a next hop list and a hop condition of the current script node, and judging whether the next hop list of the current script node is empty or not;
if the next jump list of the current script node is not empty, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-round dialog script test according to the next jump list and the jump condition of the current script node;
and if the next jump list of the current script node is empty, returning to the script node corresponding to the previous multi-turn dialog script test, updating the current script node, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-turn dialog script test.
The nodes in the human-computer dialog script are usually in an incidence relation, and the test of the multi-turn human-computer dialog script mainly comprises the setting of the opening time and the setting of the jumping request between the nodes. In the embodiment, the script request structure is based on script file analysis, the script can be directly executed after the script is modified to generate a new script request, the script test complexity is greatly reduced, the test period is shortened, and the test cost is reduced.
Fig. 4 above describes the multi-turn dialog scenario test apparatus in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the multi-turn dialog scenario test apparatus in the embodiment of the present invention is described in detail from the perspective of the hardware processing.
Fig. 5 is a schematic structural diagram of a multi-turn dialog script testing apparatus 500 according to an embodiment of the present invention, which may have relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, one or more storage media 530 (e.g., one or more mass storage devices) storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations for the multi-turn dialog testing device 500. Further, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the multi-session transcript testing device 500.
The multi-turn dialog testing device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows server, Mac OS X, Unix, Linux, FreeBSD, and so forth. Those skilled in the art will appreciate that the configuration of the multi-turn dialog script testing apparatus shown in fig. 5 does not constitute a limitation of the multi-turn dialog script testing apparatus and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
The present invention also provides a multi-turn dialog script testing apparatus, which includes a memory and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the multi-turn dialog script testing method in the above embodiments.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and which may also be a volatile computer readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the multi-round dialog script testing method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A multi-turn dialog script test method is characterized by comprising the following steps:
s1, acquiring a preset single request, generating a field opening white request set based on the single request, and storing the field opening white request set into a field opening white request list;
s2, analyzing the multi-turn dialogue script to be tested to obtain a plurality of script nodes in the multi-turn dialogue script;
s3, taking a new opening request from the opening request list, and initiating a multi-turn dialog script test by taking the opening request as a question;
s4, reading the node information of the script node corresponding to the current question, and judging whether the script node corresponding to the current question is an end node;
s5, if the script node corresponding to the current question is a non-ending node, constructing a question corresponding to the next-hop script node according to the node information, initiating a next multi-turn dialog script test based on the currently constructed question, and jumping to the step S4 until the script node corresponding to the current question is an ending node;
and S6, if the script node corresponding to the current question is the ending node, jumping to the step S3 until the starting white request list is empty, and ending the multi-turn dialog script test.
2. The method for multi-turn dialog testing of claim 1, wherein the data type of the request field in the single request comprises: binary type, enumerated type, string type.
3. The multi-turn dialog script testing method of claim 2, wherein generating the set of open white requests based on the single request comprises:
if the data type of the request field in the single request is a binary type, generating a multi-bit binary anti-gray code in a recursive mode, and taking the generated binary anti-gray code as an open field white request set;
if the data type of the request field in the single request is an enumeration type, traversing the cache corresponding to the request field by using a pseudo-random traversal strategy, randomly selecting a data to return after all data in the cache are traversed, and taking the minimum set of returned data after each traversal as an open field white request set;
and if the data type of the request field in the single request is the character string type, configuring scenario node jump logic corresponding to the request field, and taking the configured scenario node jump logic as an opening white request set.
4. The method for testing a multi-turn dialog scenario of claim 1, wherein the parsing the multi-turn dialog scenario to be tested to obtain a plurality of scenario nodes in the multi-turn dialog scenario comprises:
analyzing a plurality of rounds of the opposite voice files in the YML format to be tested based on YAML grammar;
and reading all script nodes in the multi-round dialogue script file, and storing all script nodes into a script node list.
5. The multi-turn dialog script testing method of claim 4, wherein reading node information of a script node corresponding to the current question and determining whether the script node corresponding to the current question is an end node comprises:
reading node information of a scenario node corresponding to the current question from the scenario node list, wherein the node information comprises node identification information, a next hop list and a hop condition;
and judging whether the script node corresponding to the current question is an ending node or not according to the node identification information.
6. A multi-turn dialog scenario test method according to claim 1, wherein the taking a new open-air request from the open-air request list and initiating a multi-turn dialog scenario test with the open-air request as a question comprises:
judging whether the field opening white request list is empty or not;
if the opening white request list is not empty, taking a new opening white request from the opening white request list;
generating a first question of a multi-turn dialogue according to a request keyword in the opening request;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and determining a script node corresponding to the current question according to the answer;
and if the opening white request list is empty, quitting the multi-turn dialogue script test.
7. The method for testing multi-turn dialog script of claim 5, wherein if the script node corresponding to the current question is a non-ending node, constructing the question corresponding to the next-hop script node according to the node information, and initiating the next multi-turn dialog script test based on the currently constructed question comprises:
if the scenario node corresponding to the current question is a non-ending node, establishing a question corresponding to the scenario node from the current scenario node to the next hop according to the next hop list and the hop condition of the current scenario node;
inputting the current question into a preset question-answer model for identification, outputting an answer corresponding to the current question, and updating the node of the current script to complete a multi-turn dialog script test;
reading a next hop list and a hop condition of the current script node, and judging whether the next hop list of the current script node is empty or not;
if the next jump list of the current script node is not empty, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-round dialog script test according to the next jump list and the jump condition of the current script node;
and if the next jump list of the current script node is empty, returning to the script node corresponding to the previous multi-turn dialog script test, updating the current script node, continuously constructing a question corresponding to the node from the current script node to the next jump script node and initiating a multi-turn dialog script test.
8. A multi-round dialog scenario test apparatus, characterized in that the multi-round dialog scenario test apparatus comprises:
the system comprises a field opening white generation module, a field opening white request list generation module and a field opening white generation module, wherein the field opening white generation module is used for acquiring a preset single request, generating a field opening white request set based on the single request and storing the field opening white request set into a field opening white request list;
the script analysis module is used for analyzing the multi-round dialogue scripts to be tested to obtain a plurality of script nodes in the multi-round dialogue scripts;
the first testing module is used for taking out a new opening request from the opening request list and initiating a multi-turn dialogue script test by taking the opening request as a question;
the ending node judging module is used for reading the node information of the script node corresponding to the current question and judging whether the script node corresponding to the current question is an ending node or not;
the second testing module is used for constructing a question corresponding to a next-hop script node according to the node information if the script node corresponding to the current question is a non-ending node, initiating a next multi-turn dialog script test based on the currently constructed question, and jumping to the ending node judging module until the script node corresponding to the current question is an ending node;
and the third testing module is used for jumping to the first testing module if the script node corresponding to the current question is the ending node, and ending the multi-turn dialogue script test until the starting white request list is empty.
9. A multi-turn dialog script testing apparatus, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invoking the instructions in the memory to cause the multi-turn dialog script testing apparatus to perform the multi-turn dialog script testing method of any of claims 1-7.
10. A computer readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement a multi-turn dialog testing method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011141117.3A CN112232071B (en) | 2020-10-22 | 2020-10-22 | Multi-round dialogue script test method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011141117.3A CN112232071B (en) | 2020-10-22 | 2020-10-22 | Multi-round dialogue script test method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112232071A true CN112232071A (en) | 2021-01-15 |
| CN112232071B CN112232071B (en) | 2022-07-15 |
Family
ID=74109236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011141117.3A Active CN112232071B (en) | 2020-10-22 | 2020-10-22 | Multi-round dialogue script test method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112232071B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114071236A (en) * | 2021-11-18 | 2022-02-18 | 平安普惠企业管理有限公司 | Open field white video acceleration method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2879062A2 (en) * | 2013-11-27 | 2015-06-03 | Akademia Gorniczo-Hutnicza im. Stanislawa Staszica w Krakowie | A system and a method for providing a dialog with a user |
| US20170300748A1 (en) * | 2015-04-02 | 2017-10-19 | Scripthop Llc | Screenplay content analysis engine and method |
| US20180004729A1 (en) * | 2016-06-29 | 2018-01-04 | Shenzhen Gowild Robotics Co., Ltd. | State machine based context-sensitive system for managing multi-round dialog |
| CN109977208A (en) * | 2019-03-22 | 2019-07-05 | 北京中科汇联科技股份有限公司 | It is a kind of to merge FAQ and task and the actively conversational system of guidance |
| CN111324718A (en) * | 2020-02-26 | 2020-06-23 | 网易（杭州）网络有限公司 | Conversation flow testing method and device, electronic equipment and readable storage medium |
| CN111488436A (en) * | 2019-01-29 | 2020-08-04 | 阿里巴巴集团控股有限公司 | Multi-turn dialogue model generation method, multi-turn dialogue model testing device and electronic equipment |
-
2020
- 2020-10-22 CN CN202011141117.3A patent/CN112232071B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2879062A2 (en) * | 2013-11-27 | 2015-06-03 | Akademia Gorniczo-Hutnicza im. Stanislawa Staszica w Krakowie | A system and a method for providing a dialog with a user |
| US20170300748A1 (en) * | 2015-04-02 | 2017-10-19 | Scripthop Llc | Screenplay content analysis engine and method |
| US20180004729A1 (en) * | 2016-06-29 | 2018-01-04 | Shenzhen Gowild Robotics Co., Ltd. | State machine based context-sensitive system for managing multi-round dialog |
| CN111488436A (en) * | 2019-01-29 | 2020-08-04 | 阿里巴巴集团控股有限公司 | Multi-turn dialogue model generation method, multi-turn dialogue model testing device and electronic equipment |
| CN109977208A (en) * | 2019-03-22 | 2019-07-05 | 北京中科汇联科技股份有限公司 | It is a kind of to merge FAQ and task and the actively conversational system of guidance |
| CN111324718A (en) * | 2020-02-26 | 2020-06-23 | 网易（杭州）网络有限公司 | Conversation flow testing method and device, electronic equipment and readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114071236A (en) * | 2021-11-18 | 2022-02-18 | 平安普惠企业管理有限公司 | Open field white video acceleration method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112232071B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110427331B (en) | Method for automatically generating performance test script based on interface test tool | |
| CN108829584B (en) | Service logic interface mock test method and system | |
| US8533664B2 (en) | Method and system to automatically generate GUI object addressing queries | |
| US20180075014A1 (en) | Conversational artificial intelligence system and method using advanced language elements | |
| CN102096631A (en) | Interface testing method and device | |
| EP1936607B1 (en) | Automated speech recognition application testing | |
| US10977161B2 (en) | Automatic intelligent cloud service testing tool | |
| CN106547520B (en) | Code path analysis method and device | |
| CN110109829B (en) | Intelligent dialogue automatic checking method and storage medium | |
| JP2018088242A (en) | Data processing apparatus, method, and program | |
| WO2021164189A1 (en) | Android application microservice automatic generation method driven by application scenario | |
| CN113986241B (en) | Configuration method and device of business rules based on knowledge graph | |
| CN113051172B (en) | Execution method and device of test script, computer equipment and storage medium | |
| CN112232071B (en) | Multi-round dialogue script test method, device, equipment and storage medium | |
| CN111488436A (en) | Multi-turn dialogue model generation method, multi-turn dialogue model testing device and electronic equipment | |
| CN111813675A (en) | SSA structure analysis method and device, electronic equipment and storage medium | |
| CN114238151A (en) | Software testing method and device, electronic equipment and storage medium | |
| CN111259212B (en) | Telemetering data interpretation method, device, equipment and storage medium | |
| CN112015648A (en) | Test method, device, computer equipment and medium based on automation script | |
| CN114416547A (en) | Test case based test method | |
| CN103701671A (en) | Method and device for detecting conflicts among businesses | |
| CN113821211B (en) | Command parsing method and device, storage medium and computer equipment | |
| CN110413518A (en) | The generation method and generating means and computer readable storage medium of control object library | |
| CN116974893A (en) | Test method, test device, test equipment and storage medium | |
| CN115577363A (en) | Detection method and device for deserialization utilization chain of malicious code |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |