CN111324718A - Conversation flow testing method and device, electronic equipment and readable storage medium - Google Patents
Conversation flow testing method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN111324718A CN111324718A CN202010120853.4A CN202010120853A CN111324718A CN 111324718 A CN111324718 A CN 111324718A CN 202010120853 A CN202010120853 A CN 202010120853A CN 111324718 A CN111324718 A CN 111324718A
- Authority
- CN
- China
- Prior art keywords
- node
- tested
- dialogue
- dialog
- link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides a conversation flow testing method, a conversation flow testing device, electronic equipment and a readable storage medium, wherein the conversation flow testing method comprises the following steps: acquiring a conversation full-flow chart matched with a conversation scene to be tested and semantics of each conversation node in the conversation full-flow chart; on the basis of the semantic relation between each dialogue node determined according to the semantics of each dialogue node, starting to search and traverse each dialogue node in the dialogue full-flow diagram from the initial node of the dialogue full-flow diagram to obtain a plurality of simplified dialogue links to be tested; and testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested. Therefore, the number of the dialogue links to be tested can be reduced under the condition of ensuring that all dialogue nodes in the dialogue full-flow diagram are traversed, so that the testing time of the dialogue full-flow diagram is reduced, and the testing efficiency is improved.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for testing a dialog flow, an electronic device, and a readable storage medium.
Background
With the development of scientific technology, artificial intelligence technology has been developed, and artificial intelligence includes many research directions, wherein conversational AI (conversational AI) refers to a series of AI technologies that machines and people exhibit in the process of simulating real conversation. There are various implementations of dialog flows, and how to efficiently test the actual effect of an edited dialog flow in the mainstream slot filling and dialog flow implementation of a dialog flow diagram is a problem to be solved in the industry.
At present, the test scheme for the session flow of the slot filling and session flow chart mainly includes that each session branch in the whole session flow chart is tested respectively, and session parameters are modified for each session branch for multiple times, so that redundant data and repeated tests are generated in the test process, and the test efficiency is low and the test cost is high.
Disclosure of Invention
In view of this, an object of the present application is to provide a method, an apparatus, an electronic device, and a readable storage medium for testing a dialog flow, in which a plurality of simplified dialog links to be tested are obtained according to a semantic relationship between each dialog node in a dialog full-flow graph, and the number of the dialog links to be tested can be reduced under the condition that all the dialog nodes in the dialog full-flow graph are guaranteed to be traversed, so as to reduce the testing time of the dialog full-flow graph, and help to improve the testing efficiency.
The embodiment of the present application further provides a dialog flow testing method, where the dialog flow testing method includes:
acquiring a conversation full-flow chart matched with a conversation scene to be tested and semantics of each conversation node in the conversation full-flow chart;
on the basis of the semantic relation between each dialogue node determined according to the semantics of each dialogue node, starting to search and traverse each dialogue node in the dialogue full-flow diagram from the initial node of the dialogue full-flow diagram to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested;
and testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested.
Further, before the obtaining of the dialog full-flow graph matched with the dialog scenario to be tested and the semantics of each dialog node in the dialog full-flow graph, the dialog flow testing method includes:
determining dialog scene information to be tested and scene test data corresponding to the dialog scene information to be tested;
and generating a conversation full-flow chart matched with the conversation scene to be tested based on the conversation scene information to be tested and the scene test data, and determining the test semantic data of each conversation node in the conversation full-flow chart.
Further, the scene test data comprises at least one of the following data:
flow chart data, dialog intent data, dialect data, and proper noun data.
Further, the searching and traversing, starting from the start node of the full dialog flow chart, each dialog node in the full dialog flow chart based on the semantic relationship between each dialog node determined according to the semantics of each dialog node to obtain a plurality of simplified dialog links to be tested, where leaf nodes in the full dialog flow chart only appear in one dialog link to be tested, including:
based on the semantic relation, starting node searching traversal in the full dialog flow chart from a root node of the full dialog flow chart to obtain a first dialog link to be tested and a plurality of marked first link nodes in the first dialog link to be tested;
based on the semantic relation, backtracking the conversation nodes in the conversation full-flow chart from the first leaf node of the first conversation link to be tested, and marking the unmarked nodes in the backtracking process to obtain at least one marked second link node;
obtaining at least one second dialogue link to be tested based on the at least one second link node and at least one first link node which is positioned on the same semantic link with the at least one second link node according to the semantic relation;
and according to the semantic relation, taking each second dialogue link to be tested as the first dialogue link to be tested, and backtracking and marking the dialogue nodes in the dialogue full-flow diagram from the first leaf node of the first dialogue link to be tested until all the dialogue nodes in the dialogue full-flow diagram are backtracked and marked to obtain the dialogue links to be tested, which comprise the first dialogue link to be tested and a plurality of second dialogue links to be tested, wherein the leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested.
Furthermore, the conversation full-flow graph comprises at least one conversation flow directed ring graph, and the conversation nodes in the conversation flow directed ring graph are located in the same conversation link to be tested.
Further, the testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested includes:
for each dialog link to be tested, modifying question parameters of question nodes in the dialog link to be tested according to the corresponding test semantic data;
and determining the question sentence of each question node from the root node in the dialogue link to be tested based on the question parameters of the question nodes in the dialogue link to be tested, and obtaining the reply sentence of the answer node corresponding to each question node in the dialogue link to be tested to the question sentence.
Further, after the question parameters based on the question nodes in the dialog link to be tested determine the question sentences of each question node from the root nodes in the dialog link to be tested and obtain the reply sentences of the answer nodes corresponding to each question node in the dialog link to be tested to the question sentences, the dialog flow testing method includes:
uploading a test result comprising a question sentence and a corresponding reply sentence to a cloud for storage, and sending the test result to a user after receiving a test result checking instruction.
An embodiment of the present application further provides a dialog flow testing apparatus, where the dialog flow testing apparatus includes:
the full-flow chart acquisition module is used for acquiring a conversation full-flow chart matched with a conversation scene to be tested and semantics of each conversation node in the conversation full-flow chart;
the dialogue link determination module is used for searching and traversing each dialogue node in the full dialogue flow diagram from a starting node of the full dialogue flow diagram based on the semantic relation between each dialogue node determined according to the semantics of each dialogue node to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the full dialogue flow diagram only appear in one dialogue link to be tested;
and the dialogue link testing module is used for testing the plurality of dialogue links to be tested according to the testing semantic data of each dialogue node in each dialogue link to be tested.
Further, the dialog flow testing apparatus further includes a full-flow determination module, where the full-flow determination module is configured to:
determining dialog scene information to be tested and scene test data corresponding to the dialog scene information to be tested;
and generating a conversation full-flow chart matched with the conversation scene to be tested based on the conversation scene information to be tested and the scene test data, and determining the test semantic data of each conversation node in the conversation full-flow chart.
Further, the scene test data comprises at least one of the following data:
flow chart data, dialog intent data, dialect data, and proper noun data.
Further, when the dialogue link determination module is configured to search and traverse each dialogue node in the full dialogue flow diagram from an initial node of the full dialogue flow diagram based on the semantic relationship between each dialogue node determined according to the semantics of each dialogue node, and obtain a plurality of simplified dialogue links to be tested, the dialogue link determination module is configured to:
based on the semantic relation, starting node searching traversal in the full dialog flow chart from a root node of the full dialog flow chart to obtain a first dialog link to be tested and a plurality of marked first link nodes in the first dialog link to be tested;
based on the semantic relation, backtracking the conversation nodes in the conversation full-flow chart from the first leaf node of the first conversation link to be tested, and marking the unmarked nodes in the backtracking process to obtain at least one marked second link node;
obtaining at least one second dialogue link to be tested based on the at least one second link node and at least one first link node which is positioned on the same semantic link with the at least one second link node according to the semantic relation;
and according to the semantic relation, taking each second dialogue link to be tested as the first dialogue link to be tested, and backtracking and marking the dialogue nodes in the dialogue full-flow diagram from the first leaf node of the first dialogue link to be tested until all the dialogue nodes in the dialogue full-flow diagram are backtracked and marked to obtain the dialogue links to be tested, which comprise the first dialogue link to be tested and a plurality of second dialogue links to be tested, wherein the leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested.
Furthermore, the conversation full-flow graph comprises at least one conversation flow directed ring graph, and the conversation nodes in the conversation flow directed ring graph are located in the same conversation link to be tested.
Further, when the session link testing module is configured to test the plurality of session links to be tested according to the test semantic data of each session node in each session link to be tested, the session link testing module is configured to:
for each dialog link to be tested, modifying question parameters of question nodes in the dialog link to be tested according to the corresponding test semantic data;
and determining the question sentence of each question node from the root node in the dialogue link to be tested based on the question parameters of the question nodes in the dialogue link to be tested, and obtaining the reply sentence of the answer node corresponding to each question node in the dialogue link to be tested to the question sentence.
Further, the dialog flow testing apparatus further includes a test result storage module, where the test result storage module is configured to:
uploading a test result comprising a question sentence and a corresponding reply sentence to a cloud for storage, and sending the test result to a user after receiving a test result checking instruction.
An embodiment of the present application further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the dialog flow test method as described above.
Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the dialog flow testing method as described above.
The embodiment of the application provides a conversation flow testing method, a conversation flow testing device, an electronic device and a readable storage medium, and the conversation full-flow chart matched with a conversation scene to be tested and the semantics of each conversation node in the conversation full-flow chart are obtained; on the basis of the semantic relation between each dialogue node determined according to the semantics of each dialogue node, starting to search and traverse each dialogue node in the dialogue full-flow diagram from the initial node of the dialogue full-flow diagram to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested; and testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested.
Therefore, by acquiring a conversation full-flow chart matched with a conversation scene to be tested and the semantics of each conversation node, determining the semantic relationship between the nodes according to the semantics of the conversation nodes, determining a plurality of simplified conversation links to be tested from the conversation full-flow chart based on the semantic relationship, and then testing each conversation link to be tested according to corresponding test semantic data, the number of the conversation links to be tested can be reduced under the condition of ensuring that all the conversation nodes in the conversation full-flow chart are traversed, so that the test time of the conversation full-flow chart is reduced, and the test efficiency is improved.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a flowchart of a method for testing a dialog flow according to an embodiment of the present application;
fig. 2 is a flowchart of a method for testing a dialog flow according to another embodiment of the present application;
FIG. 3 is a schematic diagram of a dialog flow diagram;
fig. 4 is a schematic structural diagram of a conversational flow testing apparatus according to an embodiment of the present disclosure;
fig. 5 is a second schematic structural diagram of a dialog flow testing apparatus according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. Every other embodiment that can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present application falls within the protection scope of the present application.
First, an application scenario to which the present application is applicable will be described. The application can be applied to the technical field of artificial intelligence. Conversational AI (conversational AI) refers to a series of AI techniques that machines and people exhibit in a process of simulating a real conversation, and it is necessary to give reasonable responses to users by comprehensively considering conversation context, the properties of the machines themselves, the properties of chat users, and even external information. The logic relation and the data relation which are required to be designed by the conversational AI are complex and many, and after the design process is finished, the testing process of the conversational flow is particularly important in order to ensure the serviceability and the practicability of the machine when the machine is on line.
Research shows that, in the present stage, a test scheme for a session flow of a slot filling and session flow chart mainly includes that each session branch in the whole session flow chart is tested, and session parameters are modified for each session branch for multiple times, so that redundant data and repeated tests are generated in the test process, and the test efficiency is low and the test cost is high.
Based on this, the embodiment of the present application provides a dialog flow testing method, which obtains a plurality of simplified dialog links to be tested according to a semantic relationship between each dialog node in a dialog full-flow diagram. The number of the dialogue links to be tested can be reduced under the condition of ensuring that all dialogue nodes in the dialogue full-flow diagram are traversed, so that the testing time of the dialogue full-flow diagram is reduced, and the testing efficiency is improved.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for testing a dialog flow according to an embodiment of the present disclosure. As shown in fig. 1, a method for testing a dialog flow provided in an embodiment of the present application includes:
In the step, a dialog full-flow chart matched with a dialog scene to be tested and semantics of each dialog node contained in the dialog full-flow chart are obtained.
Here, the dialog full-flow diagram includes an initial node, a trigger node, a reply node, and a merge node. The initial node is the beginning of the whole dialogue full-flow chart; trigger nodes represent related statements to be sent to the machine that trigger some intent; when the machine receives the statement and the intention is correctly identified, the content of the reply node is replied; the merge node has no specific meaning, and only plays a role in simplifying the conversation full-flow chart. The trigger nodes are most concerned in the dialog full-flow chart, the trigger nodes replace manual and automatic sending corresponding trigger statements to the machine, and receiving reply statements correspondingly returned by the machine, so that the dialog full-flow chart can be simplified, and the dialog full-flow chart only containing the trigger nodes and semantic relations among the trigger nodes is generated.
The conversation nodes of the conversation full-flow diagram can comprise a slot filling node and a judgment node, wherein the slot filling node is used for determining required parameters by a machine through question asking in a conversation process, and is noteworthy that whether the slot filling node executes question asking operation or not is determined according to a specific conversation process, if parameters required by the robot appear in a previous conversation, the slot filling node is reached without question asking, and if not, the slot filling node can ask the required parameters in a wrong conversation. The statements and the required parameters of the question back are in the dialogue flow basic linguistic data information analyzed in a Natural Language Understanding (NLU) mode in the slot filling node of the flow chart; the judgment node is introduced by reasonably replying the user by the machine according to the comprehensive consideration of the conversation context, the self attribute of the machine, the attribute of the chat user and even external information, and the subsequent branches of the conversation are not fixed one reply node any more, but different personalized services are required to be performed according to the self gender attribute of the machine.
Here, the semantics of each dialogue node refer to a statement of the control intention of each node to the operation of a machine or an evaluation of the response to the machine.
The dialog scenario refers to a language environment when a test user wants to test a machine, such as a command scenario for issuing a command to the machine to make the machine perform an action or a question-and-answer scenario for interacting with the machine at a question-and-answer basis.
And 102, on the basis of the semantic relationship between each dialogue node determined according to the semantics of each dialogue node, starting to search and traverse each dialogue node in the full dialogue flow diagram from the initial node of the full dialogue flow diagram to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the full dialogue flow diagram only appear in one dialogue link to be tested.
In the step, a semantic relation between each dialogue node in the semantics of each dialogue node is determined, each node in the dialogue full-flow diagram is traversed from an initial node of the dialogue full-flow diagram, and a plurality of simplified dialogue links to be tested are obtained, wherein all the dialogue nodes in the dialogue links to be tested comprise each dialogue node in the dialogue full-flow diagram.
Wherein leaf nodes in the dialog full-flow chart only appear in one dialog link to be tested. The out-degree of the leaf node is 0, that is, the leaf node has no child node, and the leaf node generally marks the end of the full flowchart.
Here, the semantic relationship between each pair of session nodes may refer to a semantic sequence between each session node, and for session nodes on different branches of the same session node, the relationship between them is a parallel relationship without mutual interference, and may be approximately considered that there is no semantic relationship. For example, for the trigger node of "singing", the two nodes of "good hearing" and "bad hearing" must appear after the trigger node of "singing", in terms of semantic relation, that is, only after the trigger node of "singing" is executed, the two nodes of "good hearing" and "bad hearing" will appear.
In the step, question data and reply data corresponding to each dialogue node are determined according to test semantic data corresponding to each dialogue node contained in each dialogue link to be tested, and each dialogue link to be tested is tested.
Thus, the test for each dialogue link needs to be performed from the root node, and the dialogue nodes are tested one by one according to the order of the dialogue nodes indicated by the semantic relationship between the dialogue nodes on the link.
The conversation process testing method provided by the embodiment of the application obtains a conversation full-flow chart matched with a conversation scene to be tested and semantics of each conversation node in the conversation full-flow chart; on the basis of the semantic relation between each dialogue node determined according to the semantics of each dialogue node, starting to search and traverse each dialogue node in the dialogue full-flow diagram from the initial node of the dialogue full-flow diagram to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested; and testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested.
Therefore, a dialog full-flow diagram matched with a dialog scene to be tested and the semantics of each dialog node in the dialog full-flow diagram are obtained, the semantic relation between the nodes is determined according to the semantics of each dialog node, a plurality of simplified dialog links to be tested are determined from the dialog full-flow diagram based on the semantic relation between the nodes, each dialog link to be tested is tested according to corresponding test semantic data, the number of the dialog links to be tested can be reduced under the condition that all the dialog nodes in the dialog full-flow diagram are guaranteed to be traversed, the test time of the dialog full-flow diagram is shortened, and the test efficiency is improved.
Referring to fig. 2, fig. 2 is a flowchart illustrating a method for testing a dialog flow according to another embodiment of the present application. As shown in fig. 2, a method for testing a dialog flow provided in an embodiment of the present application includes:
In the step, according to the semantic relationship between each dialogue node in the dialogue full-flow chart, node searching traversal is performed from a root node in the dialogue full-flow chart, nodes with more than one child node exist in the searching process, any one child node is traversed optionally until a leaf node is met, a first dialogue link to be tested is determined, each passed node is marked in the traversal process, and therefore after the first dialogue link to be tested is obtained, a plurality of first link nodes are correspondingly obtained.
Example 1, please refer to fig. 3, fig. 3 is a schematic diagram of a dialog flow chart, starting from a root node 1 "what you will" and recursing to a child node 6 "dancing" of the root node 1 and recursing to a child node 8 "goodbye" of the child node 6, at this time, the out-degree of the node 8 is found to be 0, that is, the node 8 is a leaf node, so that the node search traversal is finished, the first dialog link to be tested 1- >6- >8 is generated, and the node 1, the node 6, and the node 8 are marked.
Example 2, referring to fig. 3, starting from the root node 1 "what you will" and recursing to a child node 2 "sing a song" of the root node 1 and then recursing to a child node 4 "not good hearing" of the child node 4, it is found that the out-degree of the node 4 is 0, that is, the node 4 is a leaf node, so the node search traversal is finished, the first dialog link to be tested 1- >2- >4 is generated, and the node 1, the node 2, and the node 4 are marked.
And 203, based on the semantic relationship, backtracking the conversation nodes in the conversation full-flow chart from the first leaf node of the first conversation link to be tested, and marking the unmarked nodes in the backtracking process to obtain at least one marked second link node.
In this step, according to the semantic relationship, the session nodes in the full session flow chart are traced back from the leaf nodes of the first to-be-tested session link determined in step 202 in the direction opposite to the direction in which the traversal nodes of the first to-be-tested session link are formed, and in the tracing back process, nodes that are not marked are marked, so as to obtain at least one marked second link node.
Here, when tracing back from the first leaf node, there may be more than one child node for the first leaf node in addition to the first leaf node, one child node is arbitrarily selected for tracing back in the tracing back process, and traversal marking will not be performed for the nodes that have been marked.
Here, there are two cases for the end of traversal of the second link node, one is that traversal is again ended to the second leaf node, or when traversing to a second link node, backtrack is performed on the second link node, and it is found that all nodes semantically associated with the second link node are completely marked, at this time, traversal marking of the second link node is ended.
Here, when a reply or trigger node is not marked yet when backtracking, the node must not go to the leaf node with out-degree 0, either the node is on the ring or the leaf node has been marked. Therefore, when a certain reply or trigger node is not marked during backtracking, the session flow is considered to be dead end, the node and the path passed before are saved, and the nodes on the whole path are marked.
Corresponding to the above example 1, referring to fig. 3 at the same time, in the first dialog link to be tested 1- >6- >8, the finding sub-node 8 is a leaf node whose out-degree is 0, and the finding sub-node cannot continue recursion, and the tracing back to the node 6 from the first leaf node 8, the node 6 recurses to the node 7, and the marking node 7, because the node 8 is marked when the first dialog link to be tested is formed, in the process of the tracing back recursion, the node 7 will not recurse to the node 8, the tracing back to the node 3 from the node 7, and the marking node 3, at this time, the subsequent node 6 of the node 3 and the node 8 are both marked, so that no further recursive search is possible, and therefore, the tracing back can only find the node 1, the node 7, and the node 6 are all marked, and at this time, the process of the tracing back and marking the second link node is ended.
The full dialog flow diagram may include at least one dialog flow directed ring graph, the dialog nodes in the dialog flow directed ring graph are located in the same dialog link to be tested, please refer to fig. 3, the directed ring graph refers to that the node 3 starts from the node 7 and can return to the node 3 through the node 6, and at this time, the node 3, the node 7, and the node 6 are always in the same dialog link to be tested.
Corresponding to the above example 2, referring to fig. 3, in the first dialog link 1- >2- >4 to be tested, the finding child node 4 is a leaf node whose out-degree is 0, and cannot continue recursion, the node 2 is traced back from the first leaf node 4, and the node 2 is then recurred to the node 5, and the node 5 is marked, and at this time, the out-degree of the finding node 5 is 0, that is, the node 5 is a leaf node, and further recursive search cannot be performed, so that only tracing back is possible, the finding node 2, the node 1, and the node 4 are all marked, and at this time, the process of tracing back and marking the second link node is completed.
And step 204, obtaining at least one second dialogue link to be tested based on the at least one second link node and at least one first link node which is positioned on the same semantic link with the at least one second link node according to the semantic relation.
In this step, after at least one second link node is obtained in step 203, at least one first link node on the same link with the at least one second link node is supplemented into the at least one second link node according to the semantic relationship, and at least one second dialogue link to be tested is obtained according to the sequence of the link nodes displayed by the semantic relationship.
In order to ensure the semantic continuity and integrity of the second dialogue link to be tested, after at least one second link node is determined, the preorder first link node which is not traced back because of being marked in the tracing back process is added into the at least one second link node to form at least one second dialogue link to be tested with the at least one second link node.
Corresponding to the above example 1, referring to fig. 3, the second link node 3 and the second link node 7 are obtained in the backtracking process, and according to the semantic "joke" corresponding to the second link node 3 and the semantic "write poem" corresponding to the second link node 7, it can be known that, in the semantic relation, the "what you will" of the first link node 1 should be on the same semantic link as the second link node 3 and the second link node 7, and the first link node 6 in the backtracking process is added to obtain the second dialogue link to be tested 1- >6- >7- > 3.
Corresponding to the above example 2, referring to fig. 3, the second link node 5 is obtained in the backtracking process, and according to the semantic "good hearing" corresponding to the second link node 5, it can be known that, in the semantic relationship, the first link node 2 "singing" and the second link node 5 should be on the same semantic link, and then the root node 1 is added to obtain the second dialogue link 1- >2- >5 to be tested.
In the step, each obtained second to-be-tested dialogue link is used as the first to-be-tested dialogue link according to the semantic relation, and the backtracking and marking process is repeated from the first leaf node of the first to-be-tested dialogue link until all dialogue nodes in the full flow chart are marked, so that to-be-tested dialogue links including the first to-be-tested dialogue link and the plurality of second to-be-tested dialogue links are obtained, and the preparation work before testing the links is completed.
Here, when performing the cyclic backtracking and marking process, there is only one dialog link to be tested for each first leaf node, that is, when performing the backtracking from the first leaf node, the link where the first leaf node of the parent node corresponding to the first leaf node is located is not selected any more. The principle of traversing the labels is to locate each parent node where a child node exists and label each link where the child node exists.
Corresponding to example 1 and example 2, the dialog links to be tested are 1- >6- >8, 1- >2- >4, 1- >6- >7- >3 and 1- >2- >5, and it is guaranteed that all trigger nodes are traversed only once as much as possible to avoid redundancy.
And step 206, testing the plurality of dialogue links to be tested according to the test semantic data of each dialogue node in each dialogue link to be tested.
The descriptions of step 201 and step 206 may refer to the descriptions of step 101 and step 103, and the same technical effects can be achieved, which is not described in detail herein.
Further, before step 201, the method for testing dialog flow includes: determining dialog scene information to be tested and scene test data corresponding to the dialog scene information to be tested; and generating a conversation full-flow chart matched with the conversation scene to be tested based on the conversation scene information to be tested and the scene test data, and determining the test semantic data of each conversation node in the conversation full-flow chart.
In the step, scene information of a dialog to be tested and scene test data corresponding to the scene information of the dialog to be tested are determined, a plurality of test nodes and semantic relations among the nodes are determined according to the scene information of the dialog to be tested and the scene test data, a dialog full-flow chart matched with the dialog scene is generated, and test semantic data corresponding to each dialog node is determined.
Here, the scenario test data includes one of flowchart data, dialogue intention data, dialogue skill data, and proper noun data. And the scene test data is obtained from the database directly according to the ID of the dialog scene information to be tested when the test process is initiated. After acquiring corresponding flow chart data, conversation intention data, dialect data and proper noun data from a database, analyzing the flow chart data, the conversation intention data, the dialect data and the proper noun data, and initializing global parameter data such as machine parameters, user parameters and the like which are depended on by a conversation. After processing, the dialog intention data, the linguistic data and the proper noun data are sent to a Natural Language Understanding (NLU) service for algorithm model training, and the flow chart data is sent to a logic management module (LLP) service of the dialog flow for relevant training before the dialog starts.
Further, step 206 includes: for each dialog link to be tested, modifying question parameters of question nodes in the dialog link to be tested according to the corresponding test semantic data; and determining the question sentence of each question node from the root node in the dialogue link to be tested based on the question parameters of the question nodes in the dialogue link to be tested, and obtaining the reply sentence of the answer node corresponding to each question node in the dialogue link to be tested to the question sentence.
In the step, for each dialog link to be tested, according to test semantic data matched with the dialog scene information to be tested, a question parameter of each question node in the test dialog link is modified, according to the question parameter of each question node, traversal is started for a root node in each dialog link to be tested according to a node sequence in the dialog link to be tested, when the question node meets the question, a question statement is determined according to the question parameter, and a reply statement of a reply node associated with the question node is obtained.
Here, in the process of calculating the dialog full flowchart, a robot parameter list containing all parameter attributes is maintained in the corresponding test system, all parameter conditions depended on by each branch are counted simultaneously in the process of calculating the flowchart, and the calculation results of each branch are stored in a one-to-one correspondence manner simultaneously. Before the simulation session test process is performed, the global robot parameter information list is modified according to the parameter information, and then the modified global robot parameter information list is sent to a total Policy management module (Top-level Policy, TLP) of the session system to modify the corresponding session parameter data.
Here, for different question nodes, the processing modes are different, if a trigger node is encountered, the node data is analyzed to obtain the trigger intention information, the intention corpus data processed by the preprocessing and model training module is used to obtain the statement to be sent to the TLP, and then the statement is sent to the TLP service; if meeting the filling node, analyzing the content of the filling node, and correspondingly replying a corresponding proper noun according to a question-returning sentence; if a reply node is encountered, a reply statement returned by the TLP service is added to the conversation statement list, and at the moment, one pair of conversations of a certain conversation branch is tested.
Before computing the dialog full-flow chart, all proper nouns establish proper noun hash mapping by the names and synonyms, and then establish question-back sentence hash mapping by question-back sentences in the slot-filling nodes and the corresponding proper noun names. And when the conversation flow path is counted, the slot filling node is also saved. And traversing the slot filling nodes during the testing of the conversation process, judging the slot filling nodes according to the reply of the robot, if the reply of the robot is in the question-reversing sentence Hash mapping, taking the corresponding proper nouns in the proper noun Hash mapping and replying, and if the reply of the robot is not in the question-reversing sentence Hash mapping, skipping the slot filling nodes to continue the conversation.
Here, during the session flow test, corresponding session time statistics can be added, and the performance of the machine can be evaluated according to the session time statistics.
Further, after the question parameters based on the question nodes in the dialog link to be tested determine the question sentences of each question node from the root nodes in the dialog link to be tested and obtain the reply sentences of the answer nodes corresponding to each question node in the dialog link to be tested to the question sentences, the dialog flow testing method includes: uploading a test result comprising a question sentence and a corresponding reply sentence to a cloud for storage, and sending the test result to a user after receiving a test result checking instruction.
In the step, after the completion of all the test procedures of the conversation branches is determined, the test results of all the question sentences and the corresponding reply sentences are stored locally in a text form, then the test results are uploaded to a cloud for storage, and after a test result checking instruction of a user is received, the corresponding results are sent to the user.
When the test result is stored, the corresponding test dialogue scene and the test result are required to be stored correspondingly, after the test result checking instruction of the user is received, the test result checking instruction is analyzed to obtain the dialogue scene which needs to be checked by the user at this time, and the test result corresponding to the dialogue scene is sent to the user according to the dialogue scene.
The user can simultaneously check one or more test results corresponding to one test scenario or a plurality of test scenarios.
The conversation flow testing method provided by the embodiment of the application obtains a conversation full-flow chart matched with a conversation scene to be tested and semantics of each conversation node in the conversation full-flow chart; based on the semantic relation, starting node searching traversal in the full dialog flow chart from a root node of the full dialog flow chart to obtain a first dialog link to be tested and a plurality of marked first link nodes in the first dialog link to be tested; based on the semantic relation, backtracking the conversation nodes in the conversation full-flow chart from the first leaf node of the first conversation link to be tested, and marking the unmarked nodes in the backtracking process to obtain at least one marked second link node; obtaining at least one second dialogue link to be tested based on the at least one second link node and at least one first link node which is positioned on the same semantic link with the at least one second link node according to the semantic relation; according to the semantic relation, taking each second dialogue link to be tested as the first dialogue link to be tested, and backtracking and marking dialogue nodes in the dialogue full-flow diagram from a first leaf node of the first dialogue link to be tested until all dialogue nodes in the dialogue full-flow diagram are backtracked and marked, so as to obtain a dialogue link to be tested, which comprises the first dialogue link to be tested and a plurality of second dialogue links to be tested, wherein the leaf node in the dialogue full-flow diagram only appears in one dialogue link to be tested; and testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested.
Therefore, a dialog full-flow diagram matched with a dialog scene to be tested and the semantics of each dialog node in the dialog full-flow diagram are obtained, the semantic relation between the nodes is determined according to the semantics of each dialog node, a plurality of simplified dialog links to be tested are determined from the dialog full-flow diagram based on the semantic relation between the nodes, each dialog link to be tested is tested according to corresponding test semantic data, the number of the dialog links to be tested can be reduced under the condition that all the dialog nodes in the dialog full-flow diagram are guaranteed to be traversed, the test time of the dialog full-flow diagram is shortened, and the test efficiency is improved.
Referring to fig. 4 and 5, fig. 4 is a first schematic structural diagram of a conversational flow testing apparatus according to an embodiment of the present application, and fig. 5 is a second schematic structural diagram of a conversational flow testing apparatus according to an embodiment of the present application. As shown in fig. 4, the dialog flow test apparatus 400 includes:
and the full-flow diagram acquisition module 410 is configured to acquire a dialog full-flow diagram matched with the dialog scenario to be tested, and semantics of each dialog node in the dialog full-flow diagram.
The dialog link determining module 420 is configured to search and traverse each dialog node in the dialog full-flow graph from a start node of the dialog full-flow graph based on a semantic relationship between each dialog node determined according to the semantics of each dialog node, so as to obtain a plurality of simplified dialog links to be tested, where leaf nodes in the dialog full-flow graph only appear in one dialog link to be tested.
The session link testing module 430 is configured to test the session links to be tested according to the test semantic data of each session node in each session link to be tested.
Further, as shown in fig. 5, the dialog flow testing apparatus 400 further includes a full-flow determination module 440, where the full-flow determination module 440 is configured to:
determining dialog scene information to be tested and scene test data corresponding to the dialog scene information to be tested;
and generating a conversation full-flow chart matched with the conversation scene to be tested based on the conversation scene information to be tested and the scene test data, and determining the test semantic data of each conversation node in the conversation full-flow chart.
Further, the scene test data comprises at least one of the following data:
flow chart data, dialog intent data, dialect data, and proper noun data.
Further, when the dialog link determining module 420 is configured to search and traverse each dialog node in the dialog full-flow graph from the start node of the dialog full-flow graph based on the semantic relationship between each dialog node determined according to the semantics of each dialog node, so as to obtain a plurality of simplified dialog links to be tested, the dialog link determining module 420 is configured to:
based on the semantic relation, starting node searching traversal in the full dialog flow chart from a root node of the full dialog flow chart to obtain a first dialog link to be tested and a plurality of marked first link nodes in the first dialog link to be tested;
based on the semantic relation, backtracking the conversation nodes in the conversation full-flow chart from the first leaf node of the first conversation link to be tested, and marking the unmarked nodes in the backtracking process to obtain at least one marked second link node;
obtaining at least one second dialogue link to be tested based on the at least one second link node and at least one first link node which is positioned on the same semantic link with the at least one second link node according to the semantic relation;
and according to the semantic relation, taking each second dialogue link to be tested as the first dialogue link to be tested, and backtracking and marking the dialogue nodes in the dialogue full-flow diagram from the first leaf node of the first dialogue link to be tested until all the dialogue nodes in the dialogue full-flow diagram are backtracked and marked to obtain the dialogue links to be tested, which comprise the first dialogue link to be tested and a plurality of second dialogue links to be tested, wherein the leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested.
Furthermore, the conversation full-flow graph comprises at least one conversation flow directed ring graph, and the conversation nodes in the conversation flow directed ring graph are located in the same conversation link to be tested.
Further, when the dialog link testing module 430 is configured to test the dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested, the dialog link testing module 430 is configured to:
for each dialog link to be tested, modifying question parameters of question nodes in the dialog link to be tested according to the corresponding test semantic data;
and determining the question sentence of each question node from the root node in the dialogue link to be tested based on the question parameters of the question nodes in the dialogue link to be tested, and obtaining the reply sentence of the answer node corresponding to each question node in the dialogue link to be tested to the question sentence.
Further, as shown in fig. 5, the dialog flow testing apparatus 400 further includes a test result storage module 450, where the test result storage module 450 is configured to:
uploading a test result comprising a question sentence and a corresponding reply sentence to a cloud for storage, and sending the test result to a user after receiving a test result checking instruction.
The dialog flow testing device provided by the embodiment of the application acquires a dialog full-flow chart matched with a dialog scene to be tested and semantics of each dialog node in the dialog full-flow chart; on the basis of the semantic relation between each dialogue node determined according to the semantics of each dialogue node, starting to search and traverse each dialogue node in the dialogue full-flow diagram from the initial node of the dialogue full-flow diagram to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested; and testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested.
Therefore, a dialog full-flow diagram matched with a dialog scene to be tested and the semantics of each dialog node in the dialog full-flow diagram are obtained, the semantic relation between the nodes is determined according to the semantics of each dialog node, a plurality of simplified dialog links to be tested are determined from the dialog full-flow diagram based on the semantic relation between the nodes, each dialog link to be tested is tested according to corresponding test semantic data, the number of the dialog links to be tested can be reduced under the condition that all the dialog nodes in the dialog full-flow diagram are guaranteed to be traversed, the test time of the dialog full-flow diagram is shortened, and the test efficiency is improved.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 6, the electronic device 600 includes a processor 610, a memory 620, and a bus 630.
The memory 620 stores machine-readable instructions executable by the processor 610, when the electronic device 600 runs, the processor 610 communicates with the memory 620 through the bus 630, and when the machine-readable instructions are executed by the processor 610, the steps of the dialog flow testing method in the method embodiments shown in fig. 1 and fig. 2 may be executed.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the dialog flow testing method in the method embodiments shown in fig. 1 and fig. 2 may be executed.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A conversational flow test method, the conversational flow test comprising:
acquiring a conversation full-flow chart matched with a conversation scene to be tested and semantics of each conversation node in the conversation full-flow chart;
on the basis of the semantic relation between each dialogue node determined according to the semantics of each dialogue node, starting to search and traverse each dialogue node in the dialogue full-flow diagram from the initial node of the dialogue full-flow diagram to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested;
and testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested.
2. The dialog flow test method according to claim 1, wherein before the obtaining of the dialog full-flow graph matching the dialog scenario to be tested and the semantics of each dialog node in the dialog full-flow graph, the dialog flow test method comprises:
determining dialog scene information to be tested and scene test data corresponding to the dialog scene information to be tested;
and generating a conversation full-flow chart matched with the conversation scene to be tested based on the conversation scene information to be tested and the scene test data, and determining the test semantic data of each conversation node in the conversation full-flow chart.
3. The conversational flow test method of claim 2, wherein the scenario test data comprises at least one of:
flow chart data, dialog intent data, dialect data, and proper noun data.
4. The method for testing dialog flow according to claim 1, wherein the searching and traversing each dialog node in the dialog full-flow graph from a start node of the dialog full-flow graph based on the semantic relationship between each dialog node determined according to the semantics of each dialog node to obtain a plurality of simplified dialog links to be tested, wherein leaf nodes in the dialog full-flow graph only appear in one dialog link to be tested, includes:
based on the semantic relation, starting node searching traversal in the full dialog flow chart from a root node of the full dialog flow chart to obtain a first dialog link to be tested and a plurality of marked first link nodes in the first dialog link to be tested;
based on the semantic relation, backtracking the conversation nodes in the conversation full-flow chart from the first leaf node of the first conversation link to be tested, and marking the unmarked nodes in the backtracking process to obtain at least one marked second link node;
obtaining at least one second dialogue link to be tested based on the at least one second link node and at least one first link node which is positioned on the same semantic link with the at least one second link node according to the semantic relation;
and according to the semantic relation, taking each second dialogue link to be tested as the first dialogue link to be tested, and backtracking and marking the dialogue nodes in the dialogue full-flow diagram from the first leaf node of the first dialogue link to be tested until all the dialogue nodes in the dialogue full-flow diagram are backtracked and marked to obtain the dialogue links to be tested, which comprise the first dialogue link to be tested and a plurality of second dialogue links to be tested, wherein the leaf nodes in the dialogue full-flow diagram only appear in one dialogue link to be tested.
5. The method for testing conversation flow according to claim 1, wherein the conversation full flow graph comprises at least one conversation flow directed ring graph, and conversation nodes in the conversation flow directed ring graph are located in the same conversation link to be tested.
6. The method for testing dialog flow according to claim 1, wherein the testing the plurality of dialog links to be tested according to the test semantic data of each dialog node in each dialog link to be tested comprises:
for each dialog link to be tested, modifying question parameters of question nodes in the dialog link to be tested according to the corresponding test semantic data;
and determining the question sentence of each question node from the root node in the dialogue link to be tested based on the question parameters of the question nodes in the dialogue link to be tested, and obtaining the reply sentence of the answer node corresponding to each question node in the dialogue link to be tested to the question sentence.
7. The conversational flow testing method of claim 6, wherein after determining the question sentence of each question node from the root node in the conversational link to be tested based on the question parameters of the question nodes in the conversational link to be tested and obtaining the reply sentence to the question sentence of the answer node corresponding to each question node in the conversational link to be tested, the conversational flow testing method comprises:
uploading a test result comprising a question sentence and a corresponding reply sentence to a cloud for storage, and sending the test result to a user after receiving a test result checking instruction.
8. A conversational flow test apparatus, comprising:
the full-flow chart acquisition module is used for acquiring a conversation full-flow chart matched with a conversation scene to be tested and semantics of each conversation node in the conversation full-flow chart;
the dialogue link determination module is used for searching and traversing each dialogue node in the full dialogue flow diagram from a starting node of the full dialogue flow diagram based on the semantic relation between each dialogue node determined according to the semantics of each dialogue node to obtain a plurality of simplified dialogue links to be tested, wherein leaf nodes in the full dialogue flow diagram only appear in one dialogue link to be tested;
and the dialogue link testing module is used for testing the plurality of dialogue links to be tested according to the testing semantic data of each dialogue node in each dialogue link to be tested.
9. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the dialog flow test method of any of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the dialog flow test method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010120853.4A CN111324718B (en) | 2020-02-26 | 2020-02-26 | Session flow testing method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010120853.4A CN111324718B (en) | 2020-02-26 | 2020-02-26 | Session flow testing method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111324718A true CN111324718A (en) | 2020-06-23 |
| CN111324718B CN111324718B (en) | 2023-06-30 |
Family
ID=71173110
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010120853.4A Active CN111324718B (en) | 2020-02-26 | 2020-02-26 | Session flow testing method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111324718B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232071A (en) * | 2020-10-22 | 2021-01-15 | 中国平安人寿保险股份有限公司 | Multi-round dialogue script test method, device, equipment and storage medium |
| CN112685297A (en) * | 2020-12-25 | 2021-04-20 | 科讯嘉联信息技术有限公司 | Automatic testing method for telephone service robot dialect process |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216803A (en) * | 2008-01-09 | 2008-07-09 | 四川大学 | Test program control stream path set creation method based on base path |
| CN106155898A (en) * | 2015-04-16 | 2016-11-23 | 北京搜狗科技发展有限公司 | The method for obtaining path of a kind of flow chart and device |
| CN106371999A (en) * | 2016-10-20 | 2017-02-01 | 腾讯科技（深圳）有限公司 | Program code testing method and device |
| CN106959919A (en) * | 2016-01-08 | 2017-07-18 | 广州市动景计算机科技有限公司 | Method for testing software and device based on test path figure |
| CN108459967A (en) * | 2018-03-21 | 2018-08-28 | 东南大学 | Web application method for generating test case based on user interface state flow-chart |
-
2020
- 2020-02-26 CN CN202010120853.4A patent/CN111324718B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216803A (en) * | 2008-01-09 | 2008-07-09 | 四川大学 | Test program control stream path set creation method based on base path |
| CN106155898A (en) * | 2015-04-16 | 2016-11-23 | 北京搜狗科技发展有限公司 | The method for obtaining path of a kind of flow chart and device |
| CN106959919A (en) * | 2016-01-08 | 2017-07-18 | 广州市动景计算机科技有限公司 | Method for testing software and device based on test path figure |
| CN106371999A (en) * | 2016-10-20 | 2017-02-01 | 腾讯科技（深圳）有限公司 | Program code testing method and device |
| CN108459967A (en) * | 2018-03-21 | 2018-08-28 | 东南大学 | Web application method for generating test case based on user interface state flow-chart |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232071A (en) * | 2020-10-22 | 2021-01-15 | 中国平安人寿保险股份有限公司 | Multi-round dialogue script test method, device, equipment and storage medium |
| CN112685297A (en) * | 2020-12-25 | 2021-04-20 | 科讯嘉联信息技术有限公司 | Automatic testing method for telephone service robot dialect process |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111324718B (en) | 2023-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2555117A1 (en) | Code inspection executing system for performing a code inspection of abap source codes | |
| Fischbach et al. | Specmate: Automated creation of test cases from acceptance criteria | |
| CN106502896B (en) | A kind of generation method and device of function test code | |
| EP4322009A1 (en) | Test case generation method, apparatus and device | |
| Wood et al. | Detecting speech act types in developer question/answer conversations during bug repair | |
| Kervinen et al. | Model-based testing through a GUI | |
| CN111158656B (en) | Test code generation method and device based on fruit tree method | |
| CN111324718A (en) | Conversation flow testing method and device, electronic equipment and readable storage medium | |
| CN112286814A (en) | Automatic generation system and method of test case script | |
| EP4364044A1 (en) | Automated troubleshooter | |
| CN115803734A (en) | Natural language enrichment using action interpretation | |
| US11550703B2 (en) | Test package analyzer | |
| CN111143228B (en) | Test code generation method and device based on decision table method | |
| CN111143205B (en) | Android platform-oriented test case automatic generation method and generation system | |
| US20140325490A1 (en) | Classifying Source Code Using an Expertise Model | |
| CN115830419A (en) | Data-driven artificial intelligence technology evaluation system and method | |
| Storer et al. | Behave nicely! automatic generation of code for behaviour driven development test suites | |
| CN115658452A (en) | Buried point checking method, buried point checking device, readable storage medium and electronic equipment | |
| CN110647314B (en) | Skill generation method and device and electronic equipment | |
| CN113051262A (en) | Data quality inspection method, device, equipment and storage medium | |
| Duarte et al. | Extraction of probabilistic behaviour models based on contexts | |
| King et al. | Issues in natural language systems evaluation. | |
| US20230237275A1 (en) | Systems and methods for an end-to-end evaluation and testing framework for task-oriented dialog systems | |
| US20230116482A1 (en) | Self-creating, self-improving, and self-simulating artificial intelligence | |
| CN117369521B (en) | Method, device and equipment for generating behavior tree model path for unmanned aerial vehicle decision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |