CN112417109B - Method and device for testing man-machine dialogue system - Google Patents
Method and device for testing man-machine dialogue system Download PDFInfo
- Publication number
- CN112417109B CN112417109B CN202011155117.9A CN202011155117A CN112417109B CN 112417109 B CN112417109 B CN 112417109B CN 202011155117 A CN202011155117 A CN 202011155117A CN 112417109 B CN112417109 B CN 112417109B
- Authority
- CN
- China
- Prior art keywords
- standard
- determining
- tested
- voice
- return result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3343—Query execution using phonetics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
Abstract
The invention discloses a method and a device for testing a man-machine conversation system, comprising the following steps: inputting the voice instruction group into a standard system at least 2 times to obtain a standard return result; the voice instruction group comprises at least 2 voice instructions; determining a key information group corresponding to the voice instruction group by using the standard return result; inputting the voice instruction set into a system to be tested to obtain a return result to be tested; and determining a test result of the system to be tested according to the key information set and the return result to be tested. The voice instruction set is input into the standard system for multiple times to obtain a standard return result, the key information set is determined by utilizing the standard return result, so that the identification of the context in the multi-round dialogue is realized, the test result of the system to be tested is further determined by utilizing the key information set and the return result to be tested, and the test for the multi-round dialogue in the man-machine dialogue system is realized.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for testing a human-machine dialogue system.
Background
Man-machine conversation systems have been common in all types of intelligent terminals. And the user can control the intelligent terminal in a voice interaction mode by utilizing the man-machine interaction system to obtain the required feedback. In order to ensure the accuracy of the man-machine interactive system, the system version needs to be tested and checked again after being upgraded repeatedly; to avoid the negative impact of iterative upgrades on the performance inherent to the system.
The existing test mode can respectively re-amplify a large amount of historical voice instructions to the old version and the new version of man-machine dialogue system, and compare the returned results of the old version and the new version of the man-machine dialogue system, so that the test is realized. However, in multiple conversations, the meaning of the voice command needs to be confirmed by the context, so the voice command with the same content may represent different meanings and different returned results need to be correspondingly sent. Therefore, the above-described test method has a disadvantage in that the test cannot be performed for a plurality of rounds of dialogue.
Disclosure of Invention
The invention provides a method and a device for testing a man-machine conversation system, which are used for at least solving the technical problems in the prior art.
In a first aspect, the present invention provides a method for testing a man-machine conversation system, including:
inputting the voice instruction group into a standard system at least 2 times to obtain a standard return result; the voice instruction group comprises at least 2 voice instructions;
determining a key information group corresponding to the voice instruction group by using the standard return result;
inputting the voice instruction set into a system to be tested to obtain a return result to be tested;
and determining a test result of the system to be tested according to the key information set and the return result to be tested.
Preferably, the inputting the voice instruction set at least 2 times into a standard system to obtain a standard return result includes:
respectively obtaining corresponding single standard values of the voice instructions when the voice instructions are input into the standard system for each time;
and determining the standard return result according to each single standard value.
Preferably, the determining the key information group corresponding to the voice instruction group by using the standard return result includes:
each single standard value corresponding to the same voice command in the standard return result is determined as a comparison value group corresponding to the voice command;
determining key items corresponding to the voice instructions according to the comparison value group;
and determining the key information group according to the key item corresponding to each voice instruction.
Preferably, the determining, according to the comparison value set, the key item corresponding to the voice command includes:
and determining the same data item of each single standard value in the comparison value group, and determining the same data item as a key item corresponding to the voice instruction.
Preferably, the inputting the voice command set into the system to be tested to obtain the return result to be tested includes:
respectively obtaining corresponding single test values of each voice instruction when the voice instructions are input into the system to be tested;
and determining the return result to be tested according to each single test value.
Preferably, the determining, according to the key information set and the returned result to be tested, a test result of the system to be tested includes:
and when the single test value corresponding to any voice instruction does not comprise the key item corresponding to the voice instruction, determining the test result of the system to be tested as unqualified.
Preferably, the method further comprises:
and when the test result of the system to be tested is unqualified, adjusting the system to be tested.
In a second aspect, the present invention provides a test apparatus for a man-machine interaction system, including:
the standard return result determining module is used for inputting the voice instruction set into the standard system at least 2 times so as to obtain a standard return result; the voice instruction group comprises at least 2 voice instructions;
the key information group determining module is used for determining a key information group corresponding to the voice instruction group by utilizing the standard return result;
the to-be-tested return result determining module is used for inputting the voice instruction set into the to-be-tested system to obtain a to-be-tested return result;
and the test result determining module is used for determining the test result of the system to be tested according to the key information group and the return result to be tested.
In a third aspect, the present invention provides a computer readable storage medium storing a computer program for executing the method for testing a man-machine interaction system according to the present invention.
In a fourth aspect, the present invention provides an electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is used for reading the executable instructions from the memory and executing the instructions to realize the test method of the man-machine interaction system.
Compared with the prior art, the testing method and the testing device for the man-machine conversation system provided by the invention have the advantages that the voice instruction set is input into the standard system for multiple times to obtain the standard return result, the key information set is determined by utilizing the standard return result, so that the identification of the context in the multi-round conversation is realized, the testing result of the system to be tested is further determined by utilizing the key information set and the return result to be tested, and the testing of the man-machine conversation system for the multi-round conversation is realized.
Drawings
Fig. 1 is a flow chart of a testing method of a man-machine interaction system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a testing device of a man-machine interaction system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more comprehensible, the technical solutions according to the embodiments of the present invention will be clearly described in the following with reference to the accompanying drawings, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Summary of the application
In order to ensure the accuracy of the man-machine conversation system, the version of the system needs to be tested and checked again after being upgraded repeatedly; to avoid the negative impact of iterative upgrades on the performance inherent to the system. The existing test mode can respectively re-amplify a large amount of historical voice instructions to the old version and the new version of man-machine dialogue system, and compare the returned results of the old version and the new version of the man-machine dialogue system, so that the test is realized.
So-called "multi-turn conversations", like "talking" from person to person. The user sends out voice instructions to the man-machine dialogue system for many times, and each voice instruction has time sequence to form the context. The returned results of the human-machine dialog system have a direct relationship with the context. This results in that voice instructions of the same content may represent different meanings in different contexts, requiring corresponding different returned results.
For example, when the user issues two voice instructions in succession, "what is the tomorrow weather? "," Shanghai? ". Then the context-dependent knowledge is that the user's subsequent voice instruction "shanghai? The specific meaning of "what is the weather of the open sky Shanghai? ". When the user issues two voice commands in succession, "which country london belongs to? "," is there? ". Then the context-dependent knowledge is that the user's subsequent voice instruction "shanghai? The concrete meaning of "is" which country the Shanghai belongs to? ".
As can be seen from the above examples, the meaning of the voice instructions of the same content in different contexts is quite different. The correct return results of the human-computer interaction system are necessarily different. In the former context, the correct return should be: "open weather is weather/rain/yin". "in the latter context, the correct return should be: "Shanghai belongs to China. "
In the prior art, the test method for replaying the historical voice command, the voice command and the returned result (namely the question and the answer) are established. The recognition of the context cannot be achieved, and the 'correct answer' cannot be given in combination with the context adaptability. The prior art is not effective for the scenario of "same content with different meaning" and is not suitable for testing against multiple rounds of dialog.
Exemplary method
Therefore, the embodiment of the invention provides a testing method of a man-machine interaction system, so as to at least solve the technical problems in the prior art. As shown in fig. 1, the method in this embodiment includes the following steps:
step 101, inputting the voice instruction set into the standard system at least 2 times to obtain a standard return result.
In this embodiment, testing is performed primarily for multiple rounds of conversations. The voice command group includes at least 2 voice commands, and each voice command has a time sequence order to form a context.
Specifically, it is assumed that the voice command group includes 2 voice commands, which are "what is the tomorrow weather? "," Shanghai? ". It is evident that a context is formed between 2 voice commands, in which context the next voice command "shanghai? The specific meaning of "what is the weather of the open sky Shanghai? ".
Standard systems, typically human-machine dialog systems, are more stable in performance prior to iterative updating. That is, the return result of the standard system can be considered correct after the voice instruction is input. In this step, the voice command set is input into the standard system at least 2 times, and the standard return result is obtained.
In this embodiment, the same voice command set may be input to the standard system 2 times in different periods, and the corresponding return results may be obtained respectively, so as to be used together as the standard return results. Specifically, when each voice command is input into the standard system for each time, a corresponding single standard value can be obtained respectively; and determining a standard return result according to each single standard value. As shown in table 1.
Table 1:
as can be seen from table 1, each voice command is input 2 times into the standard system, and corresponding single standard values are obtained respectively at each input, i.e. 4 single standard values are obtained in total. Each single standard value includes 3 data items, namely time, range and content. "time" means the time at which the voice command was entered; "scope" means the dimension of information to which the voice instruction is directed; "content" means the specific information content in this dimension. For example, a first input of a voice command "what is the tomorrow weather? ", the single standard value obtained is: time: 15 days of 03 months, range: weather, namely weather is sunny. Obviously, the meaning is that the local weather is sunny for 3 months and 15 days.
The single standard values together form a standard return result. The standard return results mean that the voice instruction set of the multi-round dialog is "correct answer" in the context.
And 102, determining a key information group corresponding to the voice instruction group by using a standard return result.
The term "context" is a relatively abstract concept. In this embodiment, the voice instruction set is input into the standard system multiple times, and the purpose is to learn the corresponding regularity from the returned results of each time, so as to visualize the concept of "context" and solve the testing problem in the context. While the critical information set can be understood as a tangible representation of the "context".
In this step, first, each single standard value corresponding to the same voice command in the standard return result is determined as the comparison value group corresponding to the voice command. For the two voice commands, the corresponding control value sets are shown in tables 2 to 3. Wherein, table 2 is the voice instruction "what is the tomorrow weather? "corresponding control value group". Table 3 is the voice command "Shanghai? "corresponding control value group".
Table 2:
TABLE 3 Table 3
And then, performing comparison analysis according to the comparison value group so as to determine key items corresponding to the voice command. And according to the key items corresponding to the voice instructions, determining a key information group. The key term is the "rule" obtained by comparing each single standard value in the comparison value group. For example, according to the comparison value set, the key item corresponding to the voice command may be determined as follows: and determining the same data item of each single standard value in the comparison value group, and determining the same data item as a key item corresponding to the voice instruction. This means that in normal returns, other data items are allowed to change, except for key items.
As can be seen in conjunction with table 2, the voice command "what is the tomorrow weather? "the corresponding control value group includes 2 single standard values, each single standard value includes 3 data items. And only the range of the single standard value is: the weather "this data item is identical. That is, all returned results of the voice command are collectively directed to "weather". The control analysis may consider the meaning of the voice command to have a direct relationship to "weather". Then the "range: weather "what is this data item as a voice instruction" tomorrow weather? "corresponding key term".
Similarly, it can be seen in table 3 that the voice command "shanghai? In the corresponding control value group, the single standard value has only the range: the weather "this data item is identical. Then the "range: weather "this data item is the voice instruction" shanghai? "corresponding key term".
The key items corresponding to the voice commands together form a key information group. In this embodiment, the form of the key information set can be referred to as shown in table 4.
Table 4:
it should be noted that, according to the aforementioned visual voice instruction, "shanghai? The meaning of "is clearly influenced by the context, which in different contexts is entirely different. However, the context is fixed in the same voice command group, and the meaning of the voice command is not changed. However, the prior art does not recognize a specific context, so that the actual meaning of the voice command in the voice command group cannot be determined. Testing for multiple rounds of dialog is not possible.
For this case, the control value set shown in table 3 is first constructed in this embodiment, and it is compared with the commonalities of the returned results in each time under the specific context. Thereby determining the "range: weather "this data item is used as a key item. By key terms, the actual meaning of the voice command can be considered to be related to "weather". Therefore, the purpose of identifying the actual meaning of the voice instruction in the context is achieved, and conditions are provided for realizing the test for multiple rounds of conversations.
Step 103, inputting the voice instruction set into the system to be tested to obtain the return result to be tested.
The system to be tested can be a man-machine dialogue system after iterative upgrade, namely a target for testing. In this embodiment, the voice command set may be input into the system to be tested, so as to obtain the corresponding single test value of each voice command when the voice command is input into the system to be tested. And determining a return result to be tested according to each single test value. The contents of the returned results to be measured are shown in table 5.
Table 5:
and 104, determining a test result of the system to be tested according to the key information group and the return result to be tested.
It is known that in each normal return, other data items are allowed to change, except for key items. It is also stated that in each single test value of the return result to be tested, the corresponding key term should also be included. Otherwise, if any single test value does not include the corresponding key item, the error of the single test value is indicated, namely, the return result of the system to be tested is inaccurate. Therefore, in this embodiment, when the single test value corresponding to any voice command does not include the key item corresponding to the voice command, the test result of the system to be tested is determined as unqualified.
As can be seen from the specific comparison of tables 4 and 5, in the return results to be tested, the voice command "what is the tomorrow weather? Included in the single test values of "key term" range: weather. And the correct return result of the system to be tested to the voice instruction is indicated. The voice instruction "shanghai? The "single test value" does not include the key term "range: weather ", which indicates that the return result of the system to be tested to the voice command is wrong. Because of the single test value error corresponding to one voice command, the test result of the system to be tested can be determined as unqualified.
In addition, in this embodiment, preferably, when the test result of the system to be tested is failed, the system to be tested may be readjusted, and the test may be performed circularly according to the above method until the test result is failed, or other specific conditions are reached, so that the test procedure may be ended.
According to the technical scheme, the beneficial effects of the embodiment are as follows: the voice instruction set is input into the standard system for multiple times to obtain a standard return result, the standard return result is utilized to determine the key information set so as to realize the identification of the context in the multi-round dialogue, and then the key information set and the return result to be tested are utilized to determine the test result of the system to be tested, so that the test for the multi-round dialogue in the man-machine dialogue system is realized.
Exemplary apparatus
Fig. 2 shows a specific embodiment of a testing device of the man-machine interaction system according to the present invention. The apparatus in this embodiment is a physical apparatus for performing the method of fig. 1. The technical solution is essentially identical to the above embodiment, and the corresponding description in the above embodiment is also applicable to this embodiment. The device in this embodiment includes:
a standard return result determining module 201, configured to input the voice command set at least 2 times into the standard system to obtain a standard return result; the voice command group comprises at least 2 voice commands.
The key information set determining module 202 is configured to determine a key information set corresponding to the voice command set by using the standard return result.
The to-be-tested return result determining module 203 is configured to input a voice command set into the to-be-tested system to obtain a to-be-tested return result.
And the test result determining module 204 is configured to determine a test result of the system to be tested according to the key information set and the return result to be tested.
Further, based on the embodiment shown in fig. 2, it is preferable to further include:
the criteria return result determination module 201 includes:
and the single standard value topping unit 211 is configured to obtain a single standard value corresponding to each voice command when each voice command is input into the standard system.
The standard return result determining unit 212 is configured to determine a standard return result according to each single standard value.
The key information set determination module 202 includes:
the comparison value group determining unit 221 is configured to determine, as a comparison value group corresponding to the voice command, each single standard value corresponding to the same voice command in the standard return result.
The key item determining unit 222 is configured to determine a key item corresponding to the voice command according to the comparison value set.
The key information set determining unit 223 is configured to determine a key information set according to the key item corresponding to each voice command.
The return result determination module 203 to be tested includes:
the single test value determining unit 231 is configured to obtain corresponding single test values of each voice command when the voice command is input into the system under test.
And the to-be-tested return result determining unit 232 is configured to determine a to-be-tested return result according to each single test value.
The apparatus further comprises:
and the adjusting module 205 is configured to adjust the system under test when the test result of the system under test is failed.
Exemplary computer program product and computer readable storage Medium
In addition to the methods and apparatus described above, embodiments of the invention may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform steps in a method according to various embodiments of the invention described in the "exemplary methods" section of this specification.
The computer program product may write program code for performing operations of embodiments of the present invention in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the invention may also be a computer-readable storage medium, having stored thereon computer program instructions, which when executed by a processor, cause the processor to perform steps in a method according to various embodiments of the invention described in the "exemplary method" section of the description above.
The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The basic principles of the present invention have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present invention are merely examples and not intended to be limiting, and these advantages, benefits, effects, etc. are not to be considered as essential to the various embodiments of the present invention. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the invention is not necessarily limited to practice with the above described specific details.
The block diagrams of the devices, apparatuses, devices, systems referred to in the present invention are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.
It is also noted that in the apparatus, devices and methods of the present invention, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent aspects of the present invention.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the invention. Thus, the present invention is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the invention to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.
Claims (7)
1. A method for testing a man-machine interactive system, comprising:
inputting the voice instruction group into a standard system at least 2 times to obtain a standard return result; the voice instruction group comprises at least 2 voice instructions;
determining a key information group corresponding to the voice instruction group by using the standard return result;
inputting the voice instruction set into a system to be tested to obtain a return result to be tested;
determining a test result of the system to be tested according to the key information set and the return result to be tested;
respectively obtaining corresponding single standard values of the voice instructions when the voice instructions are input into the standard system for each time;
determining a standard return result according to each single standard value;
each single standard value corresponding to the same voice command in the standard return result is determined as a comparison value group corresponding to the voice command;
determining key items corresponding to the voice instructions according to the comparison value group;
determining the key information group according to the key item corresponding to each voice instruction;
and determining the same data item of each single standard value in the comparison value group, and determining the same data item as a key item corresponding to the voice instruction.
2. The method of claim 1, wherein inputting the set of voice instructions into a system under test to obtain a return result under test comprises:
respectively obtaining corresponding single test values of each voice instruction when the voice instructions are input into the system to be tested;
and determining the return result to be tested according to each single test value.
3. The method of claim 2, wherein determining the test result of the system under test based on the set of key information and the returned result under test comprises:
and when the single test value corresponding to any voice instruction does not comprise the key item corresponding to the voice instruction, determining the test result of the system to be tested as unqualified.
4. A method according to any one of claims 1 to 3, further comprising:
and when the test result of the system to be tested is unqualified, adjusting the system to be tested.
5. A test device for a man-machine conversation system, comprising:
the standard return result determining module is used for inputting the voice instruction group into the standard system at least 2 times so as to obtain a standard return result; the voice instruction group comprises at least 2 voice instructions;
the key information group determining module is used for determining a key information group corresponding to the voice instruction group by utilizing the standard return result;
the to-be-tested return result determining module is used for inputting the voice instruction set into the to-be-tested system to obtain a to-be-tested return result;
the test result determining module is used for determining a test result of the system to be tested according to the key information group and the return result to be tested;
the standard return result determining module is further used for respectively obtaining corresponding single standard values of the voice instructions when the voice instructions are input into the standard system each time; determining a standard return result according to each single standard value;
the key information group determining module is further configured to determine, from the standard return result, each single standard value corresponding to the same voice command as a comparison value group corresponding to the voice command; determining key items corresponding to the voice instructions according to the comparison value group; determining the key information group according to the key item corresponding to each voice instruction;
and determining the same data item of each single standard value in the comparison value group, and determining the same data item as a key item corresponding to the voice instruction.
6. A computer readable storage medium storing a computer program for executing the method of testing a man-machine conversation system as claimed in any one of the preceding claims 1 to 4.
7. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for testing a man-machine interaction system according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011155117.9A CN112417109B (en) | 2020-10-26 | 2020-10-26 | Method and device for testing man-machine dialogue system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011155117.9A CN112417109B (en) | 2020-10-26 | 2020-10-26 | Method and device for testing man-machine dialogue system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112417109A CN112417109A (en) | 2021-02-26 |
| CN112417109B true CN112417109B (en) | 2023-08-01 |
Family
ID=74840516
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011155117.9A Active CN112417109B (en) | 2020-10-26 | 2020-10-26 | Method and device for testing man-machine dialogue system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112417109B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103578463A (en) * | 2012-07-27 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Automatic testing method and automatic testing device |
| CN106601257A (en) * | 2016-12-31 | 2017-04-26 | 联想(北京)有限公司 | Sound identification method and device and first electronic device |
| CN106796788A (en) * | 2014-08-28 | 2017-05-31 | 苹果公司 | Automatic speech recognition is improved based on user feedback |
| CN108899012A (en) * | 2018-07-27 | 2018-11-27 | 中国电子产品可靠性与环境试验研究所（（工业和信息化部电子第五研究所）（中国赛宝实验室）） | Interactive voice equipment evaluating method, system, computer equipment and storage medium |
| CN109003602A (en) * | 2018-09-10 | 2018-12-14 | 百度在线网络技术（北京）有限公司 | Test method, device, equipment and the computer-readable medium of speech production |
| CN109448701A (en) * | 2018-09-19 | 2019-03-08 | 易诚博睿(南京)科技有限公司 | A kind of intelligent sound recognizes the result statistical system and method for semantic understanding |
| CN109783369A (en) * | 2018-12-20 | 2019-05-21 | 出门问问信息科技有限公司 | A kind of natural language understanding module regression testing method, device and electronic equipment |
| CN109979434A (en) * | 2019-04-30 | 2019-07-05 | 成都启英泰伦科技有限公司 | The test method of local voice mould set product acoustical behavior |
| CN110196927A (en) * | 2019-05-09 | 2019-09-03 | 大众问问(北京)信息科技有限公司 | It is a kind of to take turns interactive method, device and equipment more |
| CN110444227A (en) * | 2019-08-30 | 2019-11-12 | 惠州市德赛西威汽车电子股份有限公司 | A kind of voice automated testing method |
| CN111341296A (en) * | 2020-02-17 | 2020-06-26 | 诚迈科技（南京）股份有限公司 | Voice-controlled response test method, test machine and storage medium |
| CN111522943A (en) * | 2020-03-25 | 2020-08-11 | 平安普惠企业管理有限公司 | Automatic test method, device, equipment and storage medium for logic node |
| CN111696523A (en) * | 2019-03-12 | 2020-09-22 | 大众问问(北京)信息科技有限公司 | Accuracy testing method and device of voice recognition engine and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8296144B2 (en) * | 2008-06-04 | 2012-10-23 | Robert Bosch Gmbh | System and method for automated testing of complicated dialog systems |
| GB2559408B (en) * | 2017-02-06 | 2020-07-08 | Toshiba Kk | A spoken dialogue system, a spoken dialogue method and a method of adapting a spoken dialogue system |
-
2020
- 2020-10-26 CN CN202011155117.9A patent/CN112417109B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103578463A (en) * | 2012-07-27 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Automatic testing method and automatic testing device |
| CN106796788A (en) * | 2014-08-28 | 2017-05-31 | 苹果公司 | Automatic speech recognition is improved based on user feedback |
| CN106601257A (en) * | 2016-12-31 | 2017-04-26 | 联想(北京)有限公司 | Sound identification method and device and first electronic device |
| CN108899012A (en) * | 2018-07-27 | 2018-11-27 | 中国电子产品可靠性与环境试验研究所（（工业和信息化部电子第五研究所）（中国赛宝实验室）） | Interactive voice equipment evaluating method, system, computer equipment and storage medium |
| CN109003602A (en) * | 2018-09-10 | 2018-12-14 | 百度在线网络技术（北京）有限公司 | Test method, device, equipment and the computer-readable medium of speech production |
| CN109448701A (en) * | 2018-09-19 | 2019-03-08 | 易诚博睿(南京)科技有限公司 | A kind of intelligent sound recognizes the result statistical system and method for semantic understanding |
| CN109783369A (en) * | 2018-12-20 | 2019-05-21 | 出门问问信息科技有限公司 | A kind of natural language understanding module regression testing method, device and electronic equipment |
| CN111696523A (en) * | 2019-03-12 | 2020-09-22 | 大众问问(北京)信息科技有限公司 | Accuracy testing method and device of voice recognition engine and electronic equipment |
| CN109979434A (en) * | 2019-04-30 | 2019-07-05 | 成都启英泰伦科技有限公司 | The test method of local voice mould set product acoustical behavior |
| CN110196927A (en) * | 2019-05-09 | 2019-09-03 | 大众问问(北京)信息科技有限公司 | It is a kind of to take turns interactive method, device and equipment more |
| CN110444227A (en) * | 2019-08-30 | 2019-11-12 | 惠州市德赛西威汽车电子股份有限公司 | A kind of voice automated testing method |
| CN111341296A (en) * | 2020-02-17 | 2020-06-26 | 诚迈科技（南京）股份有限公司 | Voice-controlled response test method, test machine and storage medium |
| CN111522943A (en) * | 2020-03-25 | 2020-08-11 | 平安普惠企业管理有限公司 | Automatic test method, device, equipment and storage medium for logic node |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112417109A (en) | 2021-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9767092B2 (en) | Information extraction in a natural language understanding system | |
| CN104461863A (en) | Service system testing method, device and system | |
| CN109637536B (en) | Method and device for automatically identifying semantic accuracy | |
| CN110704597B (en) | Dialogue system reliability verification method, model generation method and device | |
| CN109947651B (en) | Artificial intelligence engine optimization method and device | |
| CN103106061A (en) | Voice input method and device | |
| CN105468582A (en) | Method and device for correcting numeric string based on human-computer interaction | |
| CN109559749A (en) | Combined decoding method and system for speech recognition system | |
| CN112836521A (en) | Question-answer matching method and device, computer equipment and storage medium | |
| CN113488023B (en) | Language identification model construction method and language identification method | |
| CN112417109B (en) | Method and device for testing man-machine dialogue system | |
| CN110364155A (en) | Voice control error-reporting method, electric appliance and computer readable storage medium | |
| CN114420102A (en) | Method and device for speech sentence-breaking, electronic equipment and storage medium | |
| CN111240971B (en) | Method and device for generating wind control rule test case, server and storage medium | |
| CN108597499A (en) | Method of speech processing and voice processing apparatus | |
| CN117033309A (en) | Data conversion method and device, electronic equipment and readable storage medium | |
| US11615789B2 (en) | Systems and methods to verify values input via optical character recognition and speech recognition | |
| CN109273004B (en) | Predictive speech recognition method and device based on big data | |
| CN109474822B (en) | Android television multi-language automatic testing method and device | |
| CN112802495A (en) | Robot voice test method and device, storage medium and terminal equipment | |
| CN115061921A (en) | Automatic test method, device, electronic equipment and readable storage medium | |
| Li et al. | Overview and Analysis of Speech Recognition | |
| CN110647314B (en) | Skill generation method and device and electronic equipment | |
| CN108269570B (en) | Method, device, equipment and storage medium for voice control of background music host | |
| US20150095373A1 (en) | String generation tool |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230629 Address after: 100044 2-406, Floor 4, Building 1, Yard 42, Gaoliangqiao Xiejie Street, Haidian District, Beijing Applicant after: WENWEN INTELLIGENT INFORMATION TECHNOLOGY Co.,Ltd. Address before: 215000 unit 4-b404, creative industry park, 328 Xinghu street, Suzhou Industrial Park, Suzhou City, Jiangsu Province Applicant before: Go out and ask (Suzhou) Information Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |