CN111708693B - Multi-round session system testing method and device - Google Patents
Multi-round session system testing method and device Download PDFInfo
- Publication number
- CN111708693B CN111708693B CN202010522145.3A CN202010522145A CN111708693B CN 111708693 B CN111708693 B CN 111708693B CN 202010522145 A CN202010522145 A CN 202010522145A CN 111708693 B CN111708693 B CN 111708693B
- Authority
- CN
- China
- Prior art keywords
- session
- round
- test case
- verification rule
- round session
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a method and a device for testing a multi-round session system, wherein the method comprises the following steps: acquiring a first multi-round session test case and a first verification rule set; identifying all user session input messages meeting the branch jump condition in the first multi-round session test case; replacing at least one user session input message meeting the branch jump condition in the first multi-round session test case with the user session input message in the preset corpus to obtain a second multi-round session test case; generating a second verification rule set corresponding to the second multi-round session test case according to the verification rule corresponding to the user session input message in the preset corpus and the first verification rule set; and testing the multi-round session system according to the second multi-round session test case and the second verification rule set. The application solves the technical problem that the prior art is difficult to test the multi-round session system efficiently.
Description
Technical Field
The application relates to the technical field of intelligent question answering, in particular to a multi-round session system testing method and device.
Background
Under the background of the artificial intelligence era, the requirement of replacing artificial customer service by using the intelligent robot is also higher and higher, and the main function of the intelligent robot is to realize multi-round conversation with people, so that in the development process of the intelligent robot, how to quickly and comprehensively test the multi-round conversation function of the robot, the development efficiency is improved, and the intelligent robot has important significance. The number of the current multi-round session test cases is small, the coverage rate of the test cases is low, and the accuracy and the test coverage rate of the current multi-round session system are poor.
Disclosure of Invention
The application provides a multi-round session system testing method and device for solving the technical problems in the background technology.
To achieve the above object, according to one aspect of the present application, there is provided a multi-round session system testing method, the method comprising:
acquiring a first multi-round session test case and a first verification rule set corresponding to the first multi-round session test case, wherein the first multi-round session test case comprises a plurality of user session input messages, and the first verification rule set comprises a verification rule corresponding to each user session input message;
identifying all user session input messages meeting the branch jump condition in the first multi-round session test case;
replacing at least one user session input message meeting the branch jump condition in the first multi-round session test case with the user session input message in the preset corpus to obtain a second multi-round session test case;
generating a second verification rule set corresponding to the second multi-round session test case according to the verification rule corresponding to the user session input message in the preset corpus and the first verification rule set;
and testing the multi-round session system according to the second multi-round session test case and the second verification rule set.
Optionally, the testing the multi-round session system according to the second multi-round session test case and the second verification rule set specifically includes:
inputting the second multi-round session test case into a multi-round session system to obtain a system output message corresponding to each user session input message in the second multi-round session test case;
verifying each system output message according to the second verification rule set to obtain a case test result;
and obtaining the test results of the multi-round session system according to the case test results of a plurality of second multi-round session test cases.
Optionally, replacing at least one user session input message meeting the branch jump condition in the first multiple session test cases with a user session input message in a preset corpus, which specifically includes:
determining a preset corpus corresponding to the user session input message meeting the branch jump condition;
randomly selecting a user session input message from the corresponding preset corpus to replace the user session input message meeting the branch jump condition.
Optionally, the multi-round session system testing method further includes:
acquiring multi-round session interaction data generated by interaction between a user and a multi-round session system;
extracting user session input messages from the multi-round session interaction data, and combining the first multi-round session test cases according to the extracted user session input messages;
and generating the first verification rule set according to the preset verification rule corresponding to each extracted user session input message.
To achieve the above object, according to another aspect of the present application, there is provided a multi-round session system testing apparatus, comprising:
the original test case acquisition unit is used for acquiring a first multi-round session test case and a first verification rule set corresponding to the first multi-round session test case, wherein the first multi-round session test case comprises a plurality of user session input messages, and the first verification rule set comprises a verification rule corresponding to each user session input message;
the identification unit is used for identifying all user session input messages meeting the branch jump condition in the first multi-round session test case;
the new test case generation unit is used for replacing at least one user session input message meeting the branch jump condition in the first multi-round session test case with the user session input message in the preset corpus to obtain a second multi-round session test case;
the new verification rule set generation unit is used for generating a second verification rule set corresponding to the second multi-round session test case according to the verification rule corresponding to the user session input message in the preset corpus and the first verification rule set;
and the system testing unit is used for testing the multi-round session system according to the second multi-round session testing case and the second verification rule set.
To achieve the above object, according to another aspect of the present application, there is also provided a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the above-mentioned multi-round session system testing method when executing the computer program.
To achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program which implements the above-described multi-round session system testing method when executed in a computer processor.
The beneficial effects of the application are as follows: the embodiment of the application adopts a self-learning method on the basis of a small number of multi-round session test cases, replaces one or more user session input messages meeting the branch jump condition in the multi-round session test cases by the preset other user session input messages, generates a large number of new multi-round session test cases, and further tests the multi-round session system according to the generated large number of multi-round session test cases. According to the application, a large number of multi-round session test cases are generated by a self-learning method, and the test cases can completely cover each branch of the multi-round session flow, so that the test efficiency and test coverage rate of the multi-round session system are obviously improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a first flowchart of a multi-round session system test method according to an embodiment of the present application;
FIG. 2 is a second flowchart of a multi-round session system test method according to an embodiment of the present application;
fig. 3 is a flow chart of generating a second multi-round session test case according to an embodiment of the present application;
FIG. 4 is a flow chart of generating a first multi-round session test case according to an embodiment of the present application;
FIG. 5 is a block diagram of a multi-round session system testing apparatus according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a computer device according to an embodiment of the application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It is noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of the present application and in the foregoing figures, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
The multi-round session system in the embodiment of the application can be an existing multi-round session intelligent question-answering system and also can be a multi-round session intelligent question-answering robot.
Multiple rounds of dialogue: users, with the express aim, wish to obtain information or services that meet certain constraints, such as: ordering a meal, ordering a ticket, finding music, a movie or a certain commodity, etc. Because the user's needs may be complex, multiple rounds of statement may be required, and the user may also continually modify or refine his own needs during the conversation. In addition, the system may also assist the user in finding satisfactory results by querying, clarifying, or confirming when the user's stated needs are not sufficiently specific or explicit.
Thus, multiple rounds of conversations are not a simple natural language understanding plus information retrieval process, but rather a decision making process that requires the machine to continuously decide the optimal actions to be taken next in the conversation process based on the current state (e.g., providing results, asking for certain constraints, clarifying or confirming needs, etc.) to most effectively assist the user in completing the task of information or service acquisition.
"today day of the week? The answer is a standard answer. But "inquiry of the ticket from Beijing to Shanghai in tomorrow", the man-machine dialogue can be branched according to the travel time of the ticket booking person and the remaining situation of the ticket. In addition, the complexity, randomness and irrational nature of people have led to dialog management facing a variety of problems in application scenarios, such as: the user dialogue deviates from the path of business design, and the user inquires weather conditions of a sentence in a certain place in the process of inquiring ticket booking information.
Aiming at the defects of large consumption of manual tests, incomplete test coverage rate, low test efficiency and the like in the multi-round session test, the application rapidly expands the multi-round session test cases through a self-learning method by predefining the multi-round session test cases and corresponding verification rules, realizes the whole-flow test coverage and completely records the test result, can accurately control the test granularity through adjusting the verification rules, and improves the test efficiency, coverage rate and accuracy of the multi-round session system.
Fig. 1 is a first flowchart of a multi-round session system testing method according to an embodiment of the present application, as shown in fig. 1, the multi-round session system testing method of the present embodiment includes steps S101 to S105.
Step S101, a first multi-round session test case and a first verification rule set corresponding to the first multi-round session test case are obtained, wherein the first multi-round session test case comprises a plurality of user session input messages, and the first verification rule set comprises verification rules corresponding to each user session input message.
In an alternative embodiment of the present application, the first multi-round session test case may be generated based on multi-round session interaction data generated by real user and multi-round session system interactions, where the multi-round session interaction data is a complete interaction branch, and includes a plurality of user session input messages and a plurality of system output messages fed back by the corresponding multi-round session system.
In an alternative embodiment of the present application, the first multi-session test case includes a plurality of user session input messages, and the present application sets corresponding verification rules for each user session input message in advance, and finally combines the verification rules corresponding to each user session input message to generate a first verification rule set. The verification rule refers to a set of correct information which should appear in a system output message fed back by the multi-round session system for a user session input message, and the correct information can be represented as a set of key value pairs, or can be some preset mark symbols, etc., and different marks represent different verification modes, for example, a value of 'NUM' representing a certain field must be a number, etc.
For example, validation rule 1 may be a pair of fixed key pairs that appear in a system output message: "instructionKey": "IDENTITY_AUTH"; validation rule 2 may define for some field values in the system output message: "payAmount" means "NUM" indicating that the payAmount field value must be a number.
The application can accurately control the granularity of test by adjusting the verification rule, thereby improving the test efficiency and accuracy of the multi-round session system.
Step S102, all user session input messages meeting the branch jump condition in the first multi-round session test case are identified.
In the embodiment of the application, the branch jump condition refers to a condition that needs to be met when jumping from one node to another node. In an alternative embodiment of the present application, the branch jump condition may be that some preset fields are present in the message. For example, the user session inputs a message for the user to answer the transfer amount "i want to transfer 100 yuan", at which time an amount may be extracted from the message, and the amount is a preset branch jump condition, so as to enter the next node, which may be the confirmation transfer destination node. In the embodiment of the application, the branch jump condition is a plurality of branch jump conditions preset according to the actual interaction scene.
The node refers to a link for completing a function in the multi-round session flow, and may be intent recognition, element extraction, or obtaining a service parameter, for example, determining a transfer destination node, determining a transfer amount node, and a transfer execution node. A complete multi-round session interaction leg includes multiple nodes.
The user's input may be of explicit purpose, such as making a transfer request, making a query, etc., or entirely non-specific purpose, such as "today's weather is good" etc., as the user interacts with the machine in multiple rounds of conversations. The user session input message meeting the branch jump condition of the application represents the input with definite destination of the user.
Step S103, replacing at least one user session input message meeting the branch jump condition in the first multi-round session test case with the user session input message in the preset corpus to obtain a second multi-round session test case.
The application adopts the self-learning idea, and generates a large number of second multi-round session test cases according to the first multi-round session test cases with a small number, so as to improve the test efficiency and the test coverage rate of the multi-round session system.
In the embodiment of the application, a plurality of preset user session input messages are stored in the corpus, and the user session input messages are all messages meeting the branch jump condition. In addition, the user session input message in the corpus may be a message with the same semantic meaning as the user session input message meeting the branch jump condition, or may be a message with no relevance to the user session input message meeting the branch jump condition.
Aiming at a first multi-session test case, each user session input message is checked one by one, for example, the user session input message is the transfer amount answered by the user: "I want to go to 100" when the multi-round session system can extract the monetary amount (branch jump condition) from the user session input message to go to the next node. The application can select the same message as the user session input message Wen Yuyi from the preset corpus, for example, other amount answer modes: "100 blocks may be changed", "i want to change to one hundred blocks", etc. to replace the user session input message in the first multi-round session test case. At this time, the verification rule of the user session input message after the replacement is consistent with that before the replacement. The application can also select the corpus from the corpus of other preset scenes to replace, for example, select the message 'I want to charge the telephone fee' to replace the user session input message in the first multi-session test case, and synchronously modify the verification rule into the payment related content at the moment. If the user session input message is in the instruction format: "ExistPayeeName": "true", i.e. when there is a payee, the next node is jumped to, it can be replaced by a user session input message that the payee does not exist: "ExistPayeeName": "false", while modifying the validation rule to the case where the payee does not exist.
In the embodiment of the application, when replacing the user session input messages meeting the branch jump condition in the first multi-round session test case, all the user session input messages meeting the branch jump condition can be replaced at the same time, and one or more of the user session input messages meeting the branch jump condition can be replaced. Therefore, the application can generate a large number of second multi-round session test cases according to the first multi-round session test cases by adopting the automatic learning idea, so that the test cases can completely cover each branch of the multi-round session flow, and the test coverage rate of the multi-round session system is obviously improved.
Step S104, a second verification rule set corresponding to the second multi-round session test case is generated according to the verification rule corresponding to the user session input message in the preset corpus and the first verification rule set.
After the second multi-round session test case is generated according to the step S104, a second verification rule set corresponding to the second multi-round session test case needs to be generated. Specifically, the user session input message corresponding verification rule before replacement in the first verification rule set is replaced by the user session input message corresponding verification rule after replacement, and a second verification rule set is generated.
Step S105, testing the multi-round session system according to the second multi-round session test case and the second verification rule set.
Therefore, the embodiment of the application adopts a self-learning method on the basis of a small number of multi-round session test cases, replaces one or more user session input messages meeting the branch jump condition in the multi-round session test cases by the preset other user session input messages, generates a large number of new multi-round session test cases, and further tests the multi-round session system according to the generated large number of multi-round session test cases. According to the application, a large number of multi-round session test cases are generated by a self-learning method, and the test cases can completely cover each branch of the multi-round session flow, so that the test efficiency and test coverage rate of the multi-round session system are obviously improved.
Fig. 2 is a second flowchart of a multi-round session system testing method according to an embodiment of the present application, as shown in fig. 2, the step S105 of testing the multi-round session system according to the second multi-round session test case and the second verification rule set specifically includes steps S201 to S203.
Step 201, inputting the second multi-round session test case into a multi-round session system, so as to obtain a system output message corresponding to each user session input message in the second multi-round session test case.
In the embodiment of the application, the step calls the interface service of the multi-round session system, and each user session input message in the second multi-round session test case is sequentially input into the multi-round session system to obtain the system output message corresponding to each user session input message. The interface service refers to an access service provided by the multi-round session system, and a calling party can acquire information according to a mode specified by a service party, for example, a REST mode, a WebService mode and the like.
Step S202, verifying each system output message according to the second verification rule set to obtain a case test result.
The step verifies the system output message corresponding to each user session input message according to the verification rule corresponding to each user session input message, and verifies whether the multi-round interaction system can process each user session input message correctly. The test result records that the test case is a success case or a failure case, and if the test case is a failure case, detailed error information is recorded.
Step 203, obtaining the test result of the multi-round session system according to the case test results of the second multi-round session test cases.
The application collects and gathers the case test results, records the number of successful cases and failed cases, and gathers detailed error information of the failed cases. The successful case refers to a case passing field integrity verification and rule verification. The failure case refers to a case which does not pass field integrity verification or rule verification.
Fig. 3 is a flowchart of generating a second multi-round session test case according to an embodiment of the present application, and as shown in fig. 3, the specific flow of generating the second multi-round session test case in step S103 includes step S301 and step S302.
Step S301, determining a preset corpus corresponding to the user session input message satisfying the branch jump condition.
In the embodiment of the application, the message to be replaced is determined from all the user session input messages meeting the branch jump condition in the first multi-round session test case, and particularly, a random number of messages can be randomly selected from all the user session input messages meeting the branch jump condition for replacement. After determining the user session input message to be replaced in the first multi-session test case, acquiring a preset corpus corresponding to the user session input message to be replaced. In an alternative embodiment, the message in the preset corpus is the same message or the same type of message as the user session input message Wen Yuyi to be replaced.
Step S302, randomly selecting a user session input message from the corresponding preset corpus to replace the user session input message meeting the branch jump condition.
Fig. 4 is a flowchart of generating a first multi-round session test case according to an embodiment of the present application, as shown in fig. 4, and in an alternative embodiment of the present application, the first multi-round session test case and the first verification rule set in step S101 are specifically generated from step S401 to step S403.
Step S401, multi-round session interaction data generated by interaction between a user and a multi-round session system is obtained.
Step S402, extracting user session input messages from the multi-round session interaction data, and combining the first multi-round session test cases according to the extracted user session input messages.
Step S403, generating the first verification rule set according to the preset verification rule corresponding to each extracted user session input message.
In an alternative embodiment of the present application, the first multi-round session test case may be generated based on multi-round session interaction data generated by real user and multi-round session system interactions, where the multi-round session interaction data is a complete interaction branch, and includes a plurality of user session input messages and a plurality of system output messages fed back by the corresponding multi-round session system.
According to the embodiment, the test cases are rapidly expanded through self-learning, the full-flow test is automatically covered, the test result is completely recorded, the control test granularity can be accurately controlled through adjusting the verification rule, and the test efficiency, coverage rate and accuracy of the multi-round session flow are improved.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
Based on the same inventive concept, the embodiment of the present application also provides a multi-round session system testing device, which can be used to implement the multi-round session system testing method described in the above embodiment, as described in the following embodiment. Because the principle of the multi-round session system testing device for solving the problem is similar to that of the multi-round session system testing method, the embodiment of the multi-round session system testing device can refer to the embodiment of the multi-round session system testing method, and the repetition is omitted. As used below, the term "unit" or "module" may be a combination of software and/or hardware that implements the intended function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Fig. 5 is a block diagram of a multi-round session system testing apparatus according to an embodiment of the present application, as shown in fig. 5, the multi-round session system testing apparatus according to the embodiment of the present application includes: an original test case acquisition unit 1, an identification unit 2, a new test case generation unit 3, a new verification rule set generation unit 4, and a system test unit 5.
The original test case obtaining unit 1 is configured to obtain a first multi-round session test case and a first verification rule set corresponding to the first multi-round session test case, where the first multi-round session test case includes a plurality of user session input messages, and the first verification rule set includes a verification rule corresponding to each user session input message.
And the identifying unit 2 is used for identifying all user session input messages meeting the branch jump condition in the first multi-session test case.
And the new test case generation unit 3 is configured to replace at least one user session input message meeting the branch jump condition in the first multiple session test cases with a user session input message in a preset corpus, so as to obtain a second multiple session test case.
And the new verification rule set generating unit 4 is used for generating a second verification rule set corresponding to the second multi-round session test case according to the verification rule corresponding to the user session input message in the preset corpus and the first verification rule set.
And the system testing unit 5 is used for testing the multi-round session system according to the second multi-round session testing case and the second verification rule set.
In an alternative embodiment of the present application, the system testing unit 5 specifically includes:
the system output message acquisition module is used for inputting the second multi-round session test case into a multi-round session system to obtain a system output message corresponding to each user session input message in the second multi-round session test case;
the message verification module is used for verifying each system output message according to the second verification rule set to obtain a case test result;
and the test result determining module is used for obtaining the test results of the multi-round session system according to the case test results of a plurality of second multi-round session test cases.
In an alternative embodiment of the present application, the new test case generating unit 3 specifically includes:
the corpus determining module is used for determining a preset corpus corresponding to the user session input message meeting the branch jump condition;
and the message replacing module is used for randomly selecting a user session input message from the corresponding preset corpus to replace the user session input message meeting the branch jump condition.
In an alternative embodiment of the present application, the multi-round session system testing apparatus of the present application further includes:
the multi-round session interaction data acquisition unit is used for acquiring multi-round session interaction data generated by interaction between a user and the multi-round session system;
the original test case generation unit is used for extracting user session input messages from the multi-round session interaction data and combining the first multi-round session test case according to the extracted user session input messages;
the original verification rule set generating unit is used for generating the first verification rule set according to the preset verification rule corresponding to each extracted user session input message.
To achieve the above object, according to another aspect of the present application, there is also provided a computer apparatus. As shown in fig. 6, the computer device includes a memory, a processor, a communication interface, and a communication bus, where a computer program executable on the processor is stored on the memory, and when the processor executes the computer program, the steps in the method of the above embodiment are implemented.
The processor may be a central processing unit (Central Processing Unit, CPU). The processor may also be any other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof.
The memory is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and units, such as corresponding program units in the above-described method embodiments of the application. The processor executes the various functional applications of the processor and the processing of the composition data by running non-transitory software programs, instructions and modules stored in the memory, i.e., implementing the methods of the method embodiments described above.
The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more units are stored in the memory, which when executed by the processor, performs the method in the above embodiments.
The details of the computer device may be correspondingly understood by referring to the corresponding relevant descriptions and effects in the above embodiments, and will not be repeated here.
To achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps of the multi-round session system testing method described above. It will be appreciated by those skilled in the art that implementing all or part of the above-described embodiment method may be implemented by a computer program to instruct related hardware, where the program may be stored in a computer readable storage medium, and the program may include the above-described embodiment method when executed. Wherein the storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a random access Memory (RandomAccessMemory, RAM), a Flash Memory (Flash Memory), a Hard Disk (HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
It will be apparent to those skilled in the art that the modules or steps of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, or they may alternatively be implemented in program code executable by computing devices, such that they may be stored in a memory device for execution by the computing devices, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. A multi-round session system testing method, comprising:
acquiring a first multi-round session test case and a first verification rule set corresponding to the first multi-round session test case, wherein the first multi-round session test case comprises a plurality of user session input messages, the first verification rule set comprises verification rules corresponding to each user session input message, and the verification rules refer to a group of correct information which should appear in a system output message fed back by a multi-round session system aiming at the user session input message;
identifying all user session input messages meeting the branch jump condition in the first multi-round session test case;
replacing at least one user session input message meeting the branch jump condition in the first multi-round session test case with the user session input message in the preset corpus to obtain a second multi-round session test case;
generating a second verification rule set corresponding to the second multi-round session test case according to the verification rule corresponding to the user session input message in the preset corpus and the first verification rule set;
and testing the multi-round session system according to the second multi-round session test case and the second verification rule set.
2. The method for testing a multi-round session system according to claim 1, wherein the testing the multi-round session system according to the second multi-round session test case and the second verification rule set specifically comprises:
inputting the second multi-round session test case into a multi-round session system to obtain a system output message corresponding to each user session input message in the second multi-round session test case;
verifying each system output message according to the second verification rule set to obtain a case test result;
and obtaining the test results of the multi-round session system according to the case test results of a plurality of second multi-round session test cases.
3. The method for testing a multiple session system according to claim 1, wherein replacing at least one of the user session input messages satisfying a branch jump condition in the first multiple session test cases with a user session input message in a preset corpus specifically comprises:
determining a preset corpus corresponding to the user session input message meeting the branch jump condition;
randomly selecting a user session input message from the corresponding preset corpus to replace the user session input message meeting the branch jump condition.
4. The multi-round session system testing method of claim 1, further comprising:
acquiring multi-round session interaction data generated by interaction between a user and a multi-round session system;
extracting user session input messages from the multi-round session interaction data, and combining the first multi-round session test cases according to the extracted user session input messages;
and generating the first verification rule set according to the preset verification rule corresponding to each extracted user session input message.
5. A multi-round session system testing apparatus, comprising:
the original test case acquisition unit is used for acquiring a first multi-round session test case and a first verification rule set corresponding to the first multi-round session test case, wherein the first multi-round session test case comprises a plurality of user session input messages, the first verification rule set comprises a verification rule corresponding to each user session input message, and the verification rule refers to a group of correct information which should appear in a system output message fed back by a multi-round session system aiming at the user session input message;
the identification unit is used for identifying all user session input messages meeting the branch jump condition in the first multi-round session test case;
the new test case generation unit is used for replacing at least one user session input message meeting the branch jump condition in the first multi-round session test case with the user session input message in the preset corpus to obtain a second multi-round session test case;
the new verification rule set generation unit is used for generating a second verification rule set corresponding to the second multi-round session test case according to the verification rule corresponding to the user session input message in the preset corpus and the first verification rule set;
and the system testing unit is used for testing the multi-round session system according to the second multi-round session testing case and the second verification rule set.
6. The multi-round session system testing apparatus of claim 5, wherein the system testing unit comprises:
the system output message acquisition module is used for inputting the second multi-round session test case into a multi-round session system to obtain a system output message corresponding to each user session input message in the second multi-round session test case;
the message verification module is used for verifying each system output message according to the second verification rule set to obtain a case test result;
and the test result determining module is used for obtaining the test results of the multi-round session system according to the case test results of a plurality of second multi-round session test cases.
7. The multi-round session system testing apparatus of claim 5, wherein the new test case generation unit comprises:
the corpus determining module is used for determining a preset corpus corresponding to the user session input message meeting the branch jump condition;
and the message replacing module is used for randomly selecting a user session input message from the corresponding preset corpus to replace the user session input message meeting the branch jump condition.
8. The multiple round session system testing apparatus of claim 5, further comprising:
the multi-round session interaction data acquisition unit is used for acquiring multi-round session interaction data generated by interaction between a user and the multi-round session system;
the original test case generation unit is used for extracting user session input messages from the multi-round session interaction data and combining the first multi-round session test case according to the extracted user session input messages;
the original verification rule set generating unit is used for generating the first verification rule set according to the preset verification rule corresponding to each extracted user session input message.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 4 when executing the computer program.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed in a computer processor implements the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010522145.3A CN111708693B (en) | 2020-06-10 | 2020-06-10 | Multi-round session system testing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010522145.3A CN111708693B (en) | 2020-06-10 | 2020-06-10 | Multi-round session system testing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111708693A CN111708693A (en) | 2020-09-25 |
| CN111708693B true CN111708693B (en) | 2023-08-11 |
Family
ID=72539960
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010522145.3A Active CN111708693B (en) | 2020-06-10 | 2020-06-10 | Multi-round session system testing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708693B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113609017A (en) * | 2021-08-06 | 2021-11-05 | 中国工商银行股份有限公司 | Interface test case generation method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059000A (en) * | 2019-03-15 | 2019-07-26 | 深圳壹账通智能科技有限公司 | Chat drama test control method, device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6782518B2 (en) * | 2002-03-28 | 2004-08-24 | International Business Machines Corporation | System and method for facilitating coverage feedback testcase generation reproducibility |
-
2020
- 2020-06-10 CN CN202010522145.3A patent/CN111708693B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059000A (en) * | 2019-03-15 | 2019-07-26 | 深圳壹账通智能科技有限公司 | Chat drama test control method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111708693A (en) | 2020-09-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109376354A (en) | Fraud recognition methods, device, electronic equipment and readable storage medium storing program for executing | |
| CN108256870A (en) | Description information and update, data processing method and device are generated based on topological structure | |
| CN105933905A (en) | Method and device for realizing wireless access point connection authentication | |
| CN113535399A (en) | NFV resource scheduling method, device and system | |
| CN111708693B (en) | Multi-round session system testing method and device | |
| CN112182046B (en) | Information recommendation method, device, equipment and medium | |
| CN111933118B (en) | Method and device for optimizing voice recognition and intelligent voice dialogue system applying same | |
| CN111563037B (en) | Test optimization method and device based on test questions, storage medium and terminal | |
| de Figueiredo et al. | Generating interaction test cases for mobile phone systems from use case specifications | |
| CN114531334A (en) | Intention processing method and device, electronic equipment and readable storage medium | |
| CN111708712A (en) | User behavior test case generation method, flow playback method and electronic equipment | |
| CN105743939B (en) | A kind of method and apparatus of message push | |
| CN110362294A (en) | Development task executes method, apparatus, electronic equipment and storage medium | |
| CN113556430B (en) | Outbound system and outbound method | |
| CN113705249A (en) | Dialogue processing method, system, device and computer readable storage medium | |
| CN112182047B (en) | Information recommendation method, device, equipment and medium | |
| CN112487170A (en) | Scene configuration-oriented human-computer interaction dialogue robot system | |
| CN113609390A (en) | Information analysis method and device, electronic equipment and computer readable storage medium | |
| CN109412931A (en) | The method, apparatus and terminal device of knowledge question are carried out in the way of instant messaging | |
| US20200285553A1 (en) | Optimizing interaction flows | |
| CN111459836A (en) | Conversation strategy configuration method and conversation system | |
| CN108536811A (en) | Interactive voice determining method of path based on machine learning and device, storage medium, terminal | |
| CN114417817B (en) | Session information cutting method and device | |
| CN109189912A (en) | The update method and device of user's consulting statement library | |
| CN115640384A (en) | Training method and device of dialogue model, electronic equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |