CN108763329A - Evaluating method, device and the computer equipment of voice interactive system IQ level - Google Patents
Evaluating method, device and the computer equipment of voice interactive system IQ level Download PDFInfo
- Publication number
- CN108763329A CN108763329A CN201810433832.0A CN201810433832A CN108763329A CN 108763329 A CN108763329 A CN 108763329A CN 201810433832 A CN201810433832 A CN 201810433832A CN 108763329 A CN108763329 A CN 108763329A
- Authority
- CN
- China
- Prior art keywords
- test problem
- interactive system
- periodicity
- instruction
- voice interactive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
Abstract
The present invention relates to the evaluating method of voice interactive system IQ level, device and computer equipment, the method includes：Test problem is obtained, the test problem is phonetic problem；Obtain the instruction cycles of test problem described in the voice interactive system success response；Wherein, an instruction cycle is the voice interactive system and an extraneous dialog turns；The evaluation result of the voice interactive system IQ level is determined by described instruction periodicity.The embodiment of the present invention solves the problems, such as that the evaluation result reliability of voice interactive system IQ level is low, and evaluation and test realization process is simple, and evaluation result is more objective.
Description
Technical field
The present invention relates to evaluation and test technical field, more particularly to the evaluating method of voice interactive system IQ level, device,
Computer equipment and storage medium.
Background technology
With the development of development of Mobile Internet technology and intelligent hardware devices, the mankind have been no longer satisfied with keyboard input and hand
The man-machine interaction modes such as input are write, voice interactive system technology is in each neck such as electronic information, internet, medical treatment, education, office
Domain is widely applied, cover intelligent sound input system, intelligent sound assistant, intelligent sound box, vehicle-mounted voice system,
The products such as intelligent sound auxiliary medical system, intelligent oral evaluation system, intelligent meeting system.
Based on voice interactive system, smart machine can be allowed to have and listen knowledge ability as the mankind, while by mankind's language
Say that the natural semanteme stated is automatically converted to the structuring semanteme that computer capacity understands and operates, and finishing man-machine interaction work(
Energy.For the performance evaluating of voice interactive system, widespread practice is in industry, according to the recognition result of voice interactive system with
The matching degree of received pronunciation recognition result is evaluated and tested.
In the implementation of the present invention, following problem, traditional performance evaluating exists in the prior art in inventor
Method need to build the database in test problem library and received pronunciation recognition result in advance, due in database sample it is limited
Property, given received pronunciation recognition result can not consider that language communicative habits, problem answers diversity, subjective randomness etc. are practical
Situation leads to the reliability of the evaluating result of voice interactive system IQ level (or make understandability, intelligence degree etc.)
It is not high.
Invention content
Based on this, it is necessary to not high to the reliability of the evaluating result of voice interactive system IQ level for existing way
The problem of, a kind of evaluating method, device, computer equipment and the storage medium of voice interactive system IQ level are provided.
First aspect of the embodiment of the present invention provides a kind of evaluating method of voice interactive system IQ level, including：
Test problem is obtained, the test problem is phonetic problem；
Obtain the instruction cycles of test problem described in the voice interactive system success response；Wherein, an instruction week
Phase is the voice interactive system and an extraneous dialog turns；
The evaluation result of the voice interactive system IQ level is determined by described instruction periodicity.
It is described in one of the embodiments, that the voice interactive system IQ level is determined by described instruction periodicity
Evaluation result, including：
The corresponding instruction cycle cost of the test problem is determined by described instruction periodicity；
The evaluation result of the voice interactive system IQ level is determined by described instruction life cycle costing.
The acquisition test problem in one of the embodiments, including：Obtain preset multiple test problems；
The evaluation result that the voice interactive system IQ level is determined by described instruction life cycle costing, including：
According to the corresponding instruction cycle cost of the multiple test problem, the voice interactive system IQ water is determined
Flat evaluation result score.
In one of the embodiments, further include：It is described according to the multiple test problem corresponding instruction cycle
Cost determines the evaluation result score of the voice interactive system IQ level, including：
Obtain the corresponding weight of the multiple test problem；
According to the corresponding instruction cycle cost of the multiple test problem and weight, the voice interactive system is determined
The evaluation result score of IQ level.
The multiple corresponding weight of test problem of acquisition in one of the embodiments, including：
Obtain the corresponding type of the multiple test problem；
All types of corresponding weights are obtained, as the corresponding weight of corresponding test problem.
It is described in one of the embodiments, that the test problem corresponding instruction week is determined by described instruction periodicity
Current cost, including：
Obtain preset periodicity limit value；
By described instruction periodicity and the periodicity limit value, determine the test problem corresponding instruction cycle at
This.
It is described by described instruction periodicity and the periodicity limit value in one of the embodiments, determine the survey
Why corresponding instruction cycle cost is inscribed, including：
If described instruction periodicity is N1, determine that the corresponding instruction cycle cost of the test problem is cost score S1；
If described instruction periodicity is N2, determine that the corresponding instruction cycle cost of the test problem is cost score S2；
If described instruction periodicity is N3, determine that the corresponding instruction cycle cost of the test problem is cost score S3；
Alternatively,
If described instruction periodicity is N1, determine that the corresponding instruction cycle cost of the test problem is cost level L1；
If described instruction periodicity is N2, determine that the corresponding instruction cycle cost of the test problem is cost level L2；
If described instruction periodicity is N3, determine that the corresponding instruction cycle cost of the test problem is cost level L3；
Wherein, 0<N1<N2<M≤N3, N1 are preset periodicity, and M indicates periodicity limit value；S3<S2<S1, L1,
The corresponding rank of L2, L3 is successively decreased successively.
In one of the embodiments, when described instruction periodicity is between N1 and M, with described instruction periodicity
It is incremented by, the value of cost score S2 is successively decreased.
In one of the embodiments, the value of the cost score S2 successively decrease including：It is linear decrease, exponential damping, right
At least one of number decaying.
Second aspect of the embodiment of the present invention provides a kind of evaluating apparatus of voice interactive system IQ level, including：
Problem acquisition module, for obtaining test problem, the test problem is phonetic problem；
Cycle count module, the instruction cycle for obtaining test problem described in the voice interactive system success response
Number；Wherein, an instruction cycle is the voice interactive system and an extraneous dialog turns；
And
As a result determining module, the evaluation and test for determining the voice interactive system IQ level by described instruction periodicity
As a result.
The third aspect of the embodiment of the present invention provides a kind of computer equipment, including memory, processor and is stored in institute
The computer program that can be run on memory and on the processor is stated, the processor executes real when the computer program
Now as above state the evaluating method of voice interactive system IQ level.
Fourth aspect of the embodiment of the present invention provides a kind of computer storage media, is stored thereon with computer program, the journey
The evaluating method for as above stating voice interactive system IQ level is realized when sequence is executed by processor.
Implement embodiment provided by the invention, be based on preset test problem, obtains voice interactive system success response institute
The dialog turns quantity of test problem is stated, which can reflect instruction cycle cost, be then based on dialog turns quantity and determine language
The evaluation result of sound interactive system IQ level；Since assessment core reference is to test to ask described in voice interactive system success response
The dialog turns quantity of topic, therefore the reality such as multilingual communicative habits, problem answers diversity, subjective randomness can be compatible with
Situation is conducive to the reliability for improving evaluation result.
Description of the drawings
Fig. 1 is the applied environment figure of the evaluating method of voice interactive system IQ level in one embodiment；
Fig. 2 is the schematic flow chart of the evaluating method of the voice interactive system IQ level of an embodiment；
Fig. 3 is the schematic flow chart of the evaluating method of the voice interactive system IQ level of another embodiment；
Fig. 4 is the schematic diagram of the evaluating apparatus of the voice interactive system IQ level of an embodiment；
Fig. 5 is the internal structure chart of one embodiment Computer equipment.
Specific implementation mode
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
Referenced herein " embodiment " is it is meant that a particular feature, structure, or characteristic described can wrap in conjunction with the embodiments
It is contained at least one embodiment of the application.Each position in the description occur the phrase might not each mean it is identical
Embodiment, nor the independent or alternative embodiment with other embodiments mutual exclusion.Those skilled in the art explicitly and
Implicitly understand, embodiment described herein can be combined with other embodiments.
The evaluating method of voice interactive system IQ level provided by the present application, can be applied to application as shown in Figure 1
In environment.Wherein, intelligent terminal 102 is the terminal equipped with voice interactive system, for example, intelligent sound box, unmanned, intelligent language
Sound assistants etc. have the terminal of voice interactive function.Evaluation and test terminal 104 is used to evaluate and test the performance of intelligent terminal 102,
In one embodiment, evaluation and test terminal 104 can export voice messaging, and according to intelligent terminal 102 to the response feelings of the voice messaging
Condition evaluates and tests the IQ level of the voice interactive system of intelligent terminal 102.In other embodiments, voice messaging is exported
The step of, it can also be executed by other third party's terminals or user.
In one embodiment, as shown in Fig. 2, providing a kind of evaluating method of voice interactive system IQ level, with
This method is applied to illustrate for the evaluation and test terminal in Fig. 1, includes the following steps：
S110, obtains test problem, and the test problem is phonetic problem.
Wherein, test problem refers to the voice messaging for needing to send out to voice interactive system, for example, " beating electricity to XXX
Words ", " weather for inquiring today " etc..
S120 obtains the instruction cycles of test problem described in the voice interactive system success response；Wherein, a finger
It is the voice interactive system and an extraneous dialog turns to enable the period.
In inventive embodiments, for following scene：
User：It makes a phone call to XXX.(user instruction)
System：Alright, if make a phone call now to contact person XXX？(system execution)
User：Yes.(user instruction)
System：It is calling.(system execution)
Instruction cycle can be understood as：The above interactive process includes 2 instruction cycles, i.e. human-computer dialogue bout
Concept.
In above-mentioned scene, test problem is：" making a phone call to XXX ".The voice interactive system success response test problem
Instruction cycles be 2.
In another scene：
User：It makes a phone call to XXX.(user instruction)
System：I'm sorry, do not catch.(system execution)
User：It makes a phone call to XXX.(user instruction)
System：Alright, if make a phone call now to contact person XXX？(system execution)
User：Yes.(user instruction)
System：It is calling.(system execution)
In above-mentioned scene, for identical test problem (" making a phone call to XXX "), voice interactive system success response should
The instruction cycles of test problem is 3.
Based on above-mentioned example, the instruction cycles of voice interactive system success response test problem refers to interactive voice
System is from the voice of test problem is received to the reply bout of the corresponding instruction of execution test problem.
S130 determines the evaluation result of the voice interactive system IQ level by described instruction periodicity.
In the embodiment of the present invention, when the IQ level of voice interactive system is relatively low, success response same test problem
Instruction cycles greatly may be more, when the IQ level of voice interactive system is higher, success response same test
The instruction cycles of problem greatly may be less.
By the method for above-described embodiment, it is based on preset test problem, is obtained described in voice interactive system success response
The instruction cycles of test problem, the data can reflect instruction cycle cost, are then based on instruction cycles and determine interactive voice
The evaluation result of system IQ level；Due to the finger that assessment core reference is test problem described in voice interactive system success response
Periodicity is enabled, therefore the actual conditions such as multilingual communicative habits, problem answers diversity, subjective randomness can be compatible with, is had
Conducive to the reliability of raising evaluation result.
Also, by voice interactive system success response test problem spend instruction cycles number evaluated and tested,
Speech recognition capabilities, semantic understanding ability and the semantic processing ability of energy comprehensive evaluating voice interactive system, evaluation and test dimension are complete
The reliability of evaluation result is improved from another level in face.
In addition, without structure standard language recognition result database in advance, the process complexity of evaluation and test is also reduced.
In one embodiment, above-mentioned steps S130 includes：Determine that the test problem corresponds to by described instruction periodicity
Instruction cycle cost, the evaluation and test knot of the voice interactive system IQ level is then determined by described instruction life cycle costing
Fruit.
The instruction cycles of voice interactive system success response same test problem is more, and the required time is more, voice
The task execution amount of interactive system is also more, therefore corresponding instruction cycle cost higher.Therefore, pass through instruction cycles, energy
Enough reflect the height of instruction cycle cost, and then voice interactive system is more intuitively determined by the height of instruction cycle cost
The evaluation result of IQ level.
In one embodiment, it can be multiple to participate in the test problem of evaluation and test, and multiple test problems are pre-stored in phase
In the database answered.When being evaluated and tested to voice interactive system, each test can be obtained with this from the database
Problem evaluates and tests interactive voice system, can also be therefrom according to certain regular selected part test problem come to voice
Interaction system is evaluated and tested, certain rule also include be random selection.
In other embodiments, can with when completely random, without preset test problem and store, user, evaluation and test
Terminal or third party's terminal send out the voice messaging of test problem at random, are hereafter based on the corresponding evaluation and test letter of each test problem
Breath, the performance of comprehensive evaluating voice interactive system.
Further, the interactive voice system can be determined according to the corresponding instruction cycle cost of multiple test problems
The evaluation result for IQ level of uniting.Such as：Obtain the corresponding weight of the multiple test problem；According to the multiple test
The corresponding instruction cycle cost of problem and weight, determine the evaluation result of the voice interactive system IQ level.Wherein,
Type can understand that complexity is divided according to problem, or the minimum instruction cycles needed for the success response problem is drawn
Point, either divided (for example, English type and Chinese type, mandarin according to the corresponding sound-type of problem or accent
Type, Guangdong language type or other dialect types etc.).For different types of test problem, corresponding weight is set, it can thus be concluded that
To voice interactive system to the response condition of different type voice, the IQ level of voice interactive system is evaluated and tested comprehensively based on this.
Voice interactive system is evaluated and tested by multiple test problems, can further improve the reliability of evaluation result.
In one embodiment, the tool of the corresponding instruction cycle cost of the test problem is determined by described instruction periodicity
Body mode may include：Obtain preset periodicity limit value；By described instruction periodicity and the periodicity limit value, determine described in
The corresponding instruction cycle cost of test problem.Such as：
If described instruction periodicity is N1, determine that the corresponding instruction cycle cost of the test problem is cost score S1；
If described instruction periodicity is N2, determine that the corresponding instruction cycle cost of the test problem is cost score S2；If the finger
It is N3 to enable periodicity, determines that the corresponding instruction cycle cost of the test problem is cost score S3.
Wherein, 0<N1<N2<M≤N3, N1 are preset periodicity, and M indicates periodicity limit value；S3<S2<S1.This hair
In bright embodiment, instruction cycle cost is that cost score is higher, shows the cost of voice interactive system success response test problem
It is lower.Wherein, preset periodicity is determined according to the minimum instruction cycles that one test problem of success response is consumed,
N1 can be 1, can also be other integers, be determined depending on actual conditions.
It is corresponding, it can determine voice interactive system intelligence according to the cost score of the corresponding instruction cycle cost of test problem
The flat evaluation result in Shangshui.
In other embodiments, determine that the corresponding instruction cycle cost of the test problem can also be other than cost score
Other modes, such as grade mode, specifically for example：
If described instruction periodicity is N1, determine that the corresponding instruction cycle cost of the test problem is cost level L1；
If described instruction periodicity is N2, determine that the corresponding instruction cycle cost of the test problem is cost level L2；If the finger
It is N3 to enable periodicity, determines that the corresponding instruction cycle cost of the test problem is cost level L3；Wherein, L1, L2, L3 are corresponded to
Rank successively decrease successively.In the embodiment of the present invention, the cost level of instruction cycle cost is higher, shows voice interactive system success
The cost for responding the test problem is lower.Wherein, rank height can be based on the determination of rating value size, such as rank 1
It is more than rank 2 less than rank 2 or rank 1, it can be determines according to actual conditions.
It is corresponding, it can determine voice interactive system intelligence according to the cost level of the corresponding instruction cycle cost of test problem
The flat evaluation result in Shangshui.
On the basis of the above embodiments, in conjunction with Fig. 3, it is applied to for the evaluation and test terminal in Fig. 1 in this way below, into
The evaluating method of voice interactive system IQ level is illustrated in one step.Wherein, the test problem for participating in evaluating and testing has more
A, accordingly the evaluating method of voice interactive system IQ level includes the following steps：
Step S301 is arranged multiple test problems, is stored in issue database.
Step S302, to multiple test problem classified types of setting, and to all types of corresponding test problem settings pair
The weight answered preserves the correspondence.
Step S303, obtains test problem from issue database, sends out the corresponding voice of the test problem.
The corresponding voice of test problem can also be that test user sends out or the either evaluation and test terminal is sent out
Three method, apparatus are sent out, such as in such a way that user mobile phone plays the voice recorded.
Step S304 obtains the instruction cycles of the voice interactive system success response test problem.
Voice interactive system can be caused in identification, reason due to the difference of IQ level for different voice interactive systems
Task can not be accurately completed when solution and processing phonetic order.In this case voice interactive system will give make mistake answer,
Situations such as executing the operation of mistake or operation can not be executed.At this moment it needs user to assign voice command again, such as reduces word speed, changes
Become phonetic order expression way etc., voice interactive system is made preferably to identify and understand the voice of user.It will make one survey of solution
Why inscribe required instruction cycles to increase, increases instruction cycle cost.Therefore, in voice interactive system success response one
When phonetic order, spent instruction cycles is more, then reflects that the IQ level of the voice interactive system is relatively low.
Step S305 determines the corresponding instruction cycle cost of the test problem by described instruction periodicity.
Such as：Assuming that the minimum instruction cycles needed for the voice interactive system success response test problem is 1, and with M
A instruction cycle is periodicity limit value.So, if voice interactive system can not provide correct option or nothing within M instruction cycle
Method rationally responds, then the cost scoring of the task is 0；If voice interactive system is provided within first instruction cycle and is correctly answered
Case or rationally response, then the cost scoring of the task is full marks S0；If voice interactive system is in N (1<N<M) a instruction cycle
Correct option or rationally response are inside provided, then the cost scoring of the task is SN(0<SN<S).Wherein, S0And SNBetween to pass
Subtract relationship, including but not limited to linear decrease, exponential damping, logarithmic decrement or other types attenuation relation, it can be according to test
The characteristics of demand and problem set of object, is selected.
Whether step S306 also has other test problems to have neither part nor lot in evaluation and test in issue database, if so, return to step
S303 obtains next test problem and is evaluated and tested；If it is not, executing next step.
Step S307 obtains the corresponding weight of each test problem for participating in evaluation and test, respectively right according to multiple test problems
The instruction cycle cost and weight answered determine the evaluation result score of the voice interactive system IQ level.
That is, the evaluation result score of final voice interactive system IQ level, is obtained by the corresponding cost of all kinds of problems
Divide and its synthesis operation of weight obtains.Comprehensive operation includes but not limited to cumulative, tired the operations such as to multiply.
The method of above-described embodiment is held by introducing the concept of instruction cycle cost using the success of voice interactive system
Instruction cycle cost that one task of row (i.e. one test problem of success response) is consumed, as weighing its IQ level
Index, and quantification treatment is carried out to its its IQ level.Due to instruction cycle cost calculation is easily operated, convenient for digitlization etc.
Feature can accurately distinguish the IQ level of each voice interactive system.Meanwhile, it is capable to which model answer is avoided to match evaluating method band
The confinement problems come make more reasonable objective, the reliability higher of evaluation result.
It should be understood that the evaluating method of the voice interactive system IQ level of above-described embodiment, application include
But being not limited to intelligent sound box, unmanned, intelligent sound assistant etc. has the terminal of intelligentized voice interactive function.
It should be understood that for each method embodiment above-mentioned, although each step in flow chart is according to arrow
Instruction is shown successively, but these steps are not the inevitable sequence indicated according to arrow to be executed successively.Unless having herein bright
True explanation, there is no stringent sequences to limit for the execution of these steps, these steps can execute in other order.And
And at least part step in the flow chart of embodiment of the method may include multiple sub-steps or multiple stages, this is a little
Step or stage are not necessarily to execute completion in synchronization, but can execute at different times, these sub-steps
Either the execution sequence in stage be also not necessarily carry out successively but can with the sub-step of other steps or other steps or
At least part in person's stage executes in turn or alternately.
Based on thought identical with the evaluating method of voice interactive system IQ level in above-described embodiment, also carry herein
For the evaluating apparatus of voice interactive system IQ level.
In one embodiment, as shown in figure 4, the evaluating apparatus packet of the voice interactive system IQ level of the present embodiment
It includes：Problem acquisition module 410, cycle count module 420 and result determining module 430, details are as follows for each module：
Above problem acquisition module 410, for obtaining test problem, the test problem is phonetic problem.
Above-mentioned cycle count module 420, the finger for obtaining test problem described in the voice interactive system success response
Enable periodicity；Wherein, an instruction cycle is the voice interactive system and an extraneous dialog turns.
The above results determining module 430 determines the voice interactive system IQ water for passing through described instruction periodicity
Flat evaluation result.
In one embodiment, the above results determining module 430 includes：
Cost accounting submodule, for pass through described instruction periodicity determine the test problem corresponding instruction cycle at
This.As a result determination sub-module, the evaluation and test for determining the voice interactive system IQ level by described instruction life cycle costing
As a result.
In one embodiment, above problem acquisition module 410：Using the preset multiple test problems of acquisition.Accordingly, on
Result determining module 430 is stated, for according to the corresponding instruction cycle cost of the multiple test problem, determining the voice
The evaluation result score of interactive system IQ level.
In one embodiment, the above results determination sub-module, including：
Weight Acquisition unit, for obtaining the corresponding weight of the multiple test problem.And it as a result calculates single
Member determines the voice interactive system using according to the corresponding instruction cycle cost of the multiple test problem and weight
The evaluation result score of IQ level.
In one embodiment, above-mentioned Weight Acquisition unit, for obtaining the corresponding type of the multiple test problem；
All types of corresponding weights are obtained, as the corresponding weight of corresponding test problem.
In one embodiment, above-mentioned cost accounting submodule includes：
Limit value acquiring unit, for obtaining preset periodicity limit value；And
Cost calculation unit, for by described instruction periodicity and the periodicity limit value, determining the test problem
Corresponding instruction cycle cost.
In one embodiment, above-mentioned cost calculation unit includes：
First cost calculation unit determines the corresponding instruction of the test problem if being N1 for described instruction periodicity
Life cycle costing is cost score S1；Second cost calculation unit determines that the test is asked if being N2 for described instruction periodicity
It is cost score S2 to inscribe corresponding instruction cycle cost；Third cost calculation unit, if being N3 for described instruction periodicity, really
It is cost score S3 to determine the corresponding instruction cycle cost of the test problem.
Wherein, 0<N1<N2<M≤N3, N1 are preset periodicity, and M indicates periodicity limit value；S3<S2<S1.
In another embodiment, the first cost calculation unit determines the test if being N1 for described instruction periodicity
The corresponding instruction cycle cost of problem is cost level L1；Second cost calculation unit, if being N2 for described instruction periodicity,
Determine that the corresponding instruction cycle cost of the test problem is cost level L2；Third cost calculation unit, if being used for the finger
It is N3 to enable periodicity, determines that the corresponding instruction cycle cost of the test problem is cost level L3；
Wherein, 0<N1<N2<M≤N3, N1 are preset periodicity, and M indicates periodicity limit value；L1, L2, L3 are corresponded to
Rank successively decrease successively.
The specific restriction of evaluating apparatus about voice interactive system IQ level may refer to hand over above for voice
The restriction of the evaluating method of mutual system IQ level, details are not described herein.The evaluation and test of above-mentioned voice interactive system IQ level fills
Modules in setting can be realized fully or partially through software, hardware and combinations thereof.Above-mentioned each module can be in the form of hardware
It is embedded in or independently of the storage that in the processor in computer equipment, can also be stored in a software form in computer equipment
In device, the corresponding operation of the above modules is executed in order to which processor calls.
In addition, in the embodiment of the evaluating apparatus of the voice interactive system IQ level of above-mentioned example, each program module
Logical partitioning be merely illustrative of, can be as needed in practical application, for example, for corresponding hardware configuration requirement or
The convenient of the realization of software considers, above-mentioned function distribution is completed by different program modules, i.e., by the voice interactive system
The internal structure of the evaluating apparatus of IQ level is divided into different program modules, described above all or part of to complete
Function.
In one embodiment, a kind of computer equipment is provided, which can be terminal, internal structure
Figure can be as shown in Figure 5.The computer equipment includes processor, memory, display screen and the input connected by system bus
Device.Wherein, the processor of the computer equipment is for providing calculating and control ability.The memory of the computer equipment includes
Non-volatile memory medium, built-in storage.The non-volatile memory medium is stored with operating system and computer program.The memory
Reservoir provides environment for the operation of operating system and computer program in non-volatile memory medium.The computer program is located
It manages when device executes to realize a kind of evaluating method of voice interactive system IQ level.The display screen of the computer equipment can be
The input unit of liquid crystal display or electric ink display screen, the computer equipment can be the touch covered on display screen
Layer, can also be the button being arranged on computer equipment shell, trace ball or Trackpad, can also be external keyboard, touch-control
Plate or mouse etc..
It will be understood by those skilled in the art that structure shown in Fig. 5, is only tied with the relevant part of application scheme
The block diagram of structure does not constitute the restriction for the computer equipment being applied thereon to application scheme, specific computer equipment
May include either combining certain components than more or fewer components as shown in the figure or being arranged with different components.
In one embodiment, a kind of computer equipment, including memory and processor are provided, is stored in memory
Computer program, the processor can realize the voice interactive system IQ water of any of the above-described embodiment when executing computer program
The step of flat evaluating method.
One of ordinary skill in the art will appreciate that realizing all or part of flow in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the computer program can be stored in a non-volatile computer
In read/write memory medium, which can realize that the voice of any of the above-described embodiment is handed over when being executed by processor
The step of evaluating method of mutual system IQ level.
Based on this, in one embodiment, a kind of computer readable storage medium is provided, be stored thereon with computer journey
Sequence, the program realize the step of the evaluating method of the voice interactive system IQ level of any of the above-described embodiment when being executed by processor
Suddenly.
Wherein, used in each embodiment provided herein to memory, storage, database or other media
Any reference may each comprise non-volatile and/or volatile memory.Nonvolatile memory may include read-only memory
(ROM), programming ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.It is volatile
Property memory may include random access memory (RAM) or external cache.By way of illustration and not limitation, RAM
It is available in many forms, such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram
(DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus
(Rambus) direct RAM (RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram
(RDRAM) etc..
Each technical characteristic of above example can be combined arbitrarily, to keep description succinct, not to above-described embodiment
In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance
Shield is all considered to be the range of this specification record.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment
Point, it may refer to the associated description of other embodiments.
The term " comprising " and " having " of embodiment hereof and their any deformations, it is intended that cover non-exclusive packet
Contain.Such as contain series of steps or the process, method, system, product or equipment of (module) unit are not limited to arrange
The step of going out or unit, but further include the steps that optionally not listing or unit, or further include optionally for these mistakes
The intrinsic other steps of journey, method, product or equipment or unit.
Referenced herein " multiple " refer to two or more."and/or", the association for describing affiliated partner are closed
System indicates may exist three kinds of relationships, for example, A and/or B, can indicate：Individualism A exists simultaneously A and B, individualism
These three situations of B.It is a kind of relationship of "or" that character "/", which typicallys represent forward-backward correlation object,.
Referenced herein " first second " be only be the similar object of difference, do not represent for the specific of object
Sequence, it is possible to understand that specific sequence or precedence can be interchanged in ground, " first second " in the case of permission.It should manage
The object that solution " first second " is distinguished can be interchanged in the appropriate case so that the embodiments described herein can in addition to
Here the sequence other than those of diagram or description is implemented.
The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously
It cannot therefore be construed as limiting the scope of the patent.It should be pointed out that coming for those of ordinary skill in the art
It says, under the premise of not departing from the application design, various modifications and improvements can be made, these belong to the protection of the application
Range.Therefore, the protection domain of the application patent should be determined by the appended claims.
Claims (12)
1. a kind of evaluating method of voice interactive system IQ level, which is characterized in that including：
Test problem is obtained, the test problem is phonetic problem；
Obtain the instruction cycles of test problem described in the voice interactive system success response；Wherein, an instruction cycle is
The voice interactive system and an extraneous dialog turns；
The evaluation result of the voice interactive system IQ level is determined by described instruction periodicity.
2. according to the method described in claim 1, it is characterized in that, described determine that the voice is handed over by described instruction periodicity
The evaluation result of mutual system IQ level, including：
The corresponding instruction cycle cost of the test problem is determined by described instruction periodicity；
The evaluation result of the voice interactive system IQ level is determined by described instruction life cycle costing.
3. according to the method described in claim 2, it is characterized in that, the acquisition test problem, including：It obtains preset multiple
Test problem；
The evaluation result that the voice interactive system IQ level is determined by described instruction life cycle costing, including：
According to the corresponding instruction cycle cost of the multiple test problem, the voice interactive system IQ level is determined
Evaluation result score.
4. according to the method described in claim 3, it is characterized in that, further including：It is described according to the multiple test problem respectively
Corresponding instruction cycle cost determines the evaluation result score of the voice interactive system IQ level, including：
Obtain the corresponding weight of the multiple test problem；
According to the corresponding instruction cycle cost of the multiple test problem and weight, the voice interactive system IQ is determined
Horizontal evaluation result.
5. according to the method described in claim 4, it is characterized in that, described obtain the multiple corresponding power of test problem
Weight, including：
Obtain the corresponding type of the multiple test problem；
All types of corresponding weights are obtained, as the corresponding weight of corresponding test problem.
6. according to any method of claim 2 to 5, which is characterized in that described to determine institute by described instruction periodicity
The corresponding instruction cycle cost of test problem is stated, including：
Obtain preset periodicity limit value；
By described instruction periodicity and the periodicity limit value, the corresponding instruction cycle cost of the test problem is determined.
7. according to the method described in claim 6, it is characterized in that, described limited by described instruction periodicity and the periodicity
Value, determines the corresponding instruction cycle cost of the test problem, including：
If described instruction periodicity is N1, determine that the corresponding instruction cycle cost of the test problem is cost score S1；
If described instruction periodicity is N2, determine that the corresponding instruction cycle cost of the test problem is cost score S2；
If described instruction periodicity is N3, determine that the corresponding instruction cycle cost of the test problem is cost score S3；
Alternatively,
If described instruction periodicity is N1, determine that the corresponding instruction cycle cost of the test problem is cost level L1；
If described instruction periodicity is N2, determine that the corresponding instruction cycle cost of the test problem is cost level L2；
If described instruction periodicity is N3, determine that the corresponding instruction cycle cost of the test problem is cost level L3；
Wherein, 0<N1<N2<M≤N3, N1 are preset periodicity, and M indicates periodicity limit value；S3<S2<S1, L1, L2, L3
Corresponding rank is successively decreased successively.
8. the method according to the description of claim 7 is characterized in that
When described instruction periodicity is between N1 and M, with being incremented by for described instruction periodicity, the value of cost score S2 is passed
Subtract.
9. according to the method described in claim 8, it is characterized in that, the value of the cost score S2 successively decrease including：Linearly pass
Subtract, at least one of exponential damping, logarithmic decrement.
10. a kind of evaluating apparatus of voice interactive system IQ level, which is characterized in that including：
Problem acquisition module, for obtaining test problem, the test problem is phonetic problem；
Cycle count module, the instruction cycles for obtaining test problem described in the voice interactive system success response；Its
In, an instruction cycle is the voice interactive system and an extraneous dialog turns；
And
As a result determining module, the evaluation and test knot for determining the voice interactive system IQ level by described instruction periodicity
Fruit.
11. a kind of computer equipment, including memory, processor and storage are on a memory and the meter that can run on a processor
Calculation machine program, which is characterized in that the processor realizes the step of claim 1 to 9 any the method when executing described program
Suddenly.
12. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The step of claim 1 to 9 any the method is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810433832.0A CN108763329A (en) | 2018-05-08 | 2018-05-08 | Evaluating method, device and the computer equipment of voice interactive system IQ level |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810433832.0A CN108763329A (en) | 2018-05-08 | 2018-05-08 | Evaluating method, device and the computer equipment of voice interactive system IQ level |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108763329A true CN108763329A (en) | 2018-11-06 |
Family
ID=64009222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810433832.0A Pending CN108763329A (en) | 2018-05-08 | 2018-05-08 | Evaluating method, device and the computer equipment of voice interactive system IQ level |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108763329A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109545185A (en) * | 2018-11-12 | 2019-03-29 | 百度在线网络技术（北京）有限公司 | Interactive system evaluation method, evaluation system, server and computer-readable medium |
| CN110009206A (en) * | 2019-03-21 | 2019-07-12 | 五邑大学 | A kind of timing speech assessment method, apparatus, equipment and storage medium |
| CN110619868A (en) * | 2019-08-29 | 2019-12-27 | 深圳市优必选科技股份有限公司 | Voice assistant optimization method, voice assistant optimization device and intelligent equipment |
| CN110689903A (en) * | 2019-09-24 | 2020-01-14 | 百度在线网络技术（北京）有限公司 | Method, device, equipment and medium for evaluating intelligent sound box |
| CN112383451A (en) * | 2020-11-30 | 2021-02-19 | 天津链数科技有限公司 | Intelligent household appliance intelligent level testing system and method based on voice interaction |
| CN113489846A (en) * | 2021-06-30 | 2021-10-08 | 未鲲(上海)科技服务有限公司 | Voice interaction testing method, device, equipment and computer storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8296144B2 (en) * | 2008-06-04 | 2012-10-23 | Robert Bosch Gmbh | System and method for automated testing of complicated dialog systems |
| US8793119B2 (en) * | 2009-07-13 | 2014-07-29 | At&T Intellectual Property I, L.P. | System and method for generating manually designed and automatically optimized spoken dialog systems |
| CN104464757A (en) * | 2014-10-28 | 2015-03-25 | 科大讯飞股份有限公司 | Voice evaluation method and device |
| CN105068661A (en) * | 2015-09-07 | 2015-11-18 | 百度在线网络技术（北京）有限公司 | Man-machine interaction method and system based on artificial intelligence |
| CN107688533A (en) * | 2017-07-25 | 2018-02-13 | 上海壹账通金融科技有限公司 | Applied program testing method, device, computer equipment and storage medium |
-
2018
- 2018-05-08 CN CN201810433832.0A patent/CN108763329A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8296144B2 (en) * | 2008-06-04 | 2012-10-23 | Robert Bosch Gmbh | System and method for automated testing of complicated dialog systems |
| US8793119B2 (en) * | 2009-07-13 | 2014-07-29 | At&T Intellectual Property I, L.P. | System and method for generating manually designed and automatically optimized spoken dialog systems |
| CN104464757A (en) * | 2014-10-28 | 2015-03-25 | 科大讯飞股份有限公司 | Voice evaluation method and device |
| CN105068661A (en) * | 2015-09-07 | 2015-11-18 | 百度在线网络技术（北京）有限公司 | Man-machine interaction method and system based on artificial intelligence |
| CN107688533A (en) * | 2017-07-25 | 2018-02-13 | 上海壹账通金融科技有限公司 | Applied program testing method, device, computer equipment and storage medium |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109545185A (en) * | 2018-11-12 | 2019-03-29 | 百度在线网络技术（北京）有限公司 | Interactive system evaluation method, evaluation system, server and computer-readable medium |
| CN109545185B (en) * | 2018-11-12 | 2022-03-18 | 百度在线网络技术（北京）有限公司 | Interactive system evaluation method, evaluation system, server, and computer-readable medium |
| CN110009206A (en) * | 2019-03-21 | 2019-07-12 | 五邑大学 | A kind of timing speech assessment method, apparatus, equipment and storage medium |
| CN110619868A (en) * | 2019-08-29 | 2019-12-27 | 深圳市优必选科技股份有限公司 | Voice assistant optimization method, voice assistant optimization device and intelligent equipment |
| CN110619868B (en) * | 2019-08-29 | 2021-12-17 | 深圳市优必选科技股份有限公司 | Voice assistant optimization method, voice assistant optimization device and intelligent equipment |
| CN110689903A (en) * | 2019-09-24 | 2020-01-14 | 百度在线网络技术（北京）有限公司 | Method, device, equipment and medium for evaluating intelligent sound box |
| CN110689903B (en) * | 2019-09-24 | 2022-05-13 | 百度在线网络技术（北京）有限公司 | Method, device, equipment and medium for evaluating intelligent sound box |
| CN112383451A (en) * | 2020-11-30 | 2021-02-19 | 天津链数科技有限公司 | Intelligent household appliance intelligent level testing system and method based on voice interaction |
| CN112383451B (en) * | 2020-11-30 | 2022-12-16 | 天津链数科技有限公司 | Intelligent household appliance intelligent level testing system and method based on voice interaction |
| CN113489846A (en) * | 2021-06-30 | 2021-10-08 | 未鲲(上海)科技服务有限公司 | Voice interaction testing method, device, equipment and computer storage medium |
| CN113489846B (en) * | 2021-06-30 | 2024-02-27 | 上海凌荣网络科技有限公司 | Voice interaction testing method, device, equipment and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108763329A (en) | Evaluating method, device and the computer equipment of voice interactive system IQ level | |
| CN109086329B (en) | Topic keyword guide-based multi-turn conversation method and device | |
| CN109949638A (en) | Acquisition of knowledge degree determines method, apparatus, terminal and medium | |
| CN110472224A (en) | Detection method, device, computer equipment and the storage medium of service quality | |
| CN112017056B (en) | Intelligent double-recording method and system | |
| CN110738998A (en) | Voice-based personal credit evaluation method, device, terminal and storage medium | |
| CN110458732A (en) | Training Methodology, device, computer equipment and storage medium | |
| CN107688533A (en) | Applied program testing method, device, computer equipment and storage medium | |
| CN109509088A (en) | Loan checking method, device, equipment and medium based on micro- Expression Recognition | |
| CN110532522A (en) | Error-detecting method, device, computer equipment and the storage medium of audio mark | |
| CN109389492A (en) | Method, system, equipment and the storage medium of probability of cheating are calculated based on micro- expression | |
| CN109473121A (en) | Speech synthesis quality detecting method and device | |
| US11170333B2 (en) | System and method for an adaptive competency assessment model | |
| CN111639218A (en) | Interactive method for spoken language training and terminal equipment | |
| CN117290694B (en) | Question-answering system evaluation method, device, computing equipment and storage medium | |
| CN112286485B (en) | Method and device for controlling application through voice, electronic equipment and storage medium | |
| CN112133289B (en) | Voiceprint identification model training method, voiceprint identification device, voiceprint identification equipment and voiceprint identification medium | |
| CN111309882B (en) | Method and device for realizing intelligent customer service question and answer | |
| CN115393077B (en) | Data processing method based on loan transaction man-machine conversation system and related device | |
| CN114330285B (en) | Corpus processing method and device, electronic equipment and computer readable storage medium | |
| CN116414959A (en) | Digital person interaction control method and device, electronic equipment and storage medium | |
| CN110265062A (en) | Collection method and device after intelligence based on mood detection is borrowed | |
| US11715470B2 (en) | Method and system for tracking in extended reality | |
| CN109388695A (en) | User's intension recognizing method, equipment and computer readable storage medium | |
| CN114595317A (en) | Method, device and equipment for generating reply text and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 511300 No.78, west of Zhucun Avenue, Zhucun street, Zengcheng District, Guangzhou City, Guangdong Province Applicant after: CHINA ELECTRONIC PRODUCT RELIABILITY AND ENVIRONMENTAL TESTING RESEARCH INSTITUTE ((THE FIFTH ELECTRONIC RESEARCH INSTITUTE OF MIIT)(CEPREI LABORATORY)) Address before: 510610 No. 110 Zhuang Road, Tianhe District, Guangdong, Guangzhou, Dongguan Applicant before: CHINA ELECTRONIC PRODUCT RELIABILITY AND ENVIRONMENTAL TESTING RESEARCH INSTITUTE ((THE FIFTH ELECTRONIC RESEARCH INSTITUTE OF MIIT)(CEPREI LABORATORY)) |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |
|
| RJ01 | Rejection of invention patent application after publication |