CN107608799A

CN107608799A - A kind of method, equipment and storage medium for being used to perform interactive instruction

Info

Publication number: CN107608799A
Application number: CN201710694514.5A
Authority: CN
Inventors: 张伟萌; 戴帅湘
Original assignee: Beijing Moran Cognitive Technology Co Ltd
Current assignee: Guangyulaite Digital Technology Shanghai Co ltd
Priority date: 2017-08-15
Filing date: 2017-08-15
Publication date: 2018-01-19
Anticipated expiration: 2037-08-15
Also published as: CN107608799B; CN109656655A

Abstract

It is an object of the invention to provide a kind of method, equipment and storage medium for being used to perform interactive instruction.Specifically, the user mutual instruction of user's input is obtained；The corresponding unified interactive instruction of user mutual instruction is determined, wherein, instruct the interactive instruction under synonymous multiple distinct interaction patterns corresponding with unified interactive instruction with user mutual；According to unified interactive instruction, user mutual instruction is performed.Compared with prior art, the present invention realizes so that no matter which kind of interactive mode user uses, can accurately it perform, reduce dependence of the equipment to interactive mode, also achieve and support being used in mixed way for a variety of distinct interaction patterns, and various distinct interaction patterns are seamlessly connected, the interbehavior of user is not construed as limiting, the instruction unified representation of synonymous all kinds of interactive modes, user does not have to concern that distinct interaction pattern can cause dialogue to be isolated and can not be connected, make entirely to interact more remarkable fluency, improve interactive efficiency, also improve interactive experience.

Description

A kind of method, equipment and storage medium for being used to perform interactive instruction

Technical field

The present invention relates to Multimodal interaction technical field, more particularly to a kind of technology for being used to perform interactive instruction.

Background technology

The interactive mode of current equipment tends to variation, except traditional Mechanical course (such as remote control), can also pass through The modes such as voice, image, gesture interact.A variety of interactive modes are mixed in together, are substantially at the state each operated, not mutually Association so that it is difficult accurate understanding user view that system is converted when interactive mode is inputted in user, or can not perform use Family instructs.For example, user first passes through phonetic search film, then lower one page is turned to remote control, at this moment said and " broadcast with phonetic order again Put first " when, system is not aware that current page state has occurred that change, can still play the first of homepage It is individual, can not meet the needs of user, reduce interactive experience.

The content of the invention

It is an object of the present invention to provide a kind of method and apparatus for being used to perform interactive instruction.

According to an aspect of the invention, there is provided a kind of method for performing interactive instruction, wherein, this method includes Following steps：

A obtains the user mutual instruction of user's input；

B determines the corresponding unified interactive instruction of the user mutual instruction, wherein, it is same with user mutual instruction Interactive instruction under multiple distinct interaction patterns of justice is corresponding with the unified interactive instruction；

C performs the user mutual instruction according to the unified interactive instruction.

According to another aspect of the present invention, a kind of execution equipment for being used to perform interactive instruction is additionally provided, wherein, this is held Row equipment includes：

First acquisition device, for obtaining the user mutual instruction of user's input；

First determining device, the unified interactive instruction corresponding for determining user mutual instruction, wherein, it is and described User mutual instructs the interactive instruction under synonymous multiple distinct interaction patterns corresponding with the unified interactive instruction；

Performs device, for according to the unified interactive instruction, performing the user mutual instruction.

According to another aspect of the invention, a kind of computing device is additionally provided, including：

One or more processors；

Memory, for storing one or more programs,

When one or more of programs are by one or more of computing devices so that one or more of places Reason device is performed such as a kind of foregoing method for being used to perform interaction execution according to one aspect of the invention.

In accordance with a further aspect of the present invention, there is provided a kind of computer-readable recording medium, be stored thereon with computer journey Sequence, wherein, realize that foregoing one kind according to one aspect of the invention such as is used to perform interactive hold when the program is executed by processor Capable method.

Compared with prior art, one embodiment of the present of invention is by determining corresponding to the user mutual instruction of user's input Unified interactive instruction, wherein, interactive instruction and the institute under synonymous multiple distinct interaction patterns are instructed with the user mutual It is corresponding to state unified interactive instruction；According to the unified interactive instruction, the user mutual instruction is performed so that no matter user adopts It with which kind of interactive mode, can accurately perform, reduce dependence of the equipment to interactive mode；Correspondingly, support is also achieved A variety of distinct interaction patterns are used in mixed way, and various distinct interaction patterns are seamlessly connected, and the interbehavior of user are not formed Limitation so that user can use any interactive mode in (more wheels) dialog procedure；Meanwhile synonymous all kinds of interactive modes Unified representation is instructed, user does not have to concern that distinct interaction pattern can cause dialogue to be isolated and can not be connected, and makes whole interaction more smooth Naturally, improving interactive efficiency, interactive experience is also improved.

Brief description of the drawings

By reading the detailed description made to non-limiting example made with reference to the following drawings, of the invention is other Feature, objects and advantages will become more apparent upon：

Fig. 1 shows to be illustrated according to a kind of equipment for being used to perform the execution equipment of interactive instruction of one aspect of the invention Figure；

Fig. 2 (a)-(c) shows that the distinct interaction pattern of one embodiment of the invention is seamlessly connected schematic diagram；

Fig. 3 shows a kind of equipment schematic diagram for being used to perform the execution equipment of interactive instruction of one embodiment of the invention；

Fig. 4 shows a kind of method flow diagram for being used to perform interactive instruction according to a further aspect of the present invention；

Fig. 5 shows a kind of method flow diagram for being used to perform interactive instruction of one embodiment of the invention；

Fig. 6 shows the block diagram suitable for being used for the exemplary computer system/server for realizing embodiment of the present invention.

Same or analogous reference represents same or analogous part in accompanying drawing.

Embodiment

The present invention is described in further detail below in conjunction with the accompanying drawings.

Fig. 1 shows a kind of execution equipment 1 for being used to perform interactive instruction according to one aspect of the invention, wherein, perform Equipment 1 includes the first acquisition device 11, the first determining device 12 and performs device 13.Specifically, the first acquisition device 11 obtains The user mutual instruction of user's input；First determining device 12 determines that the corresponding unified interaction of the user mutual instruction refers to Order, wherein, instruct the interactive instruction under synonymous multiple distinct interaction patterns to refer to unified interact with the user mutual Make corresponding；Performs device 13 performs the user mutual instruction according to the unified interactive instruction.

Here, the equipment 1 that performs refers to a kind of unified interactive instruction that can determine that user mutual instruction is corresponding, and The equipment that user mutual instruction is performed with this.In a particular embodiment, performing equipment 1 can be realized by intelligent terminal, also can be by The network equipment or the network equipment are integrated formed equipment (i.e. by intelligent terminal and the network equipment with intelligent terminal by network It is engaged) realize, it is also used as software module and/or hardware module is contained in intelligent terminal, hardware can also be used as Equipment is connected with intelligent terminal by wired or wireless mode.Here, the network equipment includes but is not limited to such as network master Machine, single network server, multiple webserver collection or set of computers based on cloud computing etc. are realized.Here, Yun Youji Formed in a large amount of main frames or the webserver of cloud computing (Cloud Computing), wherein, cloud computing is Distributed Calculation One kind, a super virtual computer being made up of the computer collection of a group loose couplings.Here, the intelligent terminal can be Any one can pass through the one or more such as keyboard, touch pad, touch-screen, remote control, interactive voice or handwriting equipment with user Mode carries out the electronic product of man-machine interaction, for example, PC, mobile phone, smart mobile phone, PDA, wearable device, palm PC PPC, can Wearable device, tablet personal computer, intelligent vehicle device, intelligent television, intelligent sound box etc..In actual applications, it is whole for intelligence to perform equipment 1 During end, can carry/install thereon it will be appreciated that, the natural language instructions that handle and respond user and response results are exported Client (can be APP forms) or the client be only capable of to user input natural language instructions carry out voice Identification but server corresponding to needing is understood, handled and responded the natural language instructions of user to the natural language instructions simultaneously Response results return client is exported.The network include but is not limited to internet, wide area network, Metropolitan Area Network (MAN), LAN, VPN, wireless self-organization network (Ad Hoc networks) etc..Those skilled in the art will be understood that above-mentioned execution equipment 1 is only Citing, other network equipments or intelligent terminal existing or be likely to occur from now on are such as applicable to the present invention, should be also included in Within the scope of the present invention, and it is incorporated herein by reference herein.Here, the network equipment and intelligent terminal include one kind Can be according to the instruction for being previously set or storing, the automatic electronic equipment for carrying out numerical computations and information processing, its hardware includes But it is not limited to microprocessor, application specific integrated circuit (ASIC), programmable gate array (FPGA), digital processing unit (DSP), embedded Equipment etc..

In one embodiment, if performing the intelligent terminal that equipment 1 is user, perform equipment 1 and pass through its own first The application programming interfaces (API) provided, obtain the user mutual instruction of user's input；It is then determined that the user mutual instruction Corresponding unified interactive instruction, wherein, instruct interacting under synonymous multiple distinct interaction patterns to refer to the user mutual Order is corresponding with the unified interactive instruction；Then, further according to the unified interactive instruction, the user mutual instruction is performed.

In another embodiment, if execution equipment 1 is the network equipment, perform what the intelligent terminal first of equipment 1 was provided Application programming interfaces (API), obtain the user mutual instruction of user's input；It is then determined that corresponding to the user mutual instruction Unified interactive instruction, wherein, instructed with the user mutual interactive instruction under synonymous multiple distinct interaction patterns with it is described Unified interactive instruction is corresponding；Then, further according to the unified interactive instruction, the user mutual instruction is performed.

In a further embodiment, if it is the equipment that the network equipment and intelligent terminal are integrated to perform equipment 1, that is, equipment is performed 1 is engaged realization by intelligent terminal and the network equipment, then the application programming interfaces that intelligent terminal is provided by its own first (API) the user mutual instruction of user's input, is obtained；Then, user mutual instruction is sent to the corresponding network equipment, by The network equipment determines the corresponding unified interactive instruction of user mutual instruction, wherein, instructed with the user mutual synonymous Interactive instruction under multiple distinct interaction patterns is corresponding with the unified interactive instruction；Then, further according to the unified interaction Instruction, perform the user mutual instruction.

In yet another embodiment, if it is the equipment that the network equipment and intelligent terminal are integrated to perform equipment 1, that is, perform and set Standby 1 is engaged realization by intelligent terminal and the network equipment, then the application program that intelligent terminal is provided by its own first connects Mouth (API), obtain the user mutual instruction of user's input；Then, it is determined that the unified interaction that the user mutual instruction is corresponding Instruction, wherein, instruct the interactive instruction under synonymous multiple distinct interaction patterns to be interacted with the unification with the user mutual Instruct corresponding；Then, the unified interactive instruction is sent to the corresponding network equipment, with by the network equipment according to the system One interactive instruction, perform the user mutual instruction.

Specifically, the application programming interfaces (API) that the first acquisition device 11 is provided by performing equipment 1 itself, obtain and use The user mutual instruction of family input, or the user mutual sent by receiving the third party devices such as intelligent terminal instruct, To obtain the user mutual instruction.Here, user mutual instruction includes but is not limited to such as in interactive voice, remote control is interactive, The instruction inputted under any one interactive mode such as gesture interaction, image interaction, vocal print interaction, body feeling interaction.If for example, user Interactive instruction is natural language instructions, then the sound pick up equipment such as microphone that the first acquisition device 11 can be by performing equipment 1 To obtain the natural language instructions of user.For another example, if user mutual instruction is the image of input, such as recognition of face, then first obtain Take device 11 to gather image or video flowing containing face by the video camera or camera for performing equipment 1, and scheming automatically The detect and track face as in, and then carry out recognition of face.Also such as, if user mutual instruction is the finger inputted by remote control Order, the then message that the first acquisition device 11 can be sent by receiving user by the button on remote control, to obtain the user Interactive instruction.And for example, if user mutual instruction is the instruction inputted by performing gesture operation, the first acquisition device 11 can lead to Cross and perform the application programming interfaces (API) that equipment 1 itself provides, get the gesture path that user performs on the touchscreen, from And get corresponding gesture operation.

Then, the first determining device 12 determines the corresponding unified interactive instruction of the user mutual instruction, wherein, with institute Stating user mutual instructs the interactive instruction under synonymous multiple distinct interaction patterns corresponding with the unified interactive instruction. This, the interactive mode includes but is not limited to such as interactive voice, remote control interaction, gesture interaction, image interaction, vocal print are interactive, Body feeling interaction etc..Here, it is described it is synonymous refer to instruction effect it is identical.Here, the unified interactive instruction refers to multiple synonymous Distinct interaction pattern under interactive instruction carry out unification, and obtained unified interactive instruction, for example, for page-turning instruction, such as Voice " lower one page " " anything else ", the turnover key of remote control, on user's point touching screen the page shaped likeLower one page/ Linking buttons such as [lower one pages], because the instruction effect of such instruction is identical, unification can be carried out to such instruction, such as set a kind of New instruction format come under various interactive modes instruction effect identical instruction carry out unified representation so that no matter what Instruction under kind interactive mode is executable, is not limited by interactive mode.

Here, the first determining device 12 determines that the mode of the corresponding unified interactive instruction of user mutual instruction is included but not It is limited to following at least any one：

1) the synonymous interactive instruction cluster according to belonging to instructing the user mutual, determine corresponding to the user mutual instruction Unified interactive instruction, wherein, interactive instruction and the institute under synonymous multiple distinct interaction patterns are instructed with the user mutual It is corresponding to state unified interactive instruction.

Here, the synonymous interactive instruction cluster refers to the interactive instruction under one group of synonymous multiple distinct interaction pattern.

For example, it is assumed that it is electronic product to perform equipment 1, certain APP is installed thereon, as user A is intended to log in the APP, its dozen After opening the APP, the login of brush face is clicked, then opens recognition of face interactive mode, under the interactive mode, triggering unlatching is held Video camera or camera in row equipment 1, so as to carry out face collection, and automatic detect and track people in the picture to user A Face, and then recognition of face is carried out, user A facial image is recognized, this is the user mutual instruction of user A inputs, it is assumed that should (such as synonymous interactive instruction cluster also act as other interaction moulds " logged in " to synonymous interactive instruction cluster belonging to instruction comprising instruction Formula gives an order as logged in two-dimensional code scanning, voice " login " etc.) corresponding to unified interactive instruction be { instruct:Log in, Then the first determining device 12 can determine that corresponding unified interactive instruction is { instruct:Log in }.

For another example, it is assumed that user A is intended to be paid, and in the case where paying scene, acquires user A sound, and is verified, Such as by the comparison discriminating of feature in terms of the user A collected sound and the progress sense of hearing of known sound and frequency spectrum two and total score Analysis, it is determined that whether the two comes from same sound source (sound of same person), to verify sounder identity, it is assumed that be verified, phase Ying Di, the user A collected sound are the user mutual instruction of user A inputs, and the synonymous interaction belonging to the instruction refers to Making cluster, (other interactive modes that such as the synonymous interactive instruction cluster also act as " paying " comprising instruction give an order such as Quick Response Code of paying the bill Scanning, voice " payment " etc.) corresponding to unified interactive instruction be { pay:Paying, then the first determining device 12 can determine that accordingly Unified interactive instruction be { pay:Pay }.

Also such as, it is assumed that user A is intended to carry out voice lookup most by performing equipment 1 (assuming that it is intelligent television to perform equipment 1) Nearly good-looking film, it is assumed that client such as Voice-APP, user A are equipped with the intelligent television and opens the client, Huo Zhetong Cross voice to wake up it, or directly send what good-looking electricity is natural language instructions " have recently to the intelligent television Shadow ", list-1 of the results list returned as shown in Fig. 2 (a), now user A turned over further through TV remote controller Page operations, the button for turning to lower one page (shown in such as Fig. 2 (b)) can be realized on remote control as clicked on, and it is same belonging to the instruction (such as the synonymous interactive instruction cluster also act as the finger under " turning to lower one page " other interactive modes to adopted interactive instruction cluster comprising instruction Order, as on voice " lower one page ", user's point touching screen the page shaped likeLower one pageThe linking buttons such as/[lower one page]) it is corresponding Unified interactive instruction be { page down:Lower one page }, then the first determining device 12 can determine that corresponding unified interactive instruction is {page down:Lower one page }.

2) according to the instruction format of predetermined unified interactive instruction, the form that the user mutual instructs is converted to and met The instruction format of unified interactive instruction, so as to obtain the corresponding unified interactive instruction of the user mutual instruction.

Specifically, the converter of corresponding every kind of interactive mode can be set, i.e., the finger under corresponding interactive mode is done by converter Order conversion, the conversion regime of every kind of converter are had nothing in common with each other.For example, for remote control interactive mode and gesture interaction pattern, can Performed previous instruction (its of the user mutual instruction is sent using remote control/gesture by current page state, user In, the instruction can be that user is sent in dialog procedure using any one interactive mode) and remote controller key operation/hand Gesture operation triple is (i.e. for remote control interactive mode, using { current page state, prior instruction, remote controller key operate } Triple, for gesture interaction pattern, using { current page state, prior instruction, gesture operation } triple), by tabling look-up Mode come determine to be transformed into it is specific which unify interactive instruction；And for speech interaction mode, need to be first by natural-sounding order Switch to text, the natural language instructions are then mapped to by unified interactive instruction by disaggregated model；For image interactive mode With vocal print interactive mode, converter inside can be a service, such as image recognition or Application on Voiceprint Recognition, identify and be successfully then mapped to Login successfully and/or pay successfully.Here, the page status is consistent with page status implication hereinafter.

Those skilled in the art will be understood that the corresponding unified interactive instruction of the above-mentioned determination user mutual instruction Mode is only for example, and the corresponding unified interaction of user mutual instruction described in other existing or determinations for being likely to occur from now on refers to The mode of order is such as applicable to the present invention, should also be included within the scope of the present invention, and be contained in by reference herein This.

Then, performs device 13 performs the user mutual instruction, the system as described in performing according to the unified interactive instruction One interactive instruction, so as to realize the execution to user mutual instruction, i.e., by performing the unified interactive instruction, reach with Perform the user mutual instruction identical implementation effect.

For example, connecing precedent, performs device 13 is as performed unified interactive instruction { instruct:Log in, then can Successful login.

For another example, performs device 13 is as performed unified interactive instruction { pay:Pay, then can successfully it pay.

Also such as, performs device 13 is as performed unified interactive instruction { page down:Lower one page }, then lower one page can be turned to, is obtained To lower one page film the results list as shown in Fig. 2 (b).

It is seamlessly connected here, the present invention realizes various distinct interaction patterns so that no matter user is using which kind of interaction mould It formula, can accurately perform, reduce dependence of the equipment to interactive mode；Correspondingly, also achieve and support a variety of distinct interactions Pattern is used in mixed way, and the interbehavior of user is not construed as limiting so that user can use in (more wheels) dialog procedure to be appointed The interactive mode of meaning；Meanwhile the instruction unified representation of synonymous all kinds of interactive modes, user distinct interaction pattern is not had to concern Dialogue can be caused, which to isolate, to be connected, and made entirely to interact more remarkable fluency, improved interactive efficiency, also improve interactive experience. For example, after user A sees lower one page film the results list as shown in Fig. 2 (b), voice says " I will see that goddess is very chivalrous " (such as Fig. 2 (c) shown in), correspondingly, the film can be commenced play out by performing equipment 1.

In one embodiment, equipment 1 is performed including the second acquisition device (not shown), the second determining device (not show also Go out), establish or updating device (not shown) and the 3rd determining device (not shown).Specifically, the second acquisition device obtains one Or multiple synonymous interactive instruction clusters, wherein, the synonymous interactive instruction cluster includes representing under synonymous multiple distinct interaction patterns Interactive instruction；Second determining device determines the unified interactive instruction corresponding to the synonymous interactive instruction cluster；Establish or update Device establish or renewal multi-module interactive instruction database, wherein, the multi-module interactive instruction database include the synonymous interactive instruction cluster with The mapping relations between unified interactive instruction corresponding to it；The user mutual that first acquisition device 11 obtains user's input refers to Order；The instruction description information that 3rd determining device instructs according to the user mutual, carried out in the multi-module interactive instruction database Matching inquiry, to determine the synonymous interactive instruction cluster belonging to the user mutual instruction；First determining device 12 is according to the use Synonymous interactive instruction cluster belonging to the interactive instruction of family, the corresponding unified interactive instruction of the user mutual instruction is determined, wherein, Instruct the interactive instruction under synonymous multiple distinct interaction patterns corresponding with the unified interactive instruction with the user mutual； Performs device 13 performs the user mutual instruction according to the unified interactive instruction.

Here, the first acquisition device 11 and performs device 13 be identical with the content of corresponding intrument in previous embodiment or phase Seemingly, for simplicity, it therefore will not be repeated here, and be incorporated herein by reference.

Here, synonymous interactive instruction cluster of first determining device 12 according to belonging to instructing the user mutual determines the use The mode of unified interactive instruction corresponding to the interactive instruction of family, with the first determining device 12 in previous embodiment according to the user Synonymous interactive instruction cluster belonging to interactive instruction determines the mode phase of the corresponding unified interactive instruction of the user mutual instruction It is same or similar, for simplicity, therefore it will not be repeated here, and be incorporated herein by reference.

Specifically, the second acquisition device obtains one or more synonymous interactive instruction clusters, wherein, the synonymous interactive instruction Cluster includes representing the interactive instruction under synonymous multiple distinct interaction patterns.Here, the second acquisition device obtain it is one or The mode of multiple synonymous interactive instruction clusters includes but is not limited to following at least any one：

1) acted on according to the instruction of interactive instruction, the interaction in each interactive mode respectively corresponding interactive instruction storehouse is referred to Order carries out classification processing, to obtain one or more of synonymous interactive instruction clusters.

For example, the second acquisition device obtains entirely from the corresponding interactive instruction storehouse of predetermined each interactive mode difference first Portion's interactive instruction, then acted on according to the instruction of the interactive instruction of mark, using sorting techniques such as decision trees, to such friendship Mutually instruction is classified, by the interactive instruction shape being divided under multiple distinct interaction patterns of same class (i.e. instruction effect is identical) Into a synonymous interactive instruction cluster, so as to obtain multiple synonymous interactive instruction clusters.

2) according to multiple interactive instruction training samples under the distinct interaction pattern for having marked synonymy, to the multiple Interactive instruction training sample carries out clustering processing, to obtain one or more of synonymous interactive instruction clusters.

It is for example, it is assumed that as follows in the presence of multiple interactive instruction training samples under the distinct interaction pattern for having marked synonymy It is shown：

1. instruction act as the synonymous instruction of " turning to lower one page "：Voice " lower one page " and " anything else " etc., use On the point touching screen of family the page shaped like lower one page/linking button such as [lower one page], turn to the button of lower one page on remote control, turn over Page gesture

2. instruction act as the synonymous instruction of " amplification of volume "：Voice " amplification of volume ", " volume again big a bit ", volume increase The button added

3. instruction act as the synonymous instruction of " amplification screen display content "：Amplifying gesture, amplifying key-press, voice " amplification "

4. instruction act as " the synonymous instruction of selection n-th "：" selection n-th ", remote control select n-th, touched voice Touch n-th of screen selection, " I will see * * * " to voice (wherein, * * * represent n-th of name, and n is natural number)

……

Then the second acquisition device can mark multiple interactive instructions instruction under the distinct interaction pattern of synonymy according to this Practice sample, clustering processing is carried out to the plurality of interactive instruction training sample, will such as mark the interactive instruction instruction for belonging to synonymy Practice sample and be classified as one kind, so as to obtain one or more of synonymous interactive instruction clusters, such as will mark synonymous pass containing above-mentioned The interactive instruction of the interactive instruction training sample of system 1. is classified as one kind and obtains cluster1, by the above-mentioned friendship for having marked synonymy Mutually the interactive instruction of instruction training sample 2. is classified as one kind and obtains cluster2, so as to obtain one or more of synonymous interactions Instruct cluster.

Those skilled in the art will be understood that the mode of the above-mentioned acquisition synonymous interactive instruction cluster is only for example, and other are existing The mode of acquisition that is having or the being likely to occur from now on synonymous interactive instruction cluster is such as applicable to the present invention, should also be included in this Within invention protection domain, and it is incorporated herein by reference herein.

Second determining device determines the unified interactive instruction corresponding to the synonymous interactive instruction cluster, is such as referred to unified interaction The instruction effect expressed to express the synonymous interactive instruction cluster to do is made, i.e., to multiple synonymous in the synonymous interactive instruction cluster Distinct interaction pattern under interactive instruction carry out unification, and obtained unified interactive instruction.

Establish or updating device is established or renewal multi-module interactive instruction database, as by establishing synonymous interactive instruction cluster and its institute Corresponding unified interactive instruction pair, to obtain multi-module interactive instruction database as shown in the following Table 1, wherein, the multi-module interactive refers to Make the mapping relations between unified interactive instruction of the storehouse including the synonymous interactive instruction cluster and corresponding to it.Here, to described The mode of multi-module interactive instruction database renewal can such as be updated by pre-defined rule by cycle mode.

Table 1

First acquisition device 11 obtains the user mutual instruction of user's input.

The instruction description information that 3rd determining device instructs according to the user mutual, in the multi-module interactive instruction database Matching inquiry is carried out, to determine the synonymous interactive instruction cluster belonging to the user mutual instruction.Here, the instruction description information It is the information for referring to that user mutual instruction is described, it can be the corresponding text of the user mutual instruction (if user mutual instruction is natural language instructions, then its instruction description information can be that the natural language is ordered to description information Order carries out text obtained from semantic parsing；If user mutual instruction is remote control interaction, gesture interaction, image are interactive, vocal print Interactive instruction under interaction isotype, then corresponding instruction description information can be the explanation text to instruction), can also It is main contents obtained from carrying out main contents extraction to text description information.

For example, for user A, it directly sends what good-looking electricity is natural language instructions " have recently to intelligent television Shadow ", list-1 of the results list returned as shown in Fig. 2 (a), now user A turned over further through TV remote controller Page operations, the button for turning to lower one page (shown in such as Fig. 2 (b)) can be realized on remote control as clicked on, then the first acquisition device 11 The application programming interfaces (API) provided by intelligent television, get the user mutual instruction of user A inputs；Then, The instruction description information that three determining devices instruct according to the user mutual, such as the explanation text to user mutual instruction " turning to lower one page ", matching inquiry is carried out in multi-module interactive instruction database as shown in table 1, obtained belonging to user mutual instruction Synonymous interactive instruction cluster be cluster1.

Fig. 3 shows a kind of execution equipment 1 for being used to perform interactive instruction of one embodiment of the invention, wherein, execution is set Standby 1 includes the first acquisition device 21, the first determining device 22, performs device 23 and provides device 24.Specifically, there is provided device 24 In response to first interactive instruction of the user under the first interactive mode, there is provided corresponding response contents；First acquisition device 21 obtain the second interactive instruction that the user performs under the second interactive mode to the response contents, to be used as the user Interactive instruction, wherein, second interactive mode is different from first interactive mode；First determining device 22 determines the use Unified interactive instruction corresponding to the interactive instruction of family, wherein, instruct synonymous multiple distinct interaction patterns with the user mutual Under interactive instruction it is corresponding with the unified interactive instruction；Performs device 23 is according to the unified interactive instruction, described in execution User mutual instructs.Here, the first determining device 22 and performs device 23 the content phase with corresponding intrument in Fig. 1 embodiments respectively It is same or similar, here, for simplicity, therefore will not be repeated here, and be incorporated herein by reference.

Specifically, there is provided first interactive instruction of the device 24 in response to the user under the first interactive mode, there is provided right The response contents answered.

For example, for user A, it directly have issued what good-looking electricity is natural language instructions " have recently to intelligent television Shadow ", then first interactive instruction that device 24 performs in response to user A is provided, corresponding response contents are obtained first, such as in electricity Inquired about in shadow database, obtain corresponding result, and with tabular form and returned, so as to which the results list is supplied into use Family A, is browsed for it, and because content is more in the results list, one page shows endless, it is necessary to which multipage is shown, homepage the results list is such as List-1 shown in Fig. 2 (a).

Then, the first acquisition device 21 obtain that the user performs under the second interactive mode to the response contents the Two interactive instructions, to be instructed as the user mutual, wherein, second interactive mode is different from the described first interaction mould Formula.

For example, connecting example, after user A browses the list-1 as shown in Fig. 2 (a), no film interested is found, in It is that user A has carried out page turn over operation by TV remote controller, the button for turning to lower one page can be realized on remote control (such as clicked on Shown in Fig. 2 (b)), then the message that the first acquisition device 21 can be sent by receiving user A by the button on remote control, obtain The page turn over operation performed to user A, and instructed as the user mutual.

Here, the present invention is by determining second interaction of the user under the second interactive mode different from the first interactive mode The unified interactive instruction of instruction so that in the case of original only can perform the instruction under the first interactive mode, also can perform and be different from Instruction under first interactive mode, interactive experience is further improved, also further such that being improved in (more wheels) session operational scenarios Intercommunity between distinct interaction pattern.

In one embodiment, if user mutual instruction meets pre-defined rule, wherein, the first determining device 22 includes First determining unit (not shown) and the second determining unit (not shown).Specifically, there is provided device 24 exists in response to the user The first interactive instruction under first interactive mode, there is provided corresponding response contents；First acquisition device 21 obtains the user and existed The second interactive instruction that the response contents are performed under second interactive mode, to be instructed as the user mutual, wherein, institute State the second interactive mode and be different from first interactive mode；First determining unit instructs according to the user mutual, with reference to institute The page status belonging to response contents is stated, determines the corresponding target instruction target word of the user mutual instruction；Second determining unit is true Unified interactive instruction corresponding to the fixed target instruction target word, wherein, the multiple distinct interaction patterns synonymous with the target instruction target word Under interactive instruction it is corresponding with the unified interactive instruction；Performs device 23 is according to the unified interactive instruction, described in execution User mutual instructs.Here, the first acquisition device 21, performs device 23 and offer device 24 and corresponding intrument in previous embodiment Content it is same or similar, for simplicity, therefore will not be repeated here, and be incorporated herein by reference.

Here, the pre-defined rule includes following at least any one：

- user mutual instruction the correspondence at least two candidate instructions intention item；

The instruction of-the user mutual is natural language instructions, and meets predetermined condition.

If instructing corresponding at least two candidate instructions to be intended to item for example, the pre-defined rule includes the user mutual, This, corresponding at least two candidate instructions of user mutual instruction are intended to item and refer to that the user mutual instruction has at least in itself Two instruction implications, at least two instructions are corresponded in other words and are intended to, such as remote control, the corresponding two kinds of functions of a button be present Situation, such as the volume amplifying key-press on remote control and to turn to the button of lower one page be same, if user clicks on the button, Corresponding two candidate instructions of user mutual instruction accordingly inputted are intended to item.

For another example, if the pre-defined rule includes the user mutual, instruction is natural language instructions, and meets predetermined condition, Here, the predetermined condition includes but is not limited to such as：1) predetermined classification is contained in the text corresponding to the natural language instructions Word, here, the predetermined classifier refers to cause the incomplete word of text semantic corresponding to the natural language instructions, such as lack Lose word, refer to word etc.；2) text corresponding to the natural language instructions possesses semantic missing, namely semantic imperfect.

Specifically, the first determining unit instructs according to the user mutual, the page shape with reference to belonging to the response contents State, determine the corresponding target instruction target word of the user mutual instruction.Here, the page status refers to residing for the response contents Page form, its include but is not limited to such as list page, details page, homepage, broadcast state.

For example, it is assumed that user A have selected certain for performing equipment 1 (assuming that it is intelligent television to perform equipment 1) by remote control Individual channel plays program, and then, user A is said " amplification ", but because the text corresponding to the natural language instructions possesses semantic lack Lose, namely it is semantic imperfect, i.e., according only to the natural language instructions and it is unaware of amplifying what, but because of response contents (i.e. just In the program of broadcasting) belonging to page status be broadcast state, then the first determining unit can order according to the user A natural language Order, with reference to the page status belonging to the response contents, can know that the natural language instructions are intended to amplify volume, that is, Determine that the target instruction target word corresponding to the user A natural language instructions is amplified for volume.

Preferably, the first determining unit can also instruct according to the user mutual, with reference to the page belonging to the response contents Surface state and first interactive instruction, determine the corresponding target instruction target word of the user mutual instruction.

For example, it is assumed that provide natural language instructions " recently have what good-looking film " of the device 24 based on user A, there is provided The list-1 of a part as response contents as shown in Fig. 2 (a), and user A browses the list-1 as shown in Fig. 2 (a) Afterwards, no film interested is found, then user A has carried out page turn over operation by TV remote controller, such as clicks on remote control On can realize the button (such as Fig. 2 (b) shown in) for turning to lower one page, but because function corresponding to the button can also realize that volume is put Big and amplify Current Content, because the page status belonging to response contents is list page, then the first determining unit can root first The message (message is the user mutual instruction of user A inputs) sent according to user A by the button on remote control, with reference to institute The page status belonging to response contents is stated, it is next to determine that possibility instruction that user A is performed corresponding to button operation is intended to turn to The page turn over operation of page amplifies Current Content, but is not volume amplifieroperation；Then, because the first interactive instruction is " recently Have the film what is good-looking ", namely user A is intended to inquiry which good-looking film has recently, therefore, the first determining unit can be true It is to turn to the page turn over operation of lower one page to determine the target instruction target word that user A is performed corresponding to button operation, rather than Current Content is put Greatly.

Those skilled in the art will be understood that the mode of the corresponding target instruction target word of the above-mentioned determination user mutual instruction It is only for example, the mode of the corresponding target instruction target word of user mutual instruction described in other existing or determinations for being likely to occur from now on The present invention is such as applicable to, should be also included within the scope of the present invention, and be incorporated herein by reference herein.

Then, the second determining unit determines the unified interactive instruction corresponding to the target instruction target word, wherein, with the target Instruct the interactive instruction under synonymous multiple distinct interaction patterns corresponding with the unified interactive instruction.Here, second determines Unit determines that the mode of the unified interactive instruction corresponding to the target instruction target word and the second determining device 12 in Fig. 1 embodiments determine The mode of the corresponding unified interactive instruction of user mutual instruction is same or similar, for simplicity, therefore will not be repeated here, and It is incorporated herein by reference.

Here, it should be noted that first interactive instruction and second interactive instruction can be users same What user performed on intelligent terminal or same performed in its corresponding different intelligent terminal.

Fig. 4 shows a kind of method flow diagram for being used to perform interactive instruction according to a further aspect of the present invention.

Wherein, the method comprising the steps of S1, step S2 and step S3.

Specifically, in step sl, the user mutual instruction that equipment 1 obtains user's input is performed；In step s 2, perform Equipment 1 determines the corresponding unified interactive instruction of the user mutual instruction, wherein, synonymous with user mutual instruction is more Interactive instruction under individual distinct interaction pattern is corresponding with the unified interactive instruction；In step s3, equipment 1 is performed according to institute Unified interactive instruction is stated, performs the user mutual instruction.

Specifically, in step sl, the application programming interfaces (API) that equipment 1 provides by performing equipment 1 itself are performed, Obtain the user mutual instruction of user's input, or the user mutual sent by receiving the third party devices such as intelligent terminal Instruction, to obtain the user mutual instruction.Here, user mutual instruction includes but is not limited to such as in interactive voice, remote control The instruction inputted under any one interactive mode such as interaction, gesture interaction, image interaction, vocal print interaction, body feeling interaction.For example, If user mutual instruction be natural language instructions, in step sl, execution equipment 1 can be by performing such as Mike of equipment 1 The sound pick up equipments such as wind obtain the natural language instructions of user.For another example, if user mutual instruction is the image of input, as face is known Not, then in step sl, perform equipment 1 can by perform equipment 1 video camera or camera gather the image containing face or Video flowing, and automatic detect and track face in the picture, and then carry out recognition of face.Also such as, if user mutual instruction is logical The instruction of remote control input is crossed, then in step sl, performing equipment 1 can be sent by receiving user by the button on remote control Message, to obtain user mutual instruction.And for example, if user mutual instruction is the finger inputted by performing gesture operation Order, then in step sl, the application programming interfaces (API) that equipment 1 can provide by performing equipment 1 itself are performed, get use The gesture path that family performs on the touchscreen, so as to get corresponding gesture operation.

Then, in step s 2, perform equipment 1 and determine the corresponding unified interactive instruction of the user mutual instruction, its In, instruct the interactive instruction under synonymous multiple distinct interaction patterns relative with the unified interactive instruction with the user mutual Should.Here, the interactive mode includes but is not limited to such as interactive voice, remote control interaction, gesture interaction, image interaction, vocal print Interaction, body feeling interaction etc..Here, it is described it is synonymous refer to instruction effect it is identical.Here, the unified interactive instruction refers to multiple Interactive instruction under synonymous distinct interaction pattern is carried out uniformly, and obtained unified interactive instruction, for example, referring to for page turning Order, as on voice " lower one page " " anything else ", the turnover key of remote control, user's point touching screen the page shaped likeIt is next PageThe linking buttons such as/[lower one page], because the instruction effect of such instruction is identical, unification can be carried out to such instruction, such as set A kind of new instruction format to carry out unified representation to the instruction effect identical instruction under various interactive modes, so that nothing It is executable by the instruction under which kind of interactive mode, do not limited by interactive mode.

Here, in step s 2, perform the mode bag that equipment 1 determines the corresponding unified interactive instruction of user mutual instruction Include but be not limited to following at least any one：

For example, it is assumed that it is electronic product to perform equipment 1, certain APP is installed thereon, as user A is intended to log in the APP, its dozen After opening the APP, the login of brush face is clicked, then opens recognition of face interactive mode, under the interactive mode, triggering unlatching is held Video camera or camera in row equipment 1, so as to carry out face collection, and automatic detect and track people in the picture to user A Face, and then recognition of face is carried out, user A facial image is recognized, this is the user mutual instruction of user A inputs, it is assumed that should (such as synonymous interactive instruction cluster also act as other interaction moulds " logged in " to synonymous interactive instruction cluster belonging to instruction comprising instruction Formula gives an order as logged in two-dimensional code scanning, voice " login " etc.) corresponding to unified interactive instruction be { instruct:Log in, Then in step s 2, perform equipment 1 and can determine that corresponding unified interactive instruction is { instruct:Log in }.

For another example, it is assumed that user A is intended to be paid, and in the case where paying scene, acquires user A sound, and is verified, Such as by the comparison discriminating of feature in terms of the user A collected sound and the progress sense of hearing of known sound and frequency spectrum two and total score Analysis, it is determined that whether the two comes from same sound source (sound of same person), to verify sounder identity, it is assumed that be verified, phase Ying Di, the user A collected sound are the user mutual instruction of user A inputs, and the synonymous interaction belonging to the instruction refers to Making cluster, (other interactive modes that such as the synonymous interactive instruction cluster also act as " paying " comprising instruction give an order such as Quick Response Code of paying the bill Scanning, voice " payment " etc.) corresponding to unified interactive instruction be { pay:Pay, then in step s 2, performing equipment 1 can be true Fixed corresponding unified interactive instruction is { pay:Pay }.

Also such as, it is assumed that user A is intended to carry out voice lookup most by performing equipment 1 (assuming that it is intelligent television to perform equipment 1) Nearly good-looking film, it is assumed that client such as Voice-APP, user A are equipped with the intelligent television and opens the client, Huo Zhetong Cross voice to wake up it, or directly send what good-looking electricity is natural language instructions " have recently to the intelligent television Shadow ", list-1 of the results list returned as shown in Fig. 2 (a), now user A turned over further through TV remote controller Page operations, the button for turning to lower one page (shown in such as Fig. 2 (b)) can be realized on remote control as clicked on, and it is same belonging to the instruction (such as the synonymous interactive instruction cluster also act as the finger under " turning to lower one page " other interactive modes to adopted interactive instruction cluster comprising instruction Order, such as on voice " lower one page ", user's point touching screen the page shaped like lower one page/[lower one page] linking button) it is corresponding Unified interactive instruction be { page down:Lower one page }, then in step s 2, perform equipment 1 and can determine that corresponding unified interaction Instruct as { page down:Lower one page }.

Then, in step s3, equipment 1 is performed according to the unified interactive instruction, performs the user mutual instruction, such as The unified interactive instruction is performed, so as to realize the execution to user mutual instruction, i.e., by performing the unified interaction Instruction, reach with performing the user mutual instruction identical implementation effect.

For example, connecing precedent, in step s3, equipment 1 is performed as performed unified interactive instruction { instruct:Log in, then Can Successful login.

For another example, in step s3, equipment 1 is performed as performed unified interactive instruction { pay:Pay, then can successfully it pay.

Also such as, in step s3, equipment 1 is performed as performed unified interactive instruction { page down:Lower one page }, then it can turn over To lower one page, lower one page film the results list as shown in Fig. 2 (b) is obtained.

It is seamlessly connected here, the present invention realizes various distinct interaction patterns so that no matter user is using which kind of interaction mould It formula, can accurately perform, reduce dependence of the equipment to interactive mode, correspondingly, also achieve and support a variety of distinct interactions Pattern is used in mixed way so that user can use arbitrary interactive mode in dialog procedure, not to the interbehavior structure of user Into limitation；Meanwhile the instruction unified representation of synonymous all kinds of interactive modes, user does not have to concern that distinct interaction pattern can cause Dialogue is isolated and can not be connected, and is made entirely to interact more remarkable fluency, is improved interactive efficiency, also improve interactive experience.For example, with After family A sees lower one page film the results list as shown in Fig. 2 (b), voice says " I will see that goddess is very chivalrous " (such as Fig. 2 (c) institutes Show), correspondingly, the film can be commenced play out by performing equipment 1.

In one embodiment, this method (is not shown also including step S5 (not shown), step S6 (not shown), step S7 Go out) and step S8 (not shown).Specifically, in step s 5, perform equipment 1 and obtain one or more synonymous interactive instruction clusters, Wherein, the synonymous interactive instruction cluster includes representing the interactive instruction under synonymous multiple distinct interaction patterns；In step s 6, Execution equipment 1 determines the unified interactive instruction corresponding to the synonymous interactive instruction cluster；In the step s 7, equipment 1 is performed to establish Or renewal multi-module interactive instruction database, wherein, the multi-module interactive instruction database is including corresponding to the synonymous interactive instruction cluster and its Unified interactive instruction between mapping relations；In step sl, the user mutual instruction that equipment 1 obtains user's input is performed； In step s 8, the instruction description information that equipment 1 instructs according to the user mutual is performed, in the multi-module interactive instruction database Matching inquiry is carried out, to determine the synonymous interactive instruction cluster belonging to the user mutual instruction；In step s 2, equipment 1 is performed According to the synonymous interactive instruction cluster belonging to user mutual instruction, the corresponding unified interaction of the user mutual instruction is determined Instruction, wherein, instruct the interactive instruction under synonymous multiple distinct interaction patterns to be interacted with the unification with the user mutual Instruct corresponding；In step s3, equipment 1 is performed according to the unified interactive instruction, performs the user mutual instruction.

Here, the content that step S1 and step S3 correspond to step with previous embodiment is same or similar, for simplicity, Therefore will not be repeated here, and be incorporated herein by reference.

Here, in step s 2, perform synonymous interactive instruction cluster of the equipment 1 according to belonging to instructing the user mutual and determine The mode of the corresponding unified interactive instruction of the user mutual instruction, with performing equipment 1 in step s 2 in previous embodiment Synonymous interactive instruction cluster according to belonging to user mutual instruction determines that the corresponding unification of the user mutual instruction interacts The mode of instruction is same or similar, for simplicity, therefore will not be repeated here, and be incorporated herein by reference.

Specifically, in step s 5, perform equipment 1 and obtain one or more synonymous interactive instruction clusters, wherein, it is described synonymous Interactive instruction cluster includes representing the interactive instruction under synonymous multiple distinct interaction patterns.Here, in step s 5, perform equipment 1 mode for obtaining one or more of synonymous interactive instruction clusters includes but is not limited to following at least any one：

For example, in step s 5, equipment 1 is performed first from the corresponding interactive instruction storehouse of predetermined each interactive mode difference It is middle to obtain whole interactive instructions, then acted on according to the instruction of the interactive instruction of mark, using sorting techniques such as decision trees, Such interactive instruction is classified, by the friendship being divided under multiple distinct interaction patterns of same class (i.e. instruction effect is identical) Mutually instruction forms a synonymous interactive instruction cluster, so as to obtain multiple synonymous interactive instruction clusters.

……

Then in step s 5, execution equipment 1 can mark multiple friendships under the distinct interaction pattern of synonymy according to this Mutually instruction training sample, clustering processing is carried out to the plurality of interactive instruction training sample, will such as mark the friendship for belonging to synonymy Mutually instruction training sample is classified as one kind, so as to obtain one or more of synonymous interactive instruction clusters, will such as contain above-mentioned marked The interactive instruction training sample interactive instruction 1. for noting synonymy is classified as one kind and obtains cluster1, by it is above-mentioned marked it is synonymous The interactive instruction of the interactive instruction training sample of relation 2. is classified as one kind and obtains cluster2, one or more of so as to obtain Synonymous interactive instruction cluster.

In step s 6, execution equipment 1 determines the unified interactive instruction corresponding to the synonymous interactive instruction cluster, such as with system The instruction effect that one interactive instruction is expressed to express the synonymous interactive instruction cluster to do, i.e., in the synonymous interactive instruction cluster Interactive instruction under multiple synonymous distinct interaction patterns is carried out uniformly, and obtained unified interactive instruction.

In the step s 7, perform equipment 1 and establish or update multi-module interactive instruction database, such as by establishing synonymous interactive instruction cluster With the unified interactive instruction pair corresponding to it, to obtain multi-module interactive instruction database as shown in the following Table 2, wherein, the multimode Mapping relations between unified interactive instruction of the interactive instruction storehouse including the synonymous interactive instruction cluster and corresponding to it.Here, The mode of multi-module interactive instruction database renewal can such as be updated by pre-defined rule by cycle mode.

Table 2

In step sl, the user mutual instruction that equipment 1 obtains user's input is performed.

In step s 8, the instruction description information that equipment 1 instructs according to the user mutual is performed, in the multi-module interactive Matching inquiry is carried out in instruction database, to determine the synonymous interactive instruction cluster belonging to the user mutual instruction.Here, the instruction Description information is the information for referring to that user mutual instruction is described, and it can be that the user mutual instruction institute is right (if user mutual instruction is natural language instructions, then its instruction description information can be for oneself to this to the text description information answered Right verbal order carries out text obtained from semantic parsing；If user mutual instruction is remote control interaction, gesture interaction, image are handed over Mutually, the interactive instruction under vocal print interaction isotype, then corresponding instruction description information can be explaining in plain text to instruction This) or to main contents obtained from the progress main contents extraction of text description information.

For example, for user A, it directly sends what good-looking electricity is natural language instructions " have recently to intelligent television Shadow ", list-1 of the results list returned as shown in Fig. 2 (a), now user A turned over further through TV remote controller Page operations, the button for turning to lower one page (shown in such as Fig. 2 (b)) can be realized on remote control as clicked on, then in step sl, is performed The application programming interfaces (API) that equipment 1 is provided by intelligent television, get the user mutual instruction of user A inputs；So Afterwards, in step s 8, the instruction description information that equipment 1 instructs according to the user mutual is performed, as instructed to the user mutual Text " turning to lower one page " is illustrated, matching inquiry is carried out in multi-module interactive instruction database as shown in table 2, obtains the user Synonymous interactive instruction cluster belonging to interactive instruction is cluster1.

Fig. 5 shows a kind of method flow diagram for being used to perform interactive instruction of one embodiment of the invention.

Wherein, the method comprising the steps of S1 ', step S2 ', step S3 ' and step S4 '.

Specifically, in step S4', first interaction of the equipment 1 in response to the user under the first interactive mode is performed Instruction, there is provided corresponding response contents；In step S1', perform equipment 1 and obtain the user under the second interactive mode to institute The second interactive instruction of response contents execution is stated, to be instructed as the user mutual, wherein, second interactive mode is different In first interactive mode；In step S2', perform equipment 1 and determine the corresponding unified interaction of the user mutual instruction Instruction, wherein, instruct the interactive instruction under synonymous multiple distinct interaction patterns to be interacted with the unification with the user mutual Instruct corresponding；In step S3', equipment 1 is performed according to the unified interactive instruction, performs the user mutual instruction. This, step S2' and step S3' respectively with that the content of step is corresponded in Fig. 4 embodiments is same or similar, here, for simplicity, Therefore will not be repeated here, and be incorporated herein by reference.

Specifically, in step S4', first interaction of the equipment 1 in response to the user under the first interactive mode is performed Instruction, there is provided corresponding response contents.

For example, for user A, it directly have issued what good-looking electricity is natural language instructions " have recently to intelligent television Shadow ", then in step S4', first interactive instruction that equipment 1 performs in response to user A is performed, obtains corresponding response first Content, such as inquired about in movie database, obtain corresponding result, and with tabular form and returned, so as to by the result List is supplied to user A, is browsed for it, and because content is more in the results list, one page shows endless, it is necessary to multipage is shown, first List-1 of the page the results list as shown in Fig. 2 (a).

Then, in step S1', perform equipment 1 and obtain the user under the second interactive mode to the response contents The second interactive instruction performed, to be instructed as the user mutual, wherein, second interactive mode is different from described first Interactive mode.

For example, connecting example, after user A browses the list-1 as shown in Fig. 2 (a), no film interested is found, in It is that user A has carried out page turn over operation by TV remote controller, the button for turning to lower one page can be realized on remote control (such as clicked on Shown in Fig. 2 (b)), then in step S1', perform the news that equipment 1 can be sent by receiving user A by the button on remote control Breath, the page turn over operation of user A execution is got, and instructed as the user mutual.

Here, the present invention is by determining second interaction of the user under the second interactive mode different from the first interactive mode The unified interactive instruction of instruction so that in the case of original only can perform the instruction under the first interactive mode, also can perform and be different from Instruction under first interactive mode, further improves interactive experience.

In one embodiment, if user mutual instruction meets pre-defined rule, wherein, step S2' includes step S21'(is not shown) and step S22'(be not shown).Specifically, in step S4', equipment 1 is performed in response to the user the The first interactive instruction under one interactive mode, there is provided corresponding response contents；In step S1', perform equipment 1 and obtain the use The second interactive instruction that family performs under the second interactive mode to the response contents, to be instructed as the user mutual, its In, second interactive mode is different from first interactive mode；In step S21', equipment 1 is performed according to the user Interactive instruction, with reference to the page status belonging to the response contents, determine the corresponding target instruction target word of the user mutual instruction； In step S22', execution equipment 1 determines the unified interactive instruction corresponding to the target instruction target word, wherein, refer to the target Make the interactive instruction under synonymous multiple distinct interaction patterns corresponding with the unified interactive instruction；In step S3', perform Equipment 1 performs the user mutual instruction according to the unified interactive instruction.Here, step S1', step S3' and step S4' It is same or similar that the content of step is corresponded to previous embodiment, for simplicity, therefore will not be repeated here, and with the side of reference Formula is incorporated herein.

Here, the pre-defined rule includes following at least any one：

Specifically, in step S21', perform equipment 1 and instructed according to the user mutual, with reference to the response contents institute The page status of category, determine the corresponding target instruction target word of the user mutual instruction.Here, the page status refers to the sound The page form residing for content is answered, it includes but is not limited to such as list page, details page, homepage, broadcast state.

For example, it is assumed that user A have selected certain for performing equipment 1 (assuming that it is intelligent television to perform equipment 1) by remote control Individual channel plays program, and then, user A is said " amplification ", but because the text corresponding to the natural language instructions possesses semantic lack Lose, namely it is semantic imperfect, i.e., according only to the natural language instructions and it is unaware of amplifying what, but because of response contents (i.e. just In the program of broadcasting) belonging to page status be broadcast state, then in step S21', perform equipment 1 can according to user A should Natural language instructions, with reference to the page status belonging to the response contents, it can know that the natural language instructions are intended to volume Amplification, that is, determining that the target instruction target word corresponding to the user A natural language instructions is amplified for volume.

Preferably, in step S21', performing equipment 1 can also instruct according to the user mutual, with reference in the response Page status and first interactive instruction belonging to holding, determine the corresponding target instruction target word of the user mutual instruction.

For example, it is assumed that in step S4', perform what natural language instructions of the equipment 1 based on user A " have good-looking recently Film ", there is provided the list-1 of the part as response contents as shown in Fig. 2 (a), and user A is browsed such as Fig. 2 (a) After shown list-1, no film interested is found, then user A has carried out page turn over operation by TV remote controller, The button for turning to lower one page (shown in such as Fig. 2 (b)) can be realized by such as clicking on remote control, but due to function corresponding to the button also Volume amplification can be achieved and amplify Current Content, because the page status belonging to response contents is list page, then in step In S21', (message is user A defeated to the message that execution equipment 1 can be sent according to user A by the button on remote control first The user mutual instruction entered), with reference to the page status belonging to the response contents, determine that user A is performed corresponding to button operation Possibility instruction be intended to turn to the page turn over operation of lower one page or amplify Current Content, but be not volume amplifieroperation；Then, Because the first interactive instruction is " having what good-looking film recently ", namely user A is intended to which inquiry film good-looking recently has A bit, therefore, in step S21', perform equipment 1 and can determine that user A performs the target instruction target word corresponding to button operation to turn to down The page turn over operation of one page, rather than Current Content is amplified.

Then, in step S22', execution equipment 1 determines the unified interactive instruction corresponding to the target instruction target word, wherein, Interactive instruction under multiple distinct interaction patterns synonymous with the target instruction target word is corresponding with the unified interactive instruction. This, in step S22', performs equipment 1 and determines that the mode of the unified interactive instruction corresponding to the target instruction target word is implemented with Fig. 4 Equipment 1 is performed in example and determines that the mode of the corresponding unified interactive instruction of user mutual instruction is same or similar in step s 2, For simplicity, it therefore will not be repeated here, and be incorporated herein by reference.

Fig. 6 shows the block diagram suitable for being used for the exemplary computer system/server for realizing embodiment of the present invention.Figure The computer system/server 3 of 6 displays is only an example, should not be to the function and use range band of the embodiment of the present invention Carry out any restrictions.

As shown in fig. 6, computer system/server 3 is showed in the form of universal computing device.Computer system/service The component of device 3 can include but is not limited to：One or more processor or processing unit 31, system storage 32, connection The bus 33 of different system component (including system storage 32 and processing unit 31).

Bus 33 represents the one or more in a few class bus structures, including memory bus or Memory Controller, Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.Lift For example, these architectures include but is not limited to industry standard architecture (ISA) bus, MCA (MAC) Bus, enhanced isa bus, VESA's (VESA) local bus and periphery component interconnection (PCI) bus.

Computer system/server 3 typically comprises various computing systems computer-readable recording medium.These media can be any The usable medium that can be accessed by computer system/server 3, including volatibility and non-volatile media, movably and not Moveable medium.

System storage 32 can include the computer system readable media of form of volatile memory, such as arbitrary access Memory (RAM) 321 and/or cache memory 322.Computer system/server 3 may further include other removable Dynamic/immovable, volatile/non-volatile computer system storage medium.Only as an example, storage system 323 can be used In reading and writing immovable, non-volatile magnetic media (Fig. 6 do not show, commonly referred to as " hard disk drive ").Although do not show in Fig. 6 Going out, can providing for the disc driver to may move non-volatile magnetic disk (such as " floppy disk ") read-write, and to removable The CD drive of anonvolatile optical disk (such as CD-ROM, DVD-ROM or other optical mediums) read-write.In these cases, Each driver can be connected by one or more data media interfaces with bus 33.System storage 32 can be included extremely A few program product, the program product have one group of (for example, at least one) program module, and these program modules are configured to Perform the function of various embodiments of the present invention.

Program/utility 324 with one group of (at least one) program module 325, such as system can be stored in and deposited In reservoir 32, such program module 325 include --- but being not limited to --- operating system, one or more application program, Other program modules and routine data, the reality of network environment may be included in each or certain combination in these examples It is existing.Program module 325 generally performs function and/or method in embodiment described in the invention.

Computer system/server 3 can also be with one or more external equipments 35 (such as keyboard, sensing equipment, display Device 34 etc.) communication, the equipment communication interacted with the computer system/server 3 can be also enabled a user to one or more, And/or any equipment with enabling the computer system/server 3 to be communicated with one or more of the other computing device (such as network interface card, modem etc.) communicates.This communication can be carried out by input/output (I/O) interface 36.Also, Computer system/server 3 can also by network adapter 30 and one or more network (such as LAN (LAN), extensively Domain net (WAN) and/or public network, such as internet) communication.As shown in fig. 6, network adapter 30 passes through bus 33 and calculating Other modules communication of machine systems/servers 3.It should be understood that although not shown in Fig. 4, computer system/service can be combined Device 3 uses other hardware and/or software module, includes but is not limited to：Microcode, device driver, redundant processing unit, outside Disk drive array, RAID system, tape drive and data backup storage system etc..

Processing unit 31 is stored in program in system storage 32 by operation, so as to perform various function application and Data processing, such as a kind of following method for performing interactive instruction is realized, wherein, this method comprises the following steps：

A obtains the user mutual instruction of user's input；

B determines the corresponding unified interactive instruction of the user mutual instruction, wherein, instructed with the user mutual synonymous Multiple distinct interaction patterns under interactive instruction it is corresponding with the unified interactive instruction；

It should be noted that the present invention can be carried out in the assembly of software and/or software and hardware, for example, can adopt With application specific integrated circuit (ASIC), general purpose computer or any other realized similar to hardware device.In one embodiment In, software program of the invention can realize steps described above or son by computing device.Similarly, it is of the invention soft Part program (including related data structure) can be stored in computer readable recording medium storing program for performing, for example, RAM memory, magnetic Or CD-ROM driver or floppy disc and similar devices.In addition, some steps or son of the present invention can employ hardware to realize, for example, As with processor coordinate so as to perform each step or son circuit.

In addition, the part of the present invention can be applied to computer program product, such as computer program instructions, when its quilt When computer performs, by the operation of the computer, the method according to the invention and/or technical scheme can be called or provided. And the programmed instruction of the method for the present invention is called, it is possibly stored in fixed or moveable recording medium, and/or pass through Broadcast or the data flow in other signal bearing medias and be transmitted, and/or be stored according to described program instruction operation In the working storage of computer equipment.Here, including a device according to one embodiment of present invention, the device includes using Memory in storage computer program instructions and processor for execute program instructions, wherein, when the computer program refers to When order is by the computing device, method and/or skill of the plant running based on foregoing multiple embodiments according to the present invention are triggered Art scheme.

It is obvious to a person skilled in the art that the invention is not restricted to the details of above-mentioned one exemplary embodiment, Er Qie In the case of without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.Therefore, no matter From the point of view of which point, embodiment all should be regarded as exemplary, and be nonrestrictive, the scope of the present invention is by appended power Profit requires rather than described above limits, it is intended that all in the implication and scope of the equivalency of claim by falling Change is included in the present invention.Any reference in claim should not be considered as to the involved claim of limitation.This Outside, it is clear that the word of " comprising " one is not excluded for other units or step, and odd number is not excluded for plural number.That is stated in device claim is multiple Unit or device can also be realized by a unit or device by software or hardware.The first, the second grade word is used for table Show title, and be not offered as any specific order.

Claims

1. a kind of method for performing interactive instruction, wherein, this method comprises the following steps：

A obtains the user mutual instruction of user's input；

B determines the corresponding unified interactive instruction of the user mutual instruction, wherein, synonymous with user mutual instruction is more Interactive instruction under individual distinct interaction pattern is corresponding with the unified interactive instruction；

2. according to the method for claim 1, wherein, the step b includes：

According to the synonymous interactive instruction cluster belonging to user mutual instruction, the corresponding unification of the user mutual instruction is determined Interactive instruction, wherein, the interactive instruction under synonymous multiple distinct interaction patterns and the unification are instructed with the user mutual Interactive instruction is corresponding.

3. according to the method for claim 2, wherein, this method also includes step：

X obtains one or more synonymous interactive instruction clusters, wherein, the synonymous interactive instruction cluster include representing it is synonymous it is multiple not With the interactive instruction under interactive mode；

- determine unified interactive instruction corresponding to the synonymous interactive instruction cluster；

- establish or renewal multi-module interactive instruction database, wherein, the multi-module interactive instruction database include the synonymous interactive instruction cluster with The mapping relations between unified interactive instruction corresponding to it；

- instruction description the information instructed according to the user mutual, matching inquiry is carried out in the multi-module interactive instruction database, with Determine the synonymous interactive instruction cluster belonging to the user mutual instruction.

4. according to the method for claim 3, wherein, the step x includes：

Acted on according to the instruction of interactive instruction, the interactive instruction in the interactive instruction storehouse corresponding to each interactive mode difference is carried out Classification is handled, to obtain one or more of synonymous interactive instruction clusters, wherein, the synonymous interactive instruction cluster includes representing same Interactive instruction under multiple distinct interaction patterns of justice.

5. method according to any one of claim 1 to 4, wherein, this method also includes step：

In response to first interactive instruction of the user under the first interactive mode, there is provided corresponding response contents；

Wherein, the step a includes：

The second interactive instruction that the user performs under the second interactive mode to the response contents is obtained, to be used as the use Family interactive instruction, wherein, second interactive mode is different from first interactive mode.

6. the method according to claim 11, wherein, if user mutual instruction meets pre-defined rule, wherein, the step Rapid b includes：

B1 instructs according to the user mutual, with reference to the page status belonging to the response contents, determines that the user mutual refers to The corresponding target instruction target word of order；

The unified interactive instruction corresponding to the target instruction target word is determined, wherein, the multiple different friendships synonymous from the target instruction target word Interactive instruction under mutual pattern is corresponding with the unified interactive instruction.

7. according to the method for claim 6, wherein, the step b1 includes：

- instructed according to the user mutual, page status and first interactive instruction with reference to belonging to the response contents, Determine the corresponding target instruction target word of the user mutual instruction.

8. the method according to claim 5 or 6, wherein, the pre-defined rule includes following at least any one：

9. a kind of execution equipment for being used to perform interactive instruction, wherein, the execution equipment includes：

First determining device, the unified interactive instruction corresponding for determining the user mutual instruction, wherein, with the user Interactive instruction under the synonymous multiple distinct interaction patterns of interactive instruction is corresponding with the unified interactive instruction；

10. execution equipment according to claim 9, wherein, first determining device is used for：

11. execution equipment according to claim 10, wherein, the execution equipment also includes：

Second acquisition device, for obtaining one or more synonymous interactive instruction clusters, wherein, the synonymous interactive instruction cluster includes Represent the interactive instruction under synonymous multiple distinct interaction patterns；

Second determining device, for determining the unified interactive instruction corresponding to the synonymous interactive instruction cluster；

Foundation or updating device, for establishing or updating multi-module interactive instruction database, wherein, the multi-module interactive instruction database includes institute State the mapping relations between synonymous interactive instruction cluster and unified interactive instruction corresponding to it；

3rd determining device, for the instruction description information instructed according to the user mutual, in the multi-module interactive instruction database Middle carry out matching inquiry, to determine the synonymous interactive instruction cluster belonging to the user mutual instruction.

12. execution equipment according to claim 11, wherein, second acquisition device is used for：

13. the execution equipment according to any one of claim 9 to 12, wherein, the execution equipment also includes：

Device is provided, for the first interactive instruction in response to the user under the first interactive mode, there is provided corresponding response Content；

Wherein, first acquisition device is used for：

14. execution equipment according to claim 13, wherein, if user mutual instruction meets pre-defined rule, wherein, First determining device includes：

First determining unit, for being instructed according to the user mutual, with reference to the page status belonging to the response contents, it is determined that The corresponding target instruction target word of the user mutual instruction；

Second determining unit, for determining the unified interactive instruction corresponding to the target instruction target word, wherein, with the target instruction target word Interactive instruction under synonymous multiple distinct interaction patterns is corresponding with the unified interactive instruction.

15. execution equipment according to claim 14, wherein, first determining unit is used for：

16. the execution equipment according to claims 14 or 15, wherein, the pre-defined rule includes following at least any one：

17. a kind of computing device, including：

One or more processors；

Memory, for storing one or more programs,

When one or more of programs are by one or more of computing devices so that one or more of processors Perform the method as any one of claim 1 to 8.

18. a kind of computer-readable recording medium, is stored thereon with computer program, wherein, when the program is executed by processor Realize the method as any one of claim 1 to 8.