WO2023103917A1 - 语音控制方法、装置、电子设备及存储介质 - Google Patents

语音控制方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023103917A1
WO2023103917A1 PCT/CN2022/136341 CN2022136341W WO2023103917A1 WO 2023103917 A1 WO2023103917 A1 WO 2023103917A1 CN 2022136341 W CN2022136341 W CN 2022136341W WO 2023103917 A1 WO2023103917 A1 WO 2023103917A1
Authority
WO
WIPO (PCT)
Prior art keywords
control
identifier
node
target
controls
Prior art date
Application number
PCT/CN2022/136341
Other languages
English (en)
French (fr)
Inventor
戴强
张晓帆
陈明
曾理
Original Assignee
杭州逗酷软件科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州逗酷软件科技有限公司 filed Critical 杭州逗酷软件科技有限公司
Publication of WO2023103917A1 publication Critical patent/WO2023103917A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/16—Speech classification or search using artificial neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems

Definitions

  • the present application relates to the field of computer technology, and more specifically, to a voice control method, device, electronic equipment and storage medium.
  • voice assistants Combining artificial intelligence technology and virtual personal assistants (voice assistants), electronic devices can receive voice instructions from users through auditory modes and complete corresponding interactive tasks.
  • the user will only clarify his or her interaction intention after seeing the interactive interface, and hopes to directly operate the viewed interactive interface or the objects therein.
  • the present application proposes a voice control method, device, electronic equipment and storage medium, so as to improve the above problems.
  • the present application provides a voice control method, the method comprising: obtaining a first control identifier and a second control identifier from the acquired voice control instruction; if the target interface includes the first control identifier The corresponding control and the control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the control corresponding to the second control identifier, select from the plurality of controls corresponding to the first control identifier A target control is determined among the controls, the target interface is the interface displayed when the voice control instruction is acquired, wherein the control corresponding to the first control identifier is a control to be determined corresponding to the voice control instruction, and the The second control identifies a corresponding control and is used to determine a control representing the user's actual control target from among the controls to be determined as the target control; and execute a control operation corresponding to the target control.
  • the present application provides a voice control device, the device comprising: an identification acquisition unit, configured to acquire a first control identification and a second control identification from acquired voice control instructions; a control determination unit, configured to The target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the control corresponding to the second control identifier Determine a target control from a plurality of controls corresponding to the first control identifier, the target interface is the interface displayed when the voice control instruction is acquired, wherein the control corresponding to the first control identifier is the one corresponding to the first control identifier
  • the to-be-determined control corresponding to the voice control instruction, the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the to-be-determined controls as the target control; The control operation corresponding to the target control.
  • the present application provides an electronic device, including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, The one or more programs are configured to perform the methods described above.
  • the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, wherein the above method is executed when the program code is running.
  • FIG. 1 shows a schematic diagram of an application scenario of a voice control method proposed in an embodiment of the present application
  • FIG. 2 shows a schematic diagram of an application scenario of another voice control method proposed in the embodiment of the present application
  • FIG. 3 shows a flow chart of a voice control method proposed in an embodiment of the present application
  • FIG. 4 shows a schematic diagram of a user triggering a voice control command in an embodiment of the present application
  • FIG. 5 shows a flow chart of a voice control method proposed in another embodiment of the present application.
  • Fig. 6 shows a schematic diagram of a target interface in the embodiment of the present application
  • FIG. 7 shows a schematic diagram of a control relationship structure diagram in an embodiment of the present application.
  • Fig. 8 shows a schematic diagram of the relative position of a control in the embodiment of the present application.
  • FIG. 9 shows a flowchart of an implementation manner of S220 in the embodiment of the present application.
  • Fig. 10 shows a schematic diagram of another target interface in the embodiment of the present application.
  • FIG. 11 shows a schematic diagram of another control relationship structure diagram in the embodiment of the present application.
  • Fig. 12 shows a schematic diagram of another control relationship structure diagram in the embodiment of the present application.
  • FIG. 13 shows a flow chart of a voice control method proposed in another embodiment of the present application.
  • Figure 14 shows a schematic diagram of display distance in the embodiment of the present application.
  • Fig. 15 shows a structural block diagram of an object recognition device proposed by the embodiment of the present application.
  • Fig. 16 shows a structural block diagram of an electronic device proposed by the present application
  • Fig. 17 is a storage unit for storing or carrying program codes for realizing the voice control method according to the embodiment of the present application according to the embodiment of the present application.
  • An embodiment of the present application provides a voice control method, the method includes: obtaining the first control identifier and the second control identifier from the acquired voice control instruction; if the target interface includes a control corresponding to the first control identifier and the second control identifier The control corresponding to the second control identification, and there are multiple controls corresponding to the first control identification, based on the control corresponding to the second control identification, determine the target control from the multiple controls corresponding to the first control identification, and the target interface is the acquired voice The interface displayed when the command is controlled, wherein the control corresponding to the first control identifier is the undetermined control corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the undetermined controls as the target control;
  • determining the target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier includes: if there is only one control corresponding to the second control identifier, then based on the second control identifier The control determines the target control from multiple controls corresponding to the first control identifier.
  • the target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, including: if the second control identifier corresponds to There is one control, and the control corresponding to the second control identifier does not correspond to a similar control, and the target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier.
  • the target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, including: if the second control identifier corresponds to There is one control, and the control corresponding to the second control identifier corresponds to a similar control, and the control similar to the control corresponding to the second control identifier is obtained as the first similar control; based on the control corresponding to the second control identifier and the first similar control A control is used to determine a target control from multiple controls corresponding to the first control identifier.
  • obtaining a control similar to the control corresponding to the second control identifier as the first similar control includes: searching for the control corresponding to the second control identifier in the control relationship structure diagram based on the attributes of the control corresponding to the second control identifier.
  • a control similar to the corresponding control is used as the first similar control, and the properties include at least one of the distance from the node corresponding to the control to the root node, the type of the control, the length and width of the control, and the relative position of the control in the corresponding parent control; , the control relationship structure diagram is generated based on the containment relationship of the controls in the target interface, and the controls corresponding to the child nodes in the control relationship structure diagram are included in the controls corresponding to the parent nodes corresponding to the child nodes.
  • determining the target control from multiple controls corresponding to the first control identifier includes: The distance between the second nodes obtains multiple first distances, the first node is used to represent the control corresponding to the second control identifier, and the second node is used to represent the control corresponding to the first control identifier; obtain the control relationship structure diagram The distances between the third node and multiple second nodes are obtained to obtain multiple second distances, and the third node is the node corresponding to the first similar control; multiple reference distances are obtained, and the multiple reference distances include multiple first distances and multiple second distances; if the minimum value among the multiple reference distances is consistent with the minimum value among the multiple first distances, and the minimum value is one, identify the first control among the corresponding multiple controls, The control corresponding to the minimum value among the plurality of first distances is used as the target control.
  • the method further includes: if the minimum value among the multiple reference distances is inconsistent with the minimum value among the multiple first distances, and the multiple first distances do not have the same first distance as the minimum value among the multiple reference distances , acquiring a second similar control, where the second similar control is a control selected from the control relationship structure diagram based on the properties of the control corresponding to the first control identifier;
  • the third distance includes the distance from the node corresponding to the second similar control to the node corresponding to the second control identifier; if there is a distance in the third distance that is uniquely consistent with the minimum value among multiple reference distances, it will be uniquely consistent
  • the control corresponding to the distance is used as the target control.
  • determining the target control from multiple controls corresponding to the first control ID based on the control corresponding to the second control ID includes: obtaining, in the target interface, the multiple controls corresponding to the first control ID and the second control ID respectively.
  • the display distance between the controls corresponding to the control identifiers; the control corresponding to the smallest display distance among the controls corresponding to the first control identifier is used as the target control.
  • the method further includes: if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there is one control corresponding to the first control identifier, matching the first control identifier to control as the target control.
  • the method further includes: if there are two or more controls corresponding to the second control identifier, and there are two or more controls corresponding to the first control identifier, determining the target control by asking the user.
  • the method before acquiring the first control identifier and the second control identifier from the acquired voice control instruction, the method further includes: if the specified voice content is acquired, start acquiring the voice control instruction.
  • the specified voice content is configured by the user.
  • obtaining the first control identifier and the second control identifier from the acquired voice control instruction includes: converting the acquired voice control instruction into corresponding text content; performing identification acquisition from the text content based on semantic extraction rules, to Get the ID of the first control and the ID of the second control.
  • the method further includes: after receiving the voice control instruction, synchronously starting to identify the target interface to acquire the controls included in the target interface.
  • the manner of identifying the target interface includes: identifying the target interface through code analysis; identifying the target interface through graphic and text recognition; or identifying the target interface through icon recognition.
  • An embodiment of the present application provides a voice control device, which includes: an identification acquisition unit, configured to acquire a first control identification and a second control identification from the acquired voice control instructions; a control determination unit, configured to if the target interface includes There are controls corresponding to the first control identifier and controls corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the controls corresponding to the second control identifier
  • the target control is determined in the control, and the target interface is the interface displayed when the voice control instruction is obtained, wherein the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to obtain the voice control instruction.
  • a control that represents the user's actual control target is determined as the target control; the control unit is configured to perform a control operation corresponding to the target control.
  • An embodiment of the present application provides an electronic device, which is characterized by including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, one or more Multiple programs are configured to execute the methods provided in the embodiments of the present application.
  • An embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the method provided in the embodiment of the present application is executed when the program code is running.
  • the electronic device in the interface displayed by the electronic device, there may be multiple controls with the same name.
  • the electronic device also recognizes that the voice control instruction sent by the user includes the multiple controls with the same name. Therefore, the electronic device may not be able to accurately determine which control the user actually intends to operate, thus preventing the electronic device from accurately determining the user's actual control intention.
  • the inventor proposes a voice control method, device, electronic device and storage medium in the present application.
  • the method first obtains the first control identifier and the second control identifier from the acquired voice control instruction, and then the first control identifier
  • the corresponding control is the control to be determined corresponding to the voice control instruction
  • the control corresponding to the second control identifier is used to determine the control representing the user's actual control target as the target control from the controls to be determined, if the target interface includes
  • the target control is determined in and the control operation corresponding to the target control is executed.
  • the second control identifier can be used to identify the corresponding control.
  • the control determines the plurality of controls to be determined, so as to determine the control representing the user's actual control purpose from the plurality of controls to be determined as the target control, so that the electronic device can accurately determine the user's actual control intention.
  • the provided voice control method may be executed by an electronic device.
  • all the steps in the voice control method provided in the embodiment of the present application may be executed by the electronic device.
  • the voice collection device of the electronic device 100 can collect voice control instructions, and transmit the collected voice collection instructions and the target interface to the processor, so that the processor can learn from the acquired voice control instructions.
  • the first control identifier and the second control identifier are obtained, and then the processor determines the target control from the target interface by using the first control identifier and the second control identifier, so as to execute the control operation corresponding to the target control.
  • the voice control method provided in the embodiment of the present application may also be executed by a server.
  • the electronic device can collect voice commands, and send the collected voice commands and the target interface to the server synchronously, and then the server executes the voice control method provided by the embodiment of the present application to The target control is determined, and then the server triggers the electronic device to execute the control operation corresponding to the target control.
  • it can also be executed cooperatively by the electronic device and the server. In the way that the electronic device and the server cooperate to execute, some steps in the voice control method provided by the embodiment of the present application are executed by the electronic device, while other parts of the steps are executed by the server.
  • the electronic device 100 may execute the voice control method including: obtaining the first control identifier and the second control identifier from the acquired voice control instruction, and then executing by the server 200 if the target interface includes There are controls corresponding to the first control identifier and controls corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the control corresponding to the second control identifier
  • the first control identifies a target control among the corresponding controls, and generates a corresponding control command based on the target control, and then returns the generated control command to the electronic device 100, and triggers the electronic device 100 to execute the received control command. Control instruction.
  • the steps performed by the electronic device and the server respectively are not limited to the method described in the above examples.
  • the electronic device can be dynamically adjusted according to the actual situation Steps performed by the device and the server respectively.
  • a voice control method provided by the present application, the method includes:
  • S110 Obtain a first control identifier and a second control identifier from the acquired voice control instruction.
  • the user can express his own control target by voice.
  • the electronic device may use the voice uttered by the user as a voice control instruction, and then determine the user's control target according to the received voice control instruction.
  • the control target can be understood as the control that the user actually wants to operate on the interface displayed by the electronic device.
  • the user may have been talking and sending out a voice message.
  • the electronic device can start to obtain the voice control instruction after obtaining the specified voice content.
  • the specified voice content can be configured by the user according to his needs.
  • the identification of the control corresponding to the control target of the voice control instruction can be further obtained from the voice control instruction as the first control identification, and the identification of the control corresponding to the control target of the voice control instruction can be obtained.
  • the identification of the control corresponding to the control target of the voice control instruction is used as the second control identification. That is to say, the second control identifier may be an identifier used to assist in confirming the control actually corresponding to the first control identifier.
  • the voice control instruction can be converted into corresponding text content, and then the text content is semantically understood, so as to obtain the first control identifier and the second control identifier.
  • the semantic extraction rules can be established in advance, and then the identification can be obtained from the text content based on the semantic extraction rules.
  • the sentence pattern adopted will be relatively fixed. For example, if the user wants to download application A, the sentence pattern that may be triggered is "click the download button of application A", which can be summarized as "action words + ⁇ XXX ⁇ +of+ ⁇ XXX ⁇ " . Alternatively, the triggered sentence pattern may be "download application program A". This kind of sentence pattern can be summarized as "action words + ⁇ XXX ⁇ ".
  • the words representing the action class in the text content can be obtained based on the semantic extraction rules, and then the first control ID and the second control ID can be determined according to the sequence relationship with the words of the action class.
  • Control ID For example, if the text content converted by the voice control command successfully matches the sentence "action words+ ⁇ XXX ⁇ + ⁇ + ⁇ XXX ⁇ ", the first " ⁇ XXX ⁇ ” after the action words can be The content in is identified as the second control, and the second " ⁇ XXX ⁇ " after the action words is identified as the first control. If the text content converted by the voice control command successfully matches the sentence "action word + ⁇ XXX ⁇ ", the action word can be identified as the first control identifier, and the " ⁇ XXX ⁇ after the action word " is identified as the second control ID.
  • the first word in the text content converted by the command can be extracted through the pre-trained neural network model.
  • a control identifier and a second control identifier are examples of control identifiers.
  • the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the second control identifier
  • the corresponding control determines a target control from a plurality of controls corresponding to the first control identifier, the target interface is the interface displayed when the voice control instruction is acquired, wherein the first control identifier corresponds to the control
  • the control corresponding to the voice control instruction is to be determined, and the second control identifies the corresponding control, and is used to determine a control representing an actual control target of the user from the controls to be determined as the target control.
  • the target interface is the interface displayed by the electronic device when the voice control command is obtained.
  • the electronic device can simultaneously start to recognize the target interface to obtain the controls included in the target interface.
  • the controls included in the target interface can be identified in various ways.
  • the target interface may be identified through code analysis.
  • the target interface can be identified based on code parsing based on Google accessibility service accessibility.
  • the ID, type and description information of the control may be corresponding to the identified control.
  • the description information corresponding to the control is used to represent the operations that the control can realize. For example, if the control is a name used to represent an application, the description information of the control will include the name of the represented application. Furthermore, if the control is used to trigger the download of the application program, the description information of the control includes the download.
  • the target interface may be recognized by means of image-text recognition (for example, optical character recognition).
  • image-text recognition for example, optical character recognition
  • a screenshot of the interface currently displayed by the electronic device can be taken.
  • image-text recognition is performed on the image obtained from the screenshot.
  • the position of the control and the description information of the control may be corresponding to the identified control.
  • the description information of the control may include the text displayed in the control.
  • the target interface may be recognized by means of icon recognition.
  • icon recognition it is also possible to take a screenshot of the interface currently displayed by the electronic device. Then perform icon recognition on the image obtained from the screenshot.
  • the position of the control and the description information of the control may be corresponding to the identified control.
  • the description information of the control may include the description content of the identified function of the control.
  • the target interface when there are multiple ways to identify the target interface to obtain the controls in the target interface and the description information corresponding to the controls, one of them can be selected according to the current actual needs or Multiple ways to identify the target interface. For example, if the target interface supports identifying the target interface based on code analysis, then the target interface may be identified directly through code analysis. If the target interface does not support the identification of the interface through code analysis, the target interface can be identified jointly by means of graphic and text recognition and icon recognition.
  • the electronic device may also determine whether the target interface supports identification of controls through code analysis in a variety of ways.
  • a data table may be stored in the electronic device, and a list of application programs supporting code identification may be stored in the data table.
  • the electronic device may first inquire whether the application program to which the target interface to be identified belongs is stored in the data table. If the data table contains the application program to which the target interface to be identified belongs, then it is determined that the target interface supports identification of the target interface based on code analysis, and then the target interface can be identified directly through code analysis.
  • the target interface does not necessarily support the identification of the target interface based on code analysis. After determining that the target interface does not necessarily support the identification of the target interface based on the code analysis method, you can first try to identify the target interface through the code analysis method. If you can identify the control and the corresponding ID, type, and description information, etc., then determine The target interface supports the identification of the target interface based on the code analysis method. After the recognition result is obtained, the application program to which the target interface belongs can also be added to the data table.
  • the target interface does not support identification of the target interface based on code analysis. Furthermore, the target interface can be identified jointly by means of image-text recognition and icon recognition.
  • the identification of the target interface After the identification of the target interface is completed, it may be confirmed according to the identified controls in the target interface whether the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier. And when it is confirmed that the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, it may be based on the first control identifier The second control identifier determines a target control from multiple controls corresponding to the first control identifier.
  • the controls included in the target interface and the description information of the controls can be obtained. Then, in the process of detecting whether the target interface includes controls corresponding to the first control ID and the second control ID, the first control ID and the second control ID can be matched with the description information of the controls identified from the target control , if the description information of the control can be successfully matched with the first control identifier, it is determined that there is a control corresponding to the first control identifier in the target interface. If the description information of any control can be successfully matched with the second control identifier, it is determined that there is a control corresponding to the second control identifier in the target interface. Moreover, the number of controls corresponding to the first control identifier and the number of controls corresponding to the second control identifier may also be determined by the number of successful matches.
  • the text matching of the first control identifier and the second control identifier and the description information may be directly performed. In this manner, if it is determined that the content of the first control identifier and the description information are the same, it is determined that the description information matches the first control identifier successfully. Furthermore, if it is determined that the contents of the second control identifier and the description information are the same, it is determined that the description information matches the second control identifier successfully.
  • the first control identifier, the second control identifier and the description information may be respectively converted into corresponding pinyin content.
  • the pinyin content corresponding to the first control identifier is the first pinyin content
  • the pinyin content corresponding to the second control identifier is the second pinyin content
  • the pinyin content corresponding to the description information is the third pinyin content
  • the content and the second pinyin content will also perform phoneme replacement based on the phoneme replacement table, and the pinyin content after phoneme replacement of the first pinyin content will be used as the first replacement pinyin content, and the pinyin content after phoneme replacement of the second pinyin content will be used As the second replacement pinyin content.
  • the first pinyin content, the second pinyin content, the first alternate pinyin content, and the second alternate pinyin content are matched with the third pinyin content.
  • the control corresponding to the third pinyin content is used as the control corresponding to the first control identifier; if there is no third pinyin content successfully matched with the first pinyin content, then Match the first alternate pinyin content with the third pinyin content, if there is a first alternate pinyin content that successfully matches the first pinyin content, the description corresponding to the first alternate pinyin content that successfully matches the first pinyin content
  • the control corresponding to the information is used as the control corresponding to the first control identifier; otherwise, it is determined that there is no control corresponding to the first control identifier in the target interface.
  • the control corresponding to the third pinyin content is used as the control corresponding to the second control identifier, if there is no third pinyin content successfully matched with the second pinyin content, then Match the second alternate pinyin content with the third pinyin content, if there is a second alternate pinyin content that successfully matches the second pinyin content, the description corresponding to the second alternate pinyin content that successfully matches the second pinyin content
  • the control corresponding to the information is used as the control corresponding to the second control identifier, otherwise, it is determined that there is no control corresponding to the second control identifier in the target interface.
  • the voice control instruction triggered by the user is "install application A”
  • the first control identifier obtained according to the method in the embodiment of the present application may be "Install Application A”.
  • the second control is identified as Application A.
  • the interface diagram shown on the right side of FIG. 4 shows that there are 8 installed controls included in the description information in the interface currently displayed by the electronic device. Therefore, it may not be very clear if only relying on the first control identification itself. Determine which application the user wants to install. Then, combined with the content of the second control identifier of the application program A, it can be determined that what the user wants to trigger is the installation related to the application program A.
  • the recognition result can be stored, so that when the control description information of the same target interface needs to be obtained next time, the previous recognition can be obtained directly.
  • Recognition results instead of real-time recognition, to improve the efficiency of responding to user operations.
  • the electronic device can perform a control operation corresponding to the target control.
  • a control command corresponding to a control operation corresponding to the target control may be first generated, and then the electronic device triggers execution of a control operation corresponding to the target control by executing the control command.
  • the control instruction corresponding to the control operation corresponding to the target control can be generated by system injection (an operation mode supported by Android) or by simulating screen click.
  • the first control identifier and the second control identifier are obtained from the acquired voice control instruction, and the control corresponding to the first control identifier is the undetermined control corresponding to the voice control instruction.
  • the control corresponding to the second control identifier is used to determine the control that represents the user's actual control target as the target control from the controls to be determined, if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier control, and there are multiple controls corresponding to the first control identifier, the target control will be determined from multiple controls corresponding to the first control identifier based on the second control identifier, and the control corresponding to the target control will be executed operate.
  • the second control identifier can be used to identify the corresponding control.
  • the control determines the plurality of controls to be determined, so as to determine the control representing the user's actual control purpose from the plurality of controls to be determined as the target control, so that the electronic device can accurately determine the user's actual control intention.
  • a voice control method provided by the present application, the method includes:
  • S210 Obtain a first control identifier and a second control identifier from the acquired voice control instruction, wherein the control corresponding to the first control identifier is a control to be determined corresponding to the voice control instruction, and the second control identifier The corresponding control is used to determine, from the controls to be determined, a control representing the user's actual control target as the target control.
  • the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, if the second control identifier If there is only one corresponding control, then based on the control corresponding to the second control identifier, the target control is determined from multiple controls corresponding to the first control identifier, and the target interface is the one created when the voice control instruction is obtained. displayed interface.
  • the control that the user wants to actually touch and the control corresponding to the second control identifier usually have a certain relationship.
  • the voice control instruction triggered by the user is "install application program B”
  • the acquired first control identifier is installation
  • the second control identifier is application program B.
  • the association between controls may include a distance between controls or a containment relationship between controls, and the like.
  • some controls may have somewhat similar controls in the target interface.
  • the similarity may be a relatively similar display style, or may also be a relatively similar display position, and furthermore, may also be a relatively similar inclusion relationship with other controls.
  • the control that the user wants to actually touch can be selected as the target control from the plurality of controls corresponding to the first control identifier in combination with controls similar to the control corresponding to the second control identifier.
  • the target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier , including: if there is one control corresponding to the second control identifier, and the control corresponding to the second control identifier corresponds to a similar control, acquiring a control similar to the control corresponding to the second control identifier as the first Similar controls: determining a target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier and the first similar control. Wherein, there may be one or more first similar controls.
  • the target control is determined from a plurality of controls corresponding to the first control identifier based on the control corresponding to the second control identifier , including: if there is one control corresponding to the second control identifier, and the control corresponding to the second control identifier does not correspond to a similar control, based on the control corresponding to the second control identifier, select from multiple The target control is determined among the controls corresponding to the first control identifier.
  • the target interface can be identified based on code analysis, so as to obtain the ID, type, position, size, inclusion relationship and description of the controls included in the target interface information, etc., and then build a control relationship structure diagram based on the ID, type, position, size, inclusion relationship, and description information of the identified controls.
  • the control relationship structure diagram there are multiple nodes, and each node represents a control. And, the control corresponding to the child node is included in the control corresponding to the parent node corresponding to the child node. It should be noted that in the control relationship structure diagram, child nodes and parent nodes exist relatively.
  • control corresponding to a certain node is included in the control corresponding to a node adjacent to this node, then a certain node of this node It is the child node relative to the adjacent node, and correspondingly, the adjacent node is the parent node of the certain node.
  • analyzing the interface shown in FIG. 6 can obtain the control relationship structure diagram shown in FIG. 7 .
  • node 2 is adjacent to node 5, and from top to bottom, the lower the level of the control corresponding to the node, the level of node 2 is higher than the level of node 5, and node 5 is relative to node 2 is a child node, and node 2 is a parent node relative to node 5.
  • the node arranged at the top is the root node, and the root node in the control relationship structure diagram represents the most basic control in the target interface.
  • the controls except for the most basic control, all other controls are included in the most basic control.
  • the level of the control represented by the node whose arrangement position is closer to the top is closer to the most basic control.
  • obtaining a control similar to the control corresponding to the second control identifier as the first similar control may include: based on the attributes of the control corresponding to the second control identifier, in the control relationship structure diagram Find a control similar to the control corresponding to the second control identifier as the first similar control, and the attributes include the distance from the node corresponding to the control to the root node, the type of control, the length and width of the control, and the control in the corresponding parent At least one of the relative positions in the control.
  • the distance from the node corresponding to the control to the root node represents the number of jumps required in the process of jumping from the node corresponding to the control to the root node. For example, where node 1 in FIG. 7 needs to jump to the root node once, then the distance between node 1 and the root node is 1. Node 8 needs to jump 5 times to jump to the root node, so the distance from node 8 to the root node is 5. For another example, it takes 3 jumps from node 5 to jump to the root node, then the distance from node 5 to the root node is 3, similarly, the distances from node 6 and node 8 to the root node are both 3.
  • the type of the control may represent the use of the control in the interface.
  • the controls included in the interface can be divided into controls for outputting content, controls for displaying content, and controls for interacting with users.
  • the control for outputting content may be a text box.
  • the control for displaying content may be a control for displaying pictures or text content.
  • Controls for interacting with users may include buttons and the like.
  • the length and width of the control represent the size of the control itself.
  • the relative position of the control in the corresponding parent control can be understood as the relative display position of the control in the parent control when it is displayed in the interface.
  • the control 1 includes a control 11 and a control 12 , wherein the control 11 is used to display an icon control, a name control and an installation trigger control corresponding to the application A.
  • the control 12 is used to display the icon control, the name control and the installation trigger control corresponding to the application program B.
  • the icon control corresponding to the application program A is displayed at the relative position in the control 11
  • the icon control corresponding to the application program B is displayed at the same relative position in the control 12 .
  • controls similar to the control corresponding to the second control identifier may be selected as the first similar controls based on the properties of the controls.
  • the first similar control may be obtained based on one item in the attribute of the control, or may be obtained based on multiple items in the attribute.
  • the first similar control may be filtered based on the distance from the node corresponding to the control included in the attribute to the root node. For example, referring to FIG. 7 again, if the second control is identified as application A, then the node corresponding to application A is node 5 .
  • the distance from node 5 to the root node is 3, and other nodes with a distance of 3 to the root node include at least node 6 and node 7, then it can be determined that the controls corresponding to node 6 and node 7 are the first similar controls.
  • the icon control corresponding to application A and the icon control corresponding to application B in Figure 8 can be determined to be similar controls . Then, if the icon control corresponding to the application program A is a control corresponding to the second control identifier, then it can be determined that the icon control corresponding to the application program B is a similar control.
  • determining the target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier and the first similar control includes :
  • S221 Obtain the distances between the first node and the plurality of second nodes in the control relationship structure graph respectively, and obtain the plurality of first distances, the first nodes are used to represent the control corresponding to the second control identifier, and the first node The two nodes are used to represent the control corresponding to the first control identifier.
  • the obtaining the distances between the first node in the control relationship structure diagram and the multiple second nodes respectively, to obtain multiple first distances including:
  • the distance guarantees the number of layers that correspond to two nodes jumping mutually; the distance from the first node to the nearest common parent node, and the The sum of the distances from the second node currently performing the first distance calculation to the nearest common parent node is used as the distance between the second node currently performing the first distance calculation and the first node, so as to obtain multiple first distance.
  • the nodes corresponding to the first control identifier are node 8 , node 9 and node 10 .
  • the node corresponding to the second control identifier is node 5 .
  • the first node includes node 5, and the second node includes node 8, node 9, and node 10, and then the first distance corresponding to node 5 and node 8, and the first distance corresponding to node 5 and node 9 will be obtained respectively.
  • the first distances corresponding to nodes 5 and 10 so as to obtain multiple first distances.
  • the public parent nodes corresponding to node 5 and node 8 are node 2, node 1 and the root node, but node 2 is the parent node closest to node 5 and node 8, then node 2 is the nearest public parent node corresponding to node 5 and node 8 parent node.
  • the distance from node 5 to node 2 is 1, and the distance from node 8 to node 2 is 3, then the first distance corresponding to node 5 and node 8 is 4.
  • the common parent nodes corresponding to nodes 5 and 9 include node 1 and the root node, but node 1 is the closest parent node to nodes 5 and 9, so node 1 is the closest common parent node corresponding to nodes 5 and 9.
  • the distance from node 5 to node 1 is 2, and the distance from node 9 to node 1 is 4, then the first distance corresponding to node 5 and node 9 is 6.
  • the common parent nodes corresponding to node 5 and node 10 include node 1 and the root node, but node 1 is the closest parent node to node 5 and node 10, so node 1 is the closest common parent node corresponding to node 5 and node 10.
  • the distance from node 5 to node 1 is 2, and the distance from node 10 to node 1 is 4, then the first distance corresponding to node 5 and node 10 is 6.
  • S222 Obtain the distances between the third nodes in the control relationship structure diagram and the plurality of second nodes respectively, to obtain the plurality of second distances, the third nodes being nodes corresponding to the first similar controls.
  • the obtaining the distances between the third node in the control relationship structure diagram and the plurality of second nodes respectively to obtain the plurality of second distances includes: obtaining the distance between the third node in the control relationship structure diagram and the current second node The nearest common parent node corresponding to the second node for distance calculation; obtaining the distance from the third node to the nearest common parent node, and the second node currently performing the second distance calculation to the nearest common parent node The distance, the distance guarantees the number of levels that correspond to two nodes jumping to each other; The sum of the distances of the closest common parent nodes is used as the distance between the second node currently performing the second distance calculation and the third node, so as to obtain multiple second distances.
  • the first node still includes node 5, and the second node includes node 8, node 9, and node 10 as an example, if the distance to the root node is the same to determine the first similar control, then the determined nodes corresponding to the first similar control include node 6 and node 7. Then, the second distance between node 6 and node 8, the second distance between node 6 and node 9, and the second distance between node 6 and node 10 can be calculated according to the aforementioned visit time. Furthermore, the second distance between node 7 and node 8, the second distance between node 7 and node 9, and the second distance between node 7 and node 10 are calculated, thereby obtaining multiple second distances.
  • the manner of calculating the second distance in the embodiment itself is the same as the manner of calculating the first distance, and will not be described in detail here.
  • the calculated second distance between node 6 and node 8 is 6, the second distance between node 6 and node 9 is 4, and the second distance between node 6 and node 10 is 6.
  • the calculated second distance between node 7 and node 8 is 6, the second distance between node 7 and node 9 is 6, and the second distance between node 7 and node 10 is 4.
  • S223 Acquire multiple reference distances, where the multiple reference distances include the multiple first distances and the multiple second distances.
  • S224 Detect whether the minimum value among the multiple reference distances is consistent with the minimum value among the multiple first distances, and the minimum value is one.
  • the minimum value among the multiple reference distances is 4, and the minimum value among the multiple first distances is also 4, then it can be determined that the minimum value among the multiple reference distances is the same as that among the multiple first distances.
  • the minimum values of the distances are consistent, and then the control corresponding to the minimum value among the multiple first distances among the multiple controls corresponding to the first control identifier can be used as the target control.
  • the controls corresponding to the second control identifier include controls corresponding to node 8, node 9 and node 10 in FIG.
  • the control corresponding to node 8 is a control corresponding to the minimum value among the multiple first distances and the second control identifier, so that the control corresponding to node 8 can be used as the target control.
  • the second similar control is a control selected from the control relationship structure diagram based on the attributes of the control corresponding to the first control identifier.
  • the control that the user wants to trigger may be different from the control involved in the control target expressed by the user through the voice control instruction.
  • the operation control corresponding to application program A is update
  • the operation control corresponding to application program B is update
  • the operation control corresponding to application program C is installation.
  • the control relationship structure diagram obtained by identifying the controls in FIG. 10 may be as shown in FIG. 11 . Based on the control relationship structure diagram shown in FIG. 11 , if the voice control instruction issued by the user is "update application C", then the acquired first control is marked as update, and the second control is marked as application C.
  • the minimum value among the multiple reference distances is not consistent with the minimum value among the multiple first distances, and it can be found that the multiple first distances If the minimum value in the distance is greater than the minimum value in multiple reference distances, then a similar control selected from the control relationship structure diagram shown in FIG. Two similar controls. For example, if the second similar control is selected based on the same distance from the control to the root node (node 1 in FIG. 11 ), then the control corresponding to node 11 whose distance to the root node is also 4 can be used as the second similar control.
  • S227 Acquire a third distance, where the third distance includes a distance from a node corresponding to the second similar control to a node corresponding to the second control identifier.
  • the distance between the node 11 corresponding to the second similar control and the node 7 corresponding to the second control identifier can be obtained as 4, that is, the obtained third distance is 4.
  • the only consistency can be understood as being consistent and having only one consistency.
  • the only consistency can be understood as being consistent and having only one consistency.
  • the minimum value among the multiple reference distances is also 4, so it can be determined that there is a distance uniquely consistent with the minimum value among the multiple reference distances among the third distances. Therefore, the control corresponding to the node (node 11) corresponding to the unique distance can be used as the target control. Therefore, by obtaining the second similar control, even in the case that the user makes a mistake in the voice control command, the electronic device can intelligently correct the error in the user's voice control command, thereby improving the accuracy of execution. The probability that the user actually intended.
  • the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there is one control corresponding to the first control identifier, identify the corresponding control of the first control as the target control.
  • the minimum value among the plurality of reference distances is consistent with the minimum value among the plurality of first distances, but the minimum value may not be unique.
  • the control corresponding to the minimum value among the multiple first distances can be the control corresponding to node 8 or the control corresponding to node 9 after calculation in the aforementioned manner, so it cannot be directly determined out of the target control. Then, if the target control cannot be determined automatically through the method provided by the embodiment of the present application, the target control can be determined by asking the user.
  • control identifiers are used to determine the target control from the controls corresponding to the multiple first control identifiers, and the target control may be determined by asking the user.
  • This embodiment provides a voice control method, so that in the above-mentioned manner, there are multiple undetermined controls (controls corresponding to the first control identifier) corresponding to the voice control instruction, so that the actual control intention of the user cannot be clarified
  • the plurality of controls to be determined can be determined by means of the control corresponding to the second control identifier, so that the control representing the user's actual control purpose can be determined from the plurality of controls to be determined as the target control, so that the electronic device can accurately determine the user's actual control intent.
  • a control relationship structure diagram can be established based on the mutual inclusion relationship of controls in the target interface, so that the control corresponding to the second control identifier and the first control can be calculated by means of the control relationship structure diagram.
  • the distance between the similar controls and the controls corresponding to the first control identifier and then based on the distance, determine the target control from a plurality of controls corresponding to the first control identifier, so that the electronic device can be more convenient And accurately determine the target control.
  • a voice control method provided by this application, the method includes:
  • S310 Obtain a first control identifier and a second control identifier from the acquired voice control instruction, wherein the control corresponding to the first control identifier is a control to be determined corresponding to the voice control instruction, and the second control identifier The corresponding control is used to determine, from the controls to be determined, a control representing the user's actual control target as the target control.
  • the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, acquire in the target interface , the display distance between each of the plurality of controls corresponding to the first control identifier and the control corresponding to the second control identifier, and the target interface is the interface displayed when the voice control instruction is acquired.
  • the display distance represents the pixel distance between controls in the target interface.
  • the display distance between the two controls may include the distance between the center coordinates of the two controls.
  • the display distance between the control 20 and the control 21 is d1
  • the display distance between the control 21 and the control 22 is d2.
  • This embodiment provides a voice control method, so that in the above-mentioned manner, there are multiple undetermined controls (controls corresponding to the first control identifier) corresponding to the voice control instruction, so that the actual control intention of the user cannot be clarified
  • the plurality of controls to be determined can be determined by means of the control corresponding to the second control identifier, so that the control representing the user's actual control purpose can be determined from the plurality of controls to be determined as the target control, so that the electronic device can accurately determine the user's actual control intent.
  • multiple controls corresponding to the first control identifier can be selected directly based on the display distance between each of the controls corresponding to the first control identifier and the control corresponding to the second control identifier.
  • the target control is determined in the control, which improves the flexibility of obtaining the target control.
  • a voice control device 400 provided by the present application, the device 400 includes:
  • An identification obtaining unit 410 configured to obtain a first control identification and a second control identification from the acquired voice control instruction, wherein the control corresponding to the first control identification is a control to be determined corresponding to the voice control instruction, so The control corresponding to the second control identifier is used to determine, from the controls to be determined, a control representing the user's actual control target as the target control.
  • the control determining unit 420 is configured to: if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the The control corresponding to the second control identifier determines a target control from multiple controls corresponding to the first control identifier, and the target interface is the interface displayed when the voice control instruction is acquired.
  • the control unit 430 is configured to execute a control operation corresponding to the target control.
  • control determining unit 420 is specifically configured to, if there is only one control corresponding to the second control identification, select from a plurality of controls corresponding to the first control identification based on the control corresponding to the second control identification. Control to determine the target control.
  • control determining unit 420 is specifically configured to: if there is one control corresponding to the second control identification, and the control corresponding to the second control identification does not correspond to a similar control, based on the second control identification The corresponding control determines a target control from multiple controls corresponding to the first control identifier.
  • control determining unit 420 is specifically configured to, if there is one control corresponding to the second control identification, and the control corresponding to the second control identification corresponds to a similar control, obtain the control corresponding to the second control identification.
  • a control similar to the corresponding control is used as a first similar control; and based on the control corresponding to the second control identifier and the first similar control, a target control is determined from a plurality of controls corresponding to the first control identifier.
  • a control similar to the control corresponding to the second control identifier is searched in the control relationship structure diagram as the first similar control, and the attributes include At least one of the distance from the node corresponding to the control to the root node, the type of the control, the length and width of the control, and the relative position of the control in the corresponding parent control; wherein, the control relationship structure diagram is based on the target interface Generated by the containment relationship of the controls, the controls corresponding to the child nodes in the control relationship structure diagram are included in the controls corresponding to the parent nodes corresponding to the child nodes.
  • control determination unit 420 is specifically configured to obtain the distances between the first node and the multiple second nodes in the control relationship structure diagram, and obtain multiple first distances, and the first nodes are used to represent the second nodes.
  • the control corresponding to the control identifier, the second node is used to represent the control corresponding to the first control identifier; obtain the distance between the third node in the control relationship structure diagram and the plurality of second nodes respectively, and obtain the plurality of second distances , the third node is the node corresponding to the first similar control; multiple reference distances are obtained, and the multiple reference distances include the multiple first distances and the multiple second distances; if the multiple reference The minimum value in the distance is consistent with the minimum value in the plurality of first distances, and the number of the minimum value is one, then the first control is identified among the corresponding plurality of controls, and the plurality of first The control corresponding to the minimum value in the distance is used as the target control.
  • the first distance is to obtain the second similar control, and the second similar control is a control selected from the control relationship structure diagram based on the attribute of the corresponding control identified by the first control; to obtain the third distance, the third The distance includes the distance from the node corresponding to the second similar control to the node corresponding to the second control identifier; if there is a distance in the third distance that is uniquely consistent with the minimum value among the plurality of reference distances, the The control corresponding to the unique and consistent distance mentioned above is used as the target control.
  • control determination unit 420 is specifically configured to obtain the closest common parent node corresponding to the first node in the control relationship structure graph and the second node currently performing the first distance calculation; obtain the first node to the The distance of the closest common parent node, and the distance from the second node currently performing the first distance calculation to the closest common parent node, the distance guarantees the number of levels that correspond to two nodes jumping to each other; the first The distance from the node to the nearest common parent node, and the sum of the distances from the second node currently performing the first distance calculation to the nearest common parent node are used as the second node currently performing the first distance calculation and the second node A distance between nodes to obtain multiple first distances.
  • control determining unit 420 is specifically configured to obtain the nearest common parent node corresponding to the third node in the control relationship structure diagram and the second node currently performing the second distance calculation; obtain the third node to the The distance of the closest public parent node, and the distance from the second node currently performing the second distance calculation to the nearest common parent node, the distance guarantees the number of levels corresponding to the two nodes jumping each other; the third The distance from the node to the nearest common parent node, and the sum of the distances from the second node currently performing the second distance calculation to the nearest common parent node are used as the second node currently performing the second distance calculation and the second node The distance between three nodes to get multiple second distances.
  • control determining unit 420 is specifically configured to: if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there is one control corresponding to the first control identifier, A control corresponding to the first control identifier is used as a target control.
  • control determining unit 420 is specifically configured to acquire, in the target interface, the display distance between each of the controls corresponding to the first control identifier and the control corresponding to the second control identifier; Taking the control corresponding to the smallest display distance among the controls corresponding to the first control identifier as the target control.
  • the voice control device provided in this embodiment first obtains the first control identifier and the second control identifier from the acquired voice control instruction, and the control corresponding to the first control identifier is the undetermined control corresponding to the voice control instruction.
  • the control corresponding to the second control identifier is used to determine the control that represents the user's actual control target as the target control from the controls to be determined, if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier control, and there are multiple controls corresponding to the first control identifier, the target control will be determined from multiple controls corresponding to the first control identifier based on the second control identifier, and the control corresponding to the target control will be executed operate.
  • the second control identifier can be used to identify the corresponding control.
  • the control determines the plurality of controls to be determined, so as to determine the control representing the user's actual control purpose from the plurality of controls to be determined as the target control, so that the electronic device can accurately determine the user's actual control intention.
  • each functional module in each embodiment of the present application may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules.
  • an embodiment of the present application also provides an electronic device 1000 capable of executing the aforementioned voice control method.
  • the electronic device 1000 includes one or more (only one is shown in the figure) processors 102 , a memory 104 , a camera 106 and an audio collection device 108 coupled to each other.
  • the memory 104 stores programs capable of executing the contents of the foregoing embodiments, and the processor 102 can execute the programs stored in the memory 104 .
  • the processor 102 may include one or more processing cores.
  • the processor 102 uses various interfaces and circuits to connect various parts of the entire electronic device 1000, and executes or executes instructions, programs, code sets, or instruction sets stored in the memory 104, and calls data stored in the memory 104 to execute Various functions of the electronic device 1000 and processing data.
  • the processor 102 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware.
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 102 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, and the like.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • the CPU mainly handles the operating system, user interface and application programs, etc.
  • the GPU is used to render and draw the displayed content
  • the modem is used to handle wireless communication.
  • the processor 102 may be a neural network chip.
  • it may be an embedded neural network chip (NPU).
  • the memory 104 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory). Memory 104 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. For example, a device may be stored in memory 104 . The device may be the aforementioned device 400 .
  • the memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like.
  • the electronic device 1000 may further include a network module 110 and a sensor module 112 in addition to the aforementioned components.
  • the network module 110 is used to implement information interaction between the electronic device 1000 and other devices, for example, transmitting device control instructions, manipulation request instructions, and status information acquisition instructions. However, when the electronic device 200 is specifically a different device, its corresponding network module 110 may be different.
  • the sensor module 112 may include at least one sensor. Specifically, the sensor module 112 may include, but is not limited to: a level, a light sensor, a motion sensor, a pressure sensor, an infrared heat sensor, a distance sensor, an acceleration sensor, and other sensors.
  • the pressure sensor may be a sensor for detecting pressure generated by pressing on the electronic device 1000 . That is, the pressure sensor detects pressure generated by contact or press between the user and the electronic device, eg, contact or press between the user's ear and the mobile terminal. Therefore, the pressure sensor can be used to determine whether contact or pressure occurs between the user and the electronic device 1000, and the magnitude of the pressure.
  • the acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when it is still, and can be used to identify the application of electronic equipment 1000 attitude (such as horizontal and vertical screen switching, related games, magnetometer, etc.) Attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.
  • the electronic device 1000 may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, and a thermometer, which will not be repeated here.
  • the audio collection device 110 is configured to collect audio signals.
  • the audio collection device 110 includes multiple audio collection devices, and the audio collection devices may be microphones.
  • the network module of the electronic device 1000 is a radio frequency module, and the radio frequency module is used to receive and send electromagnetic waves, realize mutual conversion between electromagnetic waves and electrical signals, and communicate with a communication network or other devices.
  • the radio frequency module may include various existing circuit elements for performing these functions, such as antenna, radio frequency transceiver, digital signal processor, encryption/decryption chip, Subscriber Identity Module (SIM) card, memory and so on.
  • SIM Subscriber Identity Module
  • the radio frequency module can interact with external devices by sending or receiving electromagnetic waves.
  • a radio frequency module can send instructions to a target device.
  • FIG. 17 shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application.
  • Program codes are stored in the computer-readable medium 800, and the program codes can be invoked by a processor to execute the methods described in the foregoing method embodiments.
  • the computer readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium).
  • the computer-readable storage medium 800 has a storage space for program code 810 for executing any method steps in the above-mentioned methods. These program codes can be read from or written into one or more computer program products.
  • Program code 810 may, for example, be compressed in a suitable form.
  • the first control identifier and the second control identifier are first obtained from the acquired voice control instruction, and the control corresponding to the first control identifier is It is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target as the target control from the control to be determined, if the target interface includes The control corresponding to the identification and the control corresponding to the second control identification, and there are multiple controls corresponding to the first control identification, then the target will be determined from multiple controls corresponding to the first control identification based on the second control identification control, and perform a control operation corresponding to the target control.
  • the second control identifier can be used to identify the corresponding control.
  • the control determines the plurality of controls to be determined, so as to determine the control representing the user's actual control purpose from the plurality of controls to be determined as the target control, so that the electronic device can accurately determine the user's actual control intention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种语音控制方法、装置、电子设备及存储介质。该方法包括:从获取的语音控制指令中获取第一控件标识和第二控件标识（S110）；若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件的有多个，基于第二控件标识从多个与第一控件标识对应的控件中确定目标控件，目标界面为获取到语音控制指令时所显示的界面，其中，第一控件标识对应的控件为与语音控制指令对应的待确定控件，第二控件标识对应的控件用于从待确定控件中确定表征用户实际控制目标的控件作为目标控件（S120）；执行与目标控件对应的控制操作（S130）。通过该方式使得电子设备可以更为准确的确定用户的实际控制意图。

Description

语音控制方法、装置、电子设备及存储介质
相关申请的交叉引用
本申请要求于2021年12月9日提交的申请号为202111500093.0的中国申请的优先权，其在此出于所有目的通过引用将其全部内容并入本文。
技术领域
本申请涉及计算机技术领域，更具体地，涉及一种语音控制方法、装置、电子设备及存储介质。
背景技术
结合人工智能技术以及虚拟个人助理(语音助手)，可以使得电子设备通过听觉模态接收用户发出的语音指令并完成对应的交互任务。然而，在很多情况下，用户在看到交互界面后才会明确自己的交互意图，并希望对所看到的交互界面或其中的对象进行直接操作。并且，在一些情况下，交互界面中可能会存在多个与用户触发的语音指令匹配的控件，进而会造成电子设备无法准确的确定用户的实际控制意图。
发明内容
鉴于上述问题，本申请提出了一种语音控制方法、装置、电子设备及存储介质，以实现改善上述问题。
第一方面，本申请提供了一种语音控制方法，所述方法包括：从获取的语音控制指令中获取第一控件标识和第二控件标识；若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，所述目标界面为获取到所述语音控制指令时所显示的界面，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件；执行与所述目标控件对应的控制操作。
第二方面，本申请提供了一种语音控制装置，所述装置包括：标识获取单元，用于从获取的语音控制指令中获取第一控件标识和第二控件标识；控件确定单元，用于若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，所述目标界面为获取到所述语音控制指令时所显示的界面，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件；控制单元，用于执行与所述目标控件对应的控制操作。
第三方面，本申请提供了一种电子设备，包括一个或多个处理器以及存储器；一个或多个程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序配置用于执行上述的方法。
第四方面，本申请提供的一种计算机可读存储介质，所述计算机可读存储介质中存储有程序代码，其中，在所述程序代码运行时执行上述的方法。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲， 在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例提出的一种语音控制方法的一种应用场景的示意图；
图2示出了本申请实施例提出的另一种语音控制方法的一种应用场景的示意图；
图3示出了本申请实施例提出的一种语音控制方法的流程图；
图4示出了本申请实施例中用户触发语音控制指令的示意图；
图5示出了本申请另一实施例提出的一种语音控制方法的流程图；
图6示出了本申请实施例中一种目标界面的示意图；
图7示出了本申请实施例中一种控件关系结构图的示意图；
图8示出了本申请实施例中一种控件的相对位置的示意图；
图9示出了本申请实施例中S220的一种实施方式的流程图；
图10示出了本申请实施例中另一种目标界面的示意图；
图11示出了本申请实施例中另一种控件关系结构图的示意图；
图12示出了本申请实施例中再一种控件关系结构图的示意图；
图13示出了本申请再一实施例提出的一种语音控制方法的流程图；
图14示出了本申请实施例中显示距离的示意图；
图15示出了本申请实施例提出的一种目标物识别装置的结构框图；
图16示出了本申请提出的一种电子设备的结构框图；
图17是本申请实施例的用于保存或者携带实现根据本申请实施例的语音控制方法的程序代码的存储单元。
具体实施方式
本申请实施例提供了一种语音控制方法，方法包括：从获取的语音控制指令中获取第一控件标识和第二控件标识；若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件有多个，基于第二控件标识对应的控件从多个与第一控件标识对应的控件中确定目标控件，目标界面为获取到语音控制指令时所显示的界面，其中，第一控件标识对应的控件为与语音控制指令对应的待确定控件，第二控件标识对应的控件用于从待确定控件中确定表征用户实际控制目标的控件作为目标控件；
执行与目标控件对应的控制操作。
可选的，基于第二控件标识对应的控件从多个与第一控件标识对应的控件中确定目标控件，包括：若第二控件标识对应的控件为一个，则基于第二控件标识所对应的控件从多个与第一控件标识对应的控件中确定目标控件。
可选的，若第二控件标识对应的控件为一个，则基于第二控件标识所对应的控件从多个与第一控件标识对应的控件中确定目标控件，包括：若第二控件标识对应的控件为一个，且第二控件标识所对应的控件未对应有相似控件，基于第二控件标识所对应的控件从多个与第一控件标识对应的控件中确定目标控件。
可选的，若第二控件标识对应的控件为一个，则基于第二控件标识所对应的控件从多个与第一控件标识对应的控件中确定目标控件，包括：若第二控件标识对应的控件为一个，且第二控件标识所对应的控件对应有相似控件，获取与第二控件标识所对应的控件相似的控件作为第一相似控件；基于第二控件标识所对应的控件以及第一相似控件，从多个与第一控件标识对应的控件中确定目标控件。
可选的，获取与第二控件标识所对应的控件相似的控件作为第一相似控件，包括：基于第二控件标识所对应的控件的属性，在控件关系结构图中查找与第二控件标识所对应的控件相似的控件作为第一相似控件，属性包括控件对应的节点到根节点的距离、控件的类型、控件的长宽以及控件在对应的父控件中的相对位置中的至少一项；其中，控件关系结构图为基于目标界面中控件的包含关系所生成，在控件关系结构图中子节点所对应的控件包含在子节点对应的父节点所对应的控件中。
可选的，基于第二控件标识所对应的控件以及第一相似控件，从多个与第一控件标识对应的控件中确定目标控件，包括：获取控件关系结构图中第一节点分别与多个第二节点之间的距离，得到多个第一距离，第一节点用于表征第二控件标识所对应的控件，第二节点用于表征第一控件标识对应的控件；获取控件关系结构图中第三节点分别与多个第二节点之间的距离，得到多个第二距离，第三节点为第一相似控件对应的节点；获取多个参考距离，多个参考距离包括多个第一距离和多个第二距离；若多个参考距离中的最小值与多个第一距离中的最小值一致，且该最小值的数量为一个，则将第一控件标识对应的多个控件中，与多个第一距离中最小值对应的控件作为目标控件。
可选的，方法还包括：若多个参考距离中的最小值与多个第一距离中的最小值不一致，且多个 第一距离没有与多个参考距离中的最小值相同的第一距离，获取第二相似控件，第二相似控件为基于第一控件标识对应的控件的属性从控件关系结构图中选择出的控件；
获取第三距离，第三距离包括第二相似控件对应的节点到第二控件标识对应的节点的距离；若第三距离中存在与多个参考距离中的最小值唯一一致的距离，将唯一一致的距离对应的控件作为目标控件。
可选的，获取控件关系结构图中第一节点分别与多个第二节点之间的距离，得到多个第一距离，包括：获取控件关系结构图中第一节点与当前进行第一距离计算的第二节点所对应的最近公共父节点；获取第一节点到最近公共父节点的距离，以及当前进行第一距离计算的第二节点到最近公共父节点的距离；将第一节点到最近公共父节点的距离，与当前进行第一距离计算的第二节点到最近公共父节点的距离之和作为当前进行第一距离计算的第二节点与第一节点之间的距离，以得到多个第一距离。
可选的，获取控件关系结构图中第三节点分别与多个第二节点之间的距离，得到多个第二距离，包括：获取控件关系结构图中第三节点与当前进行第二距离计算的第二节点所对应的最近公共父节点；获取第三节点到最近公共父节点的距离，以及当前进行第二距离计算的第二节点到最近公共父节点的距离，距离表征对应两个节点相互跳跃的层级的数量；将第三节点到最近公共父节点的距离，与当前进行第二距离计算的第二节点到最近公共父节点的距离之和作为当前进行第二距离计算的第二节点与第三节点之间的距离，以得到多个第二距离。
可选的，基于第二控件标识对应的控件从多个与第一控件标识对应的控件中确定目标控件，包括：获取在目标界面中，多个与第一控件标识对应的控件各自与第二控件标识对应控件之间的显示距离；将第一控件标识对应的控件中对应的显示距离最小的控件作为目标控件。
可选的，方法还包括：若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件有一个，将第一控件标识对应的控件作为目标控件。
可选的，方法还包括：若与第二控件标识所对应的控件有两个及以上，且第一控件标识对应的控件有两个及以上，则通过询问用户确定目标控件。
可选的，从获取的语音控制指令中获取第一控件标识和第二控件标识之前还包括：若获取到指定语音内容，则开始获取语音控制指令。
可选的，指定语音内容由用户进行配置。
可选的，从获取的语音控制指令中获取第一控件标识和第二控件标识，包括：将获取的语音控制指令转换为对应的文本内容；基于语义提取规则从文本内容中进行标识获取，以获取第一控件标识和第二控件标识。
可选的，方法还包括：在接收到语音控制指令后，同步开始对目标界面进行识别以获取到目标界面中所包括的控件。
可选的，对目标界面进行识别的方式包括：通过代码解析方式对目标界面进行识别；通过图文识别的方式对目标界面进行识别；或者通过图标识别的方式对目标界面进行识别。
本申请实施例提供了一种语音控制装置，装置包括：标识获取单元，用于从获取的语音控制指令中获取第一控件标识和第二控件标识；控件确定单元，用于若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件有多个，基于第二控件标识对应的控件从多个与第一控件标识对应的控件中确定目标控件，目标界面为获取到语音控制指令时所显示的界面，其中，第一控件标识对应的控件为与语音控制指令对应的待确定控件，第二控件标识对应的控件用于从待确定控件中确定表征用户实际控制目标的控件作为目标控件；控制单元，用于执行与目标控件对应的控制操作。
本申请实施例提供了一种电子设备，其特征在于，包括一个或多个处理器以及存储器；一个或多个程序被存储在存储器中并被配置为由一个或多个处理器执行，一个或多个程序配置用于执行本申请实施例提供的方法。
本申请实施例提供了一种计算机可读存储介质，计算机可读存储介质中存储有程序代码，其中，在程序代码运行时执行本申请实施例提供的方法。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
智能终端设备的普及给生活带来了种种便利。结合人工智能技术以及虚拟个人助理(语音助手)，可以使得电子设备通过听觉模态接收用户发出的语音指令并完成对应的交互任务。然而，在很多情况下，用户在看到交互界面后才会明确自己的交互意图，并希望对所看到的交互界面或其中的对象进行直接操作。
但是，发明人在研究中发现，在一些情况下，交互界面中可能会存在多个与用户触发的语音指令匹配 的控件，进而会造成电子设备无法准确的确定用户的实际控制意图。具体的，在电子设备所显示的界面中，可能会存在有多个名称相同的控件。并且，电子设备也识别到用户所发送的语音控制指令中包括有该多个同名的控件。因此，电子设备可能无法准确的确定用户实际是要对哪一个控件进行操作，因而使得电子设备无法准确的确定用户的实际控制意图。
因此，发明人提出了本申请中的一种语音控制方法、装置、电子设备及存储介质，该方法先从获取的语音控制指令中获取第一控件标识和第二控件标识，在第一控件标识对应的控件为与语音控制指令对应的待确定控件，第二控件标识对应的控件用于从待确定控件中确定表征用户实际控制目标的控件作为目标控件的情况下，若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件有多个，则会基于第二控件标识从多个与所述第一控件标识对应的控件中确定目标控件，并执行与所述目标控件对应的控制操作。
从而通过上述方式使得在与语音控制指令对应的待确定控件(第一控件标识对应的控件)有多个而造成无法明确用户的实际控制意图的情况下，可以再借助于第二控件标识对应的控件对多个待确定控件进行确定，从而从多个待确定控件中确定表征用户实际控制目的控件作为目标控件，进而使得电子设备可以准确的确定用户的实际控制意图。
下面先对本申请实施例所涉及的应用场景进行介绍。
在本申请实施例中，所提供的语音控制方法可以由电子设备执行。在由电子设备执行的这种方式中，本申请实施例提供的语音控制方法中所有步骤可以均由电子设备执行。例如，如图1所示，通过电子设备100的语音采集装置可以采集语音控制指令，并将采集到的语音采集指令以及目标界面均传输给处理器，使得处理器可以从获取的语音控制指令中获取第一控件标识和第二控件标识，进而处理器再利用第一控件标识和第二控件标识从目标界面中确定目标控件，以执行与所述目标控件对应的控制操作。
再者，本申请实施例提供的语音控制方法也可以由服务器进行执行。对应的，在由服务器执行的这种方式中，可以由电子设备采集语音指令，并将采集的语音指令以及目标界面同步发送给服务器，然后由服务器来执行本申请实施例提供的语音控制方法以确定目标控件，然后由服务器触发电子设备执行该目标控件对应的控制操作。另外，还可以由电子设备和服务器协同执行。在由电子设备和服务器协同执行的这种方式中，本申请实施例提供的语音控制方法中的部分步骤由电子设备执行，而另外部分的步骤则由服务器来执行。
示例性的，如图2所示，电子设备100可以执行语音控制方法包括的：从获取的语音控制指令中获取第一控件标识和第二控件标识，然后由服务器200来执行若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，并基于所述目标控件生成对应的控制指令，然后再将所生成的控制指令返回给电子设备100，并触发电子设备100执行所接收到的控制指令。
需要说明的是，在由电子设备和服务器协同执行的这种方式中，电子设备和服务器分别执行的步骤不限于上述示例中所介绍的方式，在实际应用中，可以根据实际情况动态的调整电子设备和服务器分别执行的步骤。
下面则结合附图来对本申请所涉及的实施例进行介绍。
请参阅图3，本申请提供的一种语音控制方法，所述方法包括：
S110：从获取的语音控制指令中获取第一控件标识和第二控件标识。
在本申请实施例中，用户可以通过语音来表达自己的控制目标。对应的，电子设备可以将用户所发出的语音作为语音控制指令，并再根据接收到的语音控制指令来确定用户的控制目标。其中，控制目标可以理解为在电子设备所显示界面中用户实际想操作的控件。需要说明的是，用户在使用电子设备的过程中，可能会一直在进行说话而发出语音信息，但是，用户在发出的语音信息时，可能只是在与别人对话，而并不一定是想对电子设备进行控制，那么为了避免电子设备进行误识别，电子设备可以在获取到指定语音内容后，再开始获取语音控制指令。其中，该指定语音内容可以由用户根据自己的需要进行配置。
在获取到语音控制指令后，可以进一步的从语音控制指令中，获取到用于获取与语音控制指令的控制目标所对应的控件的标识作为第一控件标识，以及获取用于对所述与所述语音控制指令的控制目标对应的控件进行确认的标识作为第二控件标识。也就是说，其中的第二控件标识可以为用于对第一控件标识实际所对应的控件进行辅助确认的标识。
并且，在本申请实施例中可以有多种的获取第一控件标识和第二控件标识的方式。
作为一种方式，可以将语音控制指令转换为对应的文本内容，然后对文本内容进行语义理解，从而获取第一控件标识和第二控件标识。在这种方式中，可以预先建立语义提取规则，然后基于该语义提取规则从文本内容中进行标识获取。需要说明的是，发明人经过研究发现，用户在触发语音控制指令时，所采用的句式会相对比较固定。例如，若用户希望下载应用程序A，那么可能所触发的句式为“点击应用程序A 的下载按钮”，这种句式可以总结为“动作类词语+{XXX}+的+{XXX}”。再或者，所触发的句式可以为“下载应用程序A”。这种句式可以总结为“动作类词语+{XXX}”。
在这种方式下，在获取到文本内容后可以基于语义提取规则对文本内容中表征动作类的词语进行获取，然后根据与该动作类的词语的前后顺序关系来确定第一控件标识和第二控件标识。例如，若语音控制指令所转换得到的文本内容与“动作类词语+{XXX}+的+{XXX}”这一句式成功匹配，则可以将动作类词语后的第一个“{XXX}”中的内容作为第二控件标识，而将动作类词语后的第二个“{XXX}”作为第一控件标识。若语音控制指令所转换得到的文本内容与“动作类词语+{XXX}”这一句式成功匹配，则可以将动作类词语识别为第一控件标识，而将动作类词语后的“{XXX}”识别为第二控件标识。
作为另外一种方式，若无法将语音控制指令所转换得到的文本内容与预先配置的句式进行成功匹配，则可以通过预先训练好的神经网络模型来提取指令所转换得到的文本内容中的第一控件标识以及第二控件标识。
S120：若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，所述目标界面为获取到所述语音控制指令时所显示的界面，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件。
可选的，目标界面为获取到语音控制指令时电子设备所显示的界面，电子设备在接收到语音控制指令后，可以同步开始对目标界面进行识别以获取到目标界面中所包括的控件。并且，在本申请实施例中，可以通过多种的方式来对目标界面中所包括的控件进行识别。
作为一种方式，可以通过代码解析方式对所述目标界面进行识别。可选的，可以基于Google无障碍服务accessibility实现基于代码解析方式对所述目标界面进行识别。在这种方式中，对于所识别出的控件可以对应有控件的ID、类型以及描述信息等。其中，控件对应的描述信息用于表征该控件可以实现的操作。例如，若控件为用于表征应用程序的名称，则该控件的描述信息中则会包括所表征应用程序的名称。再者，若控件是用于触发对应用程序的下载，则该控件的描述信息中包括有下载。
作为另外一种方式，可以通过图文识别(例如，光学字符识别)的方式对目标界面进行识别。在这种方式中，可以对电子设备当前所显示的界面进行截图。然后再对截图得到的图像进行图文识别。在这种方式中，对于所识别出的控件可以对应有控件的位置以及控件的描述信息。并且，在这种方式中，控件的描述信息可以包括有控件中所显示的文本。
作为另外一种方式，可以通过图标识别的方式对目标界面进行识别。在这种方式中，也是可以对电子设备当前所显示的界面进行截图。然后再对截图得到的图像进行图标识别。在这种方式中，对于所识别出的控件可以对应有控件的位置以及控件的描述信息。并且，在这种方式中，控件的描述信息可以包括所识别出的控件的功能的描述内容。
需要说明的是，在本申请实施例中，对目标界面进行识别以获取目标界面中的控件以及控件对应的描述信息有多种方式的情况下，可以根据当前的实际需求选择其中的一种或者多种方式对目标界面进行识别。例如，若目标界面支持基于代码解析方式对目标界面进行识别，那么则可以直接通过代码解析方式对所述目标界面进行识别。若目标界面不支持通过代码解析方式对界面进行识别，则可以采用通过图文识别的方式和图标识别的方式共同对目标界面进行识别。
在本申请实施例中，电子设备也可以通过多种方式来确定目标界面是否支持通过代码解析方式进行控件的识别。
作为一种方式，在电子设备中可以存储有数据表，在该数据表中可以存储有支持代码识别的应用程序的名单。在电子设备对目标界面进行识别之前，可以先查询该数据表中是否存储有所要进行识别的目标界面所属的应用程序。若该数据表中有该所要进行识别的目标界面所属的应用程序，则确定目标界面支持基于代码解析方式对目标界面进行识别，进而可以直接通过代码解析方式对目标界面进行识别。
若该数据表中没有该所要进行识别的目标界面所属的应用程序，则确定该目标界面不一定支持基于代码解析方式对目标界面进行识别。在确定目标界面不一定支持基于代码解析方式对目标界面进行识别后，可以先通过代码解析方式对目标界面进行尝试性识别，若能够识别出控件以及对应的ID、类型以及描述信息等，则确定目标界面支持基于代码解析方式对目标界面进行识别，在得到识别结果后，还可以将目标界面所属的应用程序添加到该数据表中。
若不能够识别出控件，则确定目标界面并不支持基于代码解析方式对目标界面进行识别。进而可以再通过图文识别的方式和过图标识别的方式共同对目标界面进行识别。
在完成对目标界面的识别后，则可以根据从目标界面中所识别出的控件来确认目标界面中是否包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件。并在确认出目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控 件有多个的情况下，可以基于第二控件标识从多个与所述第一控件标识对应的控件中确定目标控件。
其中，如前面介绍，在对目标进行识别后，可以得到目标界面中所包括的控件以及控件的描述信息。那么检测目标界面是否包括有第一控件标识和第二控件标识对应的控件的过程中，则可以将第一控件标识和第二控件标识分别与从目标控件中识别出的控件的描述信息进行匹配，若有控件的描述信息可以与第一控件标识成功匹配，则确定目标界面中有第一控件标识对应的控件。若有控件的描述信息可以与第二控件标识成功匹配，则确定目标界面中有第二控件标识对应的控件。并且，还可以通过成功匹配的数量来确定第一控件标识对应的控件的数量以及第二控件标识对应的控件的数量。
并且，在将第一控件标识和第二控件标识分别与从目标控件中识别出的控件的描述信息进行匹配的过程中，可以有多种的比对方式。
作为一种方式，可以直接将第一控件标识和第二控件标识与描述信息进行文本匹配。在这种方式中，若确定第一控件标识和描述信息的内容相同，则确定描述信息与第一控件标识匹配成功。再者，若确定第二控件标识和描述信息的内容相同，则确定描述信息与第二控件标识匹配成功。
作为再一种方式，可以将第一控件标识、第二控件标识与描述信息分别转换为对应的拼音内容。其中，第一控件标识所对应的拼音内容为第一拼音内容，第二控件标识所对应的拼音内容为第二拼音内容，描述信息对应的拼音内容为第三拼音内容，并且，对于第一拼音内容和第二拼音内容还会基于音素替换表进行音素替换，并将对第一拼音内容进行音素替换后的拼音内容作为第一替换拼音内容，将对第二拼音内容进行音素替换后的拼音内容作为第二替换拼音内容。然后，再将第一拼音内容、第二拼音内容、第一替换拼音内容以及第二替换拼音内容，与第三拼音内容进行匹配。
若有与第一拼音内容成功匹配的第三拼音内容，则将该第三拼音内容对应控件作为第一控件标识所对应的控件，若没有与第一拼音内容成功匹配的第三拼音内容，则将第一替换拼音内容与第三拼音内容进行匹配，若有与第一拼音内容成功匹配的第一替换拼音内容，则将该与第一拼音内容成功匹配的第一替换拼音内容所对应的描述信息所对应的控件作为第一控件标识所对应的控件，否则，确定目标界面中不存在与第一控件标识所对应的控件。
若有与第二拼音内容成功匹配的第三拼音内容，则将该第三拼音内容对应控件作为第二控件标识所对应的控件，若没有与第二拼音内容成功匹配的第三拼音内容，则将第二替换拼音内容与第三拼音内容进行匹配，若有与第二拼音内容成功匹配的第二替换拼音内容，则将该与第二拼音内容成功匹配的第二替换拼音内容所对应的描述信息所对应的控件作为第二控件标识所对应的控件，否者，确定目标界面中不存在与第二控件标识所对应的控件。
如图4所示，在图4所示的场景中，若用户触发的语音控制指令为“安装应用程序A”，那么根据本申请实施例中的方式所获取得到的第一控件标识可以为安装，第二控件标识为应用程序A。并且图4的右侧所示的界面图可知，在电子设备当前所显示的界面中描述信息中包括有安装的控件有8个，因此，如果仅仅依靠第一控件标识本身可能并不能很明确的确定用户是要安装哪个应用程序。那么再结合内容为应用程序A的第二控件标识，则可以明确用户想要触发的是与应用程序A有关的安装。
需要说明的是，在对目标界面进行识别以得到识别结果后，可以对识别结果进行存储，以便于在下一次需要获取同一个目标界面的控件的描述信息时，可以直接获取之前进行识别所得到的识别结果，而不用再实时进行识别，以提升响应用户操作的效率。
S130：执行与所述目标控件对应的控制操作。
在确定目标控件后，电子设备则可以执行与目标控件所对应的控制操作。作为一种方式，在确定目标控件后，可以先生成与目标控件对应的控制操作的所对应的控制指令，进而使得电子设备通过执行该控制指令的方式来触发执行与目标控件对应的控制操作。其中，可以通过系统注入(Android所支持的一种操作方式)或模拟屏幕点击的方法生成与目标控件对应的控制操作所对应的控制指令。
本实施例提供的一种语音控制方法，先从获取的语音控制指令中获取第一控件标识和第二控件标识，在第一控件标识对应的控件为与语音控制指令对应的待确定控件，第二控件标识对应的控件用于从待确定控件中确定表征用户实际控制目标的控件作为目标控件的情况下，若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件有多个，则会基于第二控件标识从多个与所述第一控件标识对应的控件中确定目标控件，并执行与所述目标控件对应的控制操作。
从而通过上述方式使得在与语音控制指令对应的待确定控件(第一控件标识对应的控件)有多个而造成无法明确用户的实际控制意图的情况下，可以再借助于第二控件标识对应的控件对多个待确定控件进行确定，从而从多个待确定控件中确定表征用户实际控制目的控件作为目标控件，进而使得电子设备可以准确的确定用户的实际控制意图。
请参阅图5，本申请提供的一种语音控制方法，所述方法包括：
S210：从获取的语音控制指令中获取第一控件标识和第二控件标识，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件。
S220：若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，若所述第二控件标识对应的控件为一个，则基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，所述目标界面为获取到所述语音控制指令时所显示的界面。
需要说明的是，在本申请实施例中在第一控件标识所对应的控件中，用户想实际触控的控件和第二控件标识所对应的控件通常是具有一定的关联的。例如，如图6所示，若用户触发的语音控制指令为“安装应用程序B”，则所获取到的第一控件标识为安装，第二控件标识为应用程序B。虽然在图5所示的界面中可以识别到与第一控件标识对应的控件会有三个。但是，用户实际想触发的是三个与第一控件标识对应的控件中与应用程序B(第二控件标识)紧邻的那个控件(图6中虚线框所围绕的控件)。因此，可以通过第二控件标识与用户想实际触控的控件之间的关联性，来对多个与所述第一控件标识对应的控件进行筛选，以筛选出用户想实际触控的控件作为目标控件。其中，控件之间的关联性可以包括控件之间的距离或者控件之间的包含关系等。
再者，在目标界面中一些控件可能会有些相似的控件。该相似可以为显示样式比较相似，或者也可以是显示位置比较相似，再者，也可以是与其他控件的包含关系比较相似。在这种情况下，可以结合与第二控件标识所对应的控件相似的控件共同来从多个与所述第一控件标识对应的控件中筛选出用户想实际触控的控件作为目标控件。
作为另外一种方式，所述若所述第二控件标识对应的控件为一个，则基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，包括：若所述第二控件标识对应的控件为一个，且所述第二控件标识所对应的控件对应有相似控件，获取与所述第二控件标识所对应的控件相似的控件作为第一相似控件；基于所述第二控件标识所对应的控件以及所述第一相似控件，从多个与所述第一控件标识对应的控件中确定目标控件。其中，第一相似控件可以为一个也可以为多个。
那么作为一种方式，所述若所述第二控件标识对应的控件为一个，则基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，包括：若所述第二控件标识对应的控件为一个，且所述第二控件标识所对应的控件未对应有相似控件，基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件。
其中，可选的，作为一种确定相似控件的方式，可以先基于代码解析的方式对目标界面进行识别，从而获取到目标界面中所包括控件的ID、类型、位置、尺寸、包含关系以及描述信息等，然后基于所识别出的控件的ID、类型、位置、尺寸、包含关系以及描述信息等信息构建控件关系结构图。在该控件关系结构图中，会包括有多个节点，其中每个节点表征一个控件。并且，子节点所对应的控件包含在所述子节点对应的父节点所对应的控件中。需要说明的是，在控件关系结构图中子节点和父节点是相对存在的，若有某个节点对应的控件包括在该节点相邻的一个节点所对应的控件中，那么该节点某个节点则是相对于该相邻的节点为子节点，对应的，该相邻的节点则为该某个节点的父节点。示例性的，对图6所示的界面进行解析可以得到图7所示的控件关系结构图。如图7所示，节点2与节点5相邻，并且按照从上到下，节点对应的控件的层级越低的顺序，节点2的层级高于节点5的层级，并且，节点5相对于节点2为子节点，节点2相对于节点5为父节点。在图6所示控件关系结构图中，排布在最顶端的节点为根节点，其中，在控件关系结构图中根节点则表征的是目标界面中最基础的一个控件，在界面所包括的所有控件中，除了该最基础的一个控件外，其他所有控件均包含在该最基础的一个控件中。并且，排布位置越靠近顶部的节点所表征的控件的层级越接近于该最基础的一个控件。
在这种方式中，获取与所述第二控件标识所对应的控件相似的控件作为第一相似控件，可以包括：基于所述第二控件标识所对应的控件的属性，在控件关系结构图中查找与所述第二控件标识所对应的控件相似的控件作为第一相似控件，所述属性包括到控件对应的节点到根节点的距离、控件的类型、控件的长宽以及控件在对应的父控件中的相对位置中的至少一项。
其中，控件对应的节点到根节点的距离表征的是从该控件对应的节点跳转到根节点的过程中需要跳转的次数。例如，其中，图7中的节点1跳转到根节点需要跳转1次，那么节点1和根节点之间的距离为1。节点8跳转到根节点需要跳转5次，那么节点8到根节点的距离为5。又例如，节点5跳转到根节点需要跳转3次，那么节点5到根节点的距离为3，同理，节点6和节点8到根节点的距离均为3。
其中，控件的类型可以表征控件在界面中的用途。可选的，根据控件的类型可以将界面中所包括的控件分为用于输出内容的控件、用于显示内容的控件以及用于和用户进行交互的控件。其中，用于输出内容的控件可以为文本框。用于显示内容的控件可以为用于显示图片或者文本内容的控件。用于和用户进行交互的控件则可以包括按钮等。
其中，控件的长宽则表征的是控件本身的尺寸。控件在对应的父控件中的相对位置可以理解为控件在被显示在界面中时其所在父控件中的相对显示位置。如图8所示，在控件1中包括有控件11和控件12，其中，控件11用于显示应用程序A对应的图标控件、名称控件以及安装触发控件。其中，控件12用于显示应用程序B对应的图标控件、名称控件以及安装触发控件。如图8可知，应用程序A对应的图标控件显示在控件11中相对位置，与应用程序B对应的图标控件显示在控件12中相对位置是一样的。
在本申请实施例中，在获取到目标界面中所包括的控件的属性后，则可以基于控件的属性筛选出与所述第二控件标识所对应的控件相似的控件作为第一相似控件。并且，在基于属性来获取第一相似控件的过程中，可以基于控件属性中的一项来获取第一相似控件，也可以基于属性中多项来获取第一相似控件。例如，作为一种方式，可以基于属性中所包括的控件对应的节点到根节点的距离来筛选第一相似控件。例如，请再参阅图7，若第二控件标识为应用程序A，那么应用程序A对应的节点为节点5。其中，节点5到根节点的距离为3，而其他到根节点的距离为3的节点至少有节点6和节点7，那么可以确定节点6和节点7各自对应的控件为第一相似控件。
再如图8所示，若是基于控件在对应的父控件中的相对位置来确定相似控件，那么则可以确定图8中的应用程序A对应的图标控件和应用程序B对应的图标控件为相似控件。那么若应用程序A对应的图标控件为第二控件标识对应的控件，那么则可以确定应用程序B对应的图标控件为相似控件。
可选的，如图9所示，所述基于所述第二控件标识所对应的控件以及所述第一相似控件，从多个与所述第一控件标识对应的控件中确定目标控件，包括：
S221：获取控件关系结构图中第一节点分别与多个第二节点之间的距离，得到多个第一距离，所述第一节点用于表征第二控件标识所对应的控件，所述第二节点用于表征第一控件标识对应的控件。
可选的，所述获取控件关系结构图中第一节点分别与多个第二节点之间的距离，得到多个第一距离，包括：
获取控件关系结构图中第一节点与当前进行第一距离计算的第二节点所对应的最近公共父节点；获取所述第一节点到所述最近公共父节点的距离，以及所述当前进行第一距离计算的第二节点到所述最近公共父节点的距离，所述距离保证对应两个节点相互跳跃的层级的数量；将所述第一节点到所述最近公共父节点的距离，与所述当前进行第一距离计算的第二节点到所述最近公共父节点的距离之和作为当前进行第一距离计算的第二节点与所述第一节点之间的距离，以得到多个第一距离。
示例性的，如图7所示，若第一控件标识为安装，第二控件标识为应用程序A，那么第一控件标识对应的节点有节点8、节点9以及节点10。第二控件标识对应的节点有节点5。那么第一节点则包括节点5，第二节点则包括节点8、节点9以及节点10，然后会分别获取节点5和节点8所对应的第一距离，节点5和节点9所对应的第一距离，以及节点5和节点10所对应的第一距离，从而得到多个第一距离。
其中，节点5和节点8对应的公共父节点有节点2、节点1以及根节点，但是节点2是距离节点5和节点8最近的父节点，则节点2为节点5和节点8对应的最近公共父节点。其中，节点5到节点2的距离为1，节点8到节点2的距离为3，那么节点5和节点8所对应的第一距离为4。节点5和节点9对应的公共父节点有节点1以及根节点，但是节点1是距离节点5和节点9最近的父节点，则节点1为节点5和节点9对应的最近公共父节点。其中，节点5到节点1的距离为2，节点9到节点1的距离为4，那么节点5和节点9所对应的第一距离为6。节点5和节点10对应的公共父节点有节点1以及根节点，但是节点1是距离节点5和节点10最近的父节点，则节点1为节点5和节点10对应的最近公共父节点。其中，节点5到节点1的距离为2，节点10到节点1的距离为4，那么节点5和节点10所对应的第一距离为6。
S222：获取控件关系结构图中第三节点分别与多个第二节点之间的距离，得到多个第二距离，所述第三节点为第一相似控件对应的节点。
可选的，所述获取控件关系结构图中第三节点分别与多个第二节点之间的距离，得到多个第二距离，包括：获取控件关系结构图中第三节点与当前进行第二距离计算的第二节点所对应的最近公共父节点；获取所述第三节点到所述最近公共父节点的距离，以及所述当前进行第二距离计算的第二节点到所述最近公共父节点的距离，所述距离保证对应两个节点相互跳跃的层级的数量；将所述第三节点到所述最近公共父节点的距离，与所述当前进行第二距离计算的第二节点到所述最近公共父节点的距离之和作为当前进行第二距离计算的第二节点与所述第三节点之间的距离，以得到多个第二距离。
示例性，请再参阅图7，如前述内容所示，依然以第一节点则包括节点5，第二节点则包括节点8、节点9以及节点10为例，若是通过到根节点的距离是否相同来确定第一相似控件，那么所确定的出的第一相似控件所对应的节点包括有节点6和节点7。然后可以按照前述的访问时计算出节点6和节点8之间的第二距离，节点6和节点9之间的第二距离，节点6和节点10之间的第二距离。再者，会计算出节点7 和节点8之间的第二距离，节点7和节点9之间的第二距离，节点7和节点10之间的第二距离，从而得到多个第二距离。
需要说明的是，本身实施例中计算第二距离的方式和计算第一距离的方式是相同的，此处则不再细述。对应的，所计算出出的节点6和节点8之间的第二距离为6，节点6和节点9之间的第二距离为4，节点6和节点10之间的第二距离为6。所计算出出的节点7和节点8之间的第二距离为6，节点7和节点9之间的第二距离为6，节点7和节点10之间的第二距离为4。
S223：获取多个参考距离，所述多个参考距离包括所述多个第一距离和所述多个第二距离。
S224：检测多个参考距离中的最小值与所述多个第一距离中的最小值是否一致，且该最小值的数量为一个。
S225：若所述多个参考距离中的最小值与所述多个第一距离中的最小值一致，且该最小值的数量为一个，则将所述第一控件标识对应的多个控件中，与所述多个第一距离中最小值对应的控件作为目标控件。
从前述实例可以发现，多个参考距离中最小的值为4，且多个第一距离中的最小值也为4，那么则可以确定多个参考距离中的最小值与多个第一距离中的最小值一致，进而可以将第一控件标识对应的多个控件中，与多个第一距离中最小值对应的控件作为目标控件。例如，与第二控件标识对应的控件包括有图7中的节点8、节点9以及节点10所对应的控件，多个第一距离中的最小值所对应的控件为节点8和节点5所对应的控件，进而节点8所对应的控件为与多个第一距离中的最小值以及与第二控件标识均对应的控件，从而可以将节点8所对应的控件作为目标控件。
S226：若所述多个参考距离中的最小值与所述多个第一距离中的最小值不一致，且所述多个第一距离没有与所述多个参考距离中的最小值相同的第一距离，获取第二相似控件，所述第二相似控件为基于第一控件标识对应的控件的属性从所述控件关系结构图中选择出的控件。
需要说明的是，在一些情况下，因为用户的口误，用户所想要触发的控件与用户通过语音控制指令所表达的控制目标涉及的控件可能会有不同。如图10所示，在图10所示的界面中，应用程序A对应的操作控件为更新，应用程序B对应的操作控件为更新，应用程序C对应的操作控件为安装。其中，对图10中的控件进行识别所得到得到控件关系结构图，可以如图11所示。基于图11所示的控件关系结构图，若用户发出的语音控制指令为“更新应用程序C”，那么所获取到的第一控件标识为更新，第二控件标识为应用程序C。基于前述内容所介绍的获取第一距离、第二距离以及第三距离的方式，可以发现多个参考距离中的最小值与多个第一距离中的最小值不一致，并且会发现多个第一距离中的最小值均大于多个参考距离中的最小值，则会进一步则则会基于第一控件标识对应的控件的属性从图11所示的控件关系结构图中选择的相似的控件作为第二相似控件。例如，若基于控件到根节点(图11中的节点1)的距离相同来选择第二相似控件，进而可以将到根节点的距离也为4的节点11对应的控件作为第二相似控件。
S227：获取第三距离，所述第三距离包括所述第二相似控件对应的节点到所述第二控件标识对应的节点的距离。
依然如图11所示，可以获取第二相似控件对应的节点11到第二控件标识对应的节点7的距离为4，即获取到的第三距离为4。
S228：若所述第三距离中存在与所述多个参考距离中的最小值唯一一致的距离，将所述唯一一致的距离对应的控件作为目标控件。
其中，唯一一致可以理解为一致且仅有一个一致。对应的，第三距离中存在与所述多个参考距离中的最小值唯一一致的距离则可以理解为第三距离中仅有一个距离与多个参考距离中的最小值一致。
在图11所示的情况下中，多个参考距离中的最小值也为4，因此可以确定第三距离中存在与多个参考距离中的最小值唯一一致的距离。因此，可以将该唯一距离所对应的节点(节点11)所对应的控件作为目标控件。从而通过获取第二相似控件的方式，可以使得即使在用户口误而错误的发出语音控制指令的情况下，电子设备可以智能化的对用户的语音控制指令中的错误进行纠正，从而提升了准确执行用户实际意图的概率。
若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有一个，将所述第一控件标识对应的控件作为目标控件。
再者，在一些情况下，多个参考距离中的最小值与多个第一距离中的最小值一致，但是该最小值可能并不是唯一的。如图12所示，经过前述方式进行计算可以发现多个第一距离中的最小值所对应的控件可以为节点8所对应的控件，也可以为节点9所对应的控件，因此，无法直接确定出目标控件。那么在通过本申请实施例提供的方法无法自动确定出目标控件的情况下，可以通过询问用户的方式来确定目标控件。
还有，在一些情况下，与第二控件标识所对应的控件会有两个及以上，那么在这种情况下，若第一控件标识对应的控件也有两个及以上，那么则无法通过第二控件标识来从多个第一控件标识对应的控件中确定目标控件，则可以通过询问用户的方式来确定目标控件。
S230：执行与所述目标控件对应的控制操作。
本实施例提供的一种语音控制方法，从而通过上述方式使得在与语音控制指令对应的待确定控件(第一控件标识对应的控件)有多个而造成无法明确用户的实际控制意图的情况下，可以再借助于第二控件标识对应的控件对多个待确定控件进行确定，从而从多个待确定控件中确定表征用户实际控制目的控件作为目标控件，进而使得电子设备可以准确的确定用户的实际控制意图。并且，在本实施例中，可以基于目标界面中的控件的相互包含关系建立控件关系结构图，从而使得可以借助于该控件关系结构图来计算第二控件标识所对应的控件以及所述第一相似控件分别与所述第一控件标识对应的控件之间的距离，继而在基于该距离来从多个与所述第一控件标识对应的控件中确定目标控件，从而使得电子设备可以更为便利且准确的确定出目标控件。
请参阅图13，本申请提供的一种语音控制方法，所述方法包括：
S310：从获取的语音控制指令中获取第一控件标识和第二控件标识，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件。
S320：若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，获取在所述目标界面中，多个与所述第一控件标识对应的控件各自与所述第二控件标识对应控件之间的显示距离，所述目标界面为获取到所述语音控制指令时所显示的界面。
其中，在本申请实施例中，显示距离表征的是在目标界面中控件之间的像素距离。其中，两个控件之间显示距离可以包括两个控件在的中心坐标之间的距离。
如图14所示，控件20和控件21之间的显示距离为d1，控件21和控件22之间的显示距离为d2。
S330：将所述第一控件标识对应的控件中对应的显示距离最小的控件作为目标控件。
S340：执行与所述目标控件对应的控制操作。
本实施例提供的一种语音控制方法，从而通过上述方式使得在与语音控制指令对应的待确定控件(第一控件标识对应的控件)有多个而造成无法明确用户的实际控制意图的情况下，可以再借助于第二控件标识对应的控件对多个待确定控件进行确定，从而从多个待确定控件中确定表征用户实际控制目的控件作为目标控件，进而使得电子设备可以准确的确定用户的实际控制意图。并且，在本实施例中，可以直接基于多个与所述第一控件标识对应的控件各自与所述第二控件标识对应控件之间的显示距离来从多个与所述第一控件标识对应的控件中确定目标控件，提升了获取目标控件的灵活性。
请参阅图15，本申请提供的一种语音控制装置400，所述装置400包括：
标识获取单元410，用于从获取的语音控制指令中获取第一控件标识和第二控件标识，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件。
控件确定单元420，用于若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，所述目标界面为获取到所述语音控制指令时所显示的界面。
控制单元430，用于执行与所述目标控件对应的控制操作。
作为一种方式，控件确定单元420，具体用于若所述第二控件标识对应的控件为一个，则基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件。可选的，控件确定单元420，具体用于若所述第二控件标识对应的控件为一个，且所述第二控件标识所对应的控件未对应有相似控件，基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件。
可选的，控件确定单元420，具体用于若所述第二控件标识对应的控件为一个，且所述第二控件标识所对应的控件对应有相似控件，获取与所述第二控件标识所对应的控件相似的控件作为第一相似控件；基于所述第二控件标识所对应的控件以及所述第一相似控件，从多个与所述第一控件标识对应的控件中确定目标控件。
可选的，基于所述第二控件标识所对应的控件的属性，在控件关系结构图中查找与所述第二控件标识所对应的控件相似的控件作为第一相似控件，所述属性包括到控件对应的节点到根节点的距离、控件的类型、控件的长宽以及控件在对应的父控件中的相对位置中的至少一项；其中，所述控件关系结构图为基于所述目标界面中的控件的包含关系所生成，在所述控件关系结构图中子节点所对应的控件包含在所述子节点对应的父节点所对应的控件中。
可选的，控件确定单元420，具体用于获取控件关系结构图中第一节点分别与多个第二节点之间的距离，得到多个第一距离，所述第一节点用于表征第二控件标识所对应的控件，所述第二节点用于表征第一控件标识对应的控件；获取控件关系结构图中第三节点分别与多个第二节点之间的距离，得到多个第二距 离，所述第三节点为第一相似控件对应的节点；获取多个参考距离，所述多个参考距离包括所述多个第一距离和所述多个第二距离；若所述多个参考距离中的最小值与所述多个第一距离中的最小值一致，且该最小值的数量为一个，则将所述第一控件标识对应的多个控件中，与所述多个第一距离中最小值对应的控件作为目标控件。
还具体用于若所述多个参考距离中的最小值与所述多个第一距离中的最小值不一致，且所述多个第一距离没有与所述多个参考距离中的最小值相同的第一距离，获取第二相似控件，所述第二相似控件为基于第一控件标识对应的控件的属性从所述控件关系结构图中选择出的控件；获取第三距离，所述第三距离包括所述第二相似控件对应的节点到所述第二控件标识对应的节点的距离；若所述第三距离中存在与所述多个参考距离中的最小值唯一一致的距离，将所述唯一一致的距离对应的控件作为目标控件。
作为一种方式，控件确定单元420，具体用于获取控件关系结构图中第一节点与当前进行第一距离计算的第二节点所对应的最近公共父节点；获取所述第一节点到所述最近公共父节点的距离，以及所述当前进行第一距离计算的第二节点到所述最近公共父节点的距离，所述距离保证对应两个节点相互跳跃的层级的数量；将所述第一节点到所述最近公共父节点的距离，与所述当前进行第一距离计算的第二节点到所述最近公共父节点的距离之和作为当前进行第一距离计算的第二节点与所述第一节点之间的距离，以得到多个第一距离。
作为一种方式，控件确定单元420，具体用于获取控件关系结构图中第三节点与当前进行第二距离计算的第二节点所对应的最近公共父节点；获取所述第三节点到所述最近公共父节点的距离，以及所述当前进行第二距离计算的第二节点到所述最近公共父节点的距离，所述距离保证对应两个节点相互跳跃的层级的数量；将所述第三节点到所述最近公共父节点的距离，与所述当前进行第二距离计算的第二节点到所述最近公共父节点的距离之和作为当前进行第二距离计算的第二节点与所述第三节点之间的距离，以得到多个第二距离。
其中，控件确定单元420，具体用于若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有一个，将所述第一控件标识对应的控件作为目标控件。
作为另外一种方式，控件确定单元420，具体用于获取在所述目标界面中，多个与所述第一控件标识对应的控件各自与所述第二控件标识对应控件之间的显示距离；将所述第一控件标识对应的控件中对应的显示距离最小的控件作为目标控件。
本实施例提供的一种语音控制装置，先从获取的语音控制指令中获取第一控件标识和第二控件标识，在第一控件标识对应的控件为与语音控制指令对应的待确定控件，第二控件标识对应的控件用于从待确定控件中确定表征用户实际控制目标的控件作为目标控件的情况下，若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件有多个，则会基于第二控件标识从多个与所述第一控件标识对应的控件中确定目标控件，并执行与所述目标控件对应的控制操作。从而通过上述方式使得在与语音控制指令对应的待确定控件(第一控件标识对应的控件)有多个而造成无法明确用户的实际控制意图的情况下，可以再借助于第二控件标识对应的控件对多个待确定控件进行确定，从而从多个待确定控件中确定表征用户实际控制目的控件作为目标控件，进而使得电子设备可以准确的确定用户的实际控制意图。
需要说明的是，所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。在本申请所提供的几个实施例中，模块相互之间的耦合可以是电性。另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
下面将结合图16对本申请提供的一种电子设备进行说明。
请参阅图16，基于上述的语音控制方法、装置，本申请实施例还提供的一种可以执行前述语音控制方法的电子设备1000。电子设备1000包括相互耦合的一个或多个(图中仅示出一个)处理器102、存储器104、摄像头106以及音频采集装置108。其中，该存储器104中存储有可以执行前述实施例中内容的程序，而处理器102可以执行该存储器104中存储的程序。
其中，处理器102可以包括一个或者多个处理核。处理器102利用各种接口和线路连接整个电子设备1000内的各个部分，通过运行或执行存储在存储器104内的指令、程序、代码集或指令集，以及调用存储在存储器104内的数据，执行电子设备1000的各种功能和处理数据。可选地，处理器102可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器102可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器 等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器102中，单独通过一块通信芯片进行实现。作为一种方式，处理器102可以为神经网络芯片。例如，可以为嵌入式神经网络芯片(NPU)。
存储器104可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器104可用于存储指令、程序、代码、代码集或指令集。例如，存储器104中可以存储有装置。该装置可以为前述的装置400。存储器104可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。
再者，电子设备1000除了前述所示的器件外，还可以包括网络模块110以及传感器模块112。
所述网络模块110用于实现电子设备1000与其他设备之间的信息交互，例如，传输设备控制指令、操纵请求指令以及状态信息获取指令等。而当电子设备200具体为不同的设备时，其对应的网络模块110可能会有不同。
传感器模块112可以包括至少一种传感器。具体地，传感器模块112可包括但并不限于：水平仪、光传感器、运动传感器、压力传感器、红外热传感器、距离传感器、加速度传感器、以及其他传感器。
其中，压力传感器可以检测由按压在电子设备1000产生的压力的传感器。即，压力传感器检测由用户和电子设备之间的接触或按压产生的压力，例如由用户的耳朵与移动终端之间的接触或按压产生的压力。因此，压力传感器可以用来确定在用户与电子设备1000之间是否发生了接触或者按压，以及压力的大小。
其中，加速度传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向，可用于识别电子设备1000姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等。另外，电子设备1000还可配置陀螺仪、气压计、湿度计、温度计等其他传感器，在此不再赘述。
音频采集装置110，用于进行音频信号采集。可选的，音频采集装置110包括有多个音频采集器件，该音频采集器件可以为麦克风。
作为一种方式，电子设备1000的网络模块为射频模块，该射频模块用于接收以及发送电磁波，实现电磁波与电信号的相互转换，从而与通讯网络或者其他设备进行通讯。所述射频模块可包括各种现有的用于执行这些功能的电路元件，例如，天线、射频收发器、数字信号处理器、加密/解密芯片、用户身份模块(SIM)卡、存储器等等。例如，该射频模块可以通过发送或者接收的电磁波与外部设备进行交互。例如，射频模块可以向目标设备发送指令。
请参考图17，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读介质800中存储有程序代码，所述程序代码可被处理器调用执行上述方法实施例中所描述的方法。
计算机可读存储介质800可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质800包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质800具有执行上述方法中的任何方法步骤的程序代码810的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码810可以例如以适当形式进行压缩。
综上所述，本申请提供的一种语音控制方法、装置、电子设备及存储介质，先从获取的语音控制指令中获取第一控件标识和第二控件标识，在第一控件标识对应的控件为与语音控制指令对应的待确定控件，第二控件标识对应的控件用于从待确定控件中确定表征用户实际控制目标的控件作为目标控件的情况下，若目标界面中包括有与第一控件标识对应的控件以及与第二控件标识对应的控件，且与第一控件标识对应的控件有多个，则会基于第二控件标识从多个与所述第一控件标识对应的控件中确定目标控件，并执行与所述目标控件对应的控制操作。从而通过上述方式使得在与语音控制指令对应的待确定控件(第一控件标识对应的控件)有多个而造成无法明确用户的实际控制意图的情况下，可以再借助于第二控件标识对应的控件对多个待确定控件进行确定，从而从多个待确定控件中确定表征用户实际控制目的控件作为目标控件，进而使得电子设备可以准确的确定用户的实际控制意图。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (20)

  1. 一种语音控制方法，其特征在于，所述方法包括：
    从获取的语音控制指令中获取第一控件标识和第二控件标识；
    若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，所述目标界面为获取到所述语音控制指令时所显示的界面，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件；
    执行与所述目标控件对应的控制操作。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，包括：
    若所述第二控件标识对应的控件为一个，则基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件。
  3. 根据权利要求2所述的方法，其特征在于，所述若所述第二控件标识对应的控件为一个，则基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，包括：
    若所述第二控件标识对应的控件为一个，且所述第二控件标识所对应的控件未对应有相似控件，基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件。
  4. 根据权利要求2所述的方法，其特征在于，所述若所述第二控件标识对应的控件为一个，则基于所述第二控件标识所对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，包括：
    若所述第二控件标识对应的控件为一个，且所述第二控件标识所对应的控件对应有相似控件，获取与所述第二控件标识所对应的控件相似的控件作为第一相似控件；
    基于所述第二控件标识所对应的控件以及所述第一相似控件，从多个与所述第一控件标识对应的控件中确定目标控件。
  5. 根据权利要求4所述的方法，其特征在于，所述获取与所述第二控件标识所对应的控件相似的控件作为第一相似控件，包括：
    基于所述第二控件标识所对应的控件的属性，在控件关系结构图中查找与所述第二控件标识所对应的控件相似的控件作为第一相似控件，所述属性包括控件对应的节点到根节点的距离、控件的类型、控件的长宽以及控件在对应的父控件中的相对位置中的至少一项；
    其中，所述控件关系结构图为基于所述目标界面中控件的包含关系所生成，在所述控件关系结构图中子节点所对应的控件包含在所述子节点对应的父节点所对应的控件中。
  6. 根据权利要求5所述的方法，其特征在于，所述基于所述第二控件标识所对应的控件以及所述第一相似控件，从多个与所述第一控件标识对应的控件中确定目标控件，包括：
    获取控件关系结构图中第一节点分别与多个第二节点之间的距离，得到多个第一距离，所述第一节点用于表征第二控件标识所对应的控件，所述第二节点用于表征第一控件标识对应的控件；
    获取控件关系结构图中第三节点分别与多个第二节点之间的距离，得到多个第二距离，所述第三节点为第一相似控件对应的节点；
    获取多个参考距离，所述多个参考距离包括所述多个第一距离和所述多个第二距离；
    若所述多个参考距离中的最小值与所述多个第一距离中的最小值一致，且该最小值的数量为一个，则将所述第一控件标识对应的多个控件中，与所述多个第一距离中最小值对应的控件作为目标控件。
  7. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    若所述多个参考距离中的最小值与所述多个第一距离中的最小值不一致，且所述多个第一距离没有与所述多个参考距离中的最小值相同的第一距离，获取第二相似控件，所述第二相似控件为基于第一控件标识对应的控件的属性从所述控件关系结构图中选择出的控件；
    获取第三距离，所述第三距离包括所述第二相似控件对应的节点到所述第二控件标识对应的节点的距离；
    若所述第三距离中存在与所述多个参考距离中的最小值唯一一致的距离，将所述唯一一致的距离对应的控件作为目标控件。
  8. 根据权利要求6所述的方法，其特征在于，所述获取控件关系结构图中第一节点分别与多个第二节点之间的距离，得到多个第一距离，包括：
    获取控件关系结构图中第一节点与当前进行第一距离计算的第二节点所对应的最近公共父节点；
    获取所述第一节点到所述最近公共父节点的距离，以及所述当前进行第一距离计算的第二节点到所述最近公共父节点的距离；
    将所述第一节点到所述最近公共父节点的距离，与所述当前进行第一距离计算的第二节点到所述最近公共父节点的距离之和作为当前进行第一距离计算的第二节点与所述第一节点之间的距离，以得到多个第一距离。
  9. 根据权利要求6所述的方法，其特征在于，所述获取控件关系结构图中第三节点分别与多个第二节点之间的距离，得到多个第二距离，包括：
    获取控件关系结构图中第三节点与当前进行第二距离计算的第二节点所对应的最近公共父节点；
    获取所述第三节点到所述最近公共父节点的距离，以及所述当前进行第二距离计算的第二节点到所述最近公共父节点的距离，所述距离表征对应两个节点相互跳跃的层级的数量；
    将所述第三节点到所述最近公共父节点的距离，与所述当前进行第二距离计算的第二节点到所述最近公共父节点的距离之和作为当前进行第二距离计算的第二节点与所述第三节点之间的距离，以得到多个第二距离。
  10. 根据权利要求1所述的方法，其特征在于，所述基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，包括：
    获取在所述目标界面中，多个与所述第一控件标识对应的控件各自与所述第二控件标识对应控件之间的显示距离；
    将所述第一控件标识对应的控件中对应的显示距离最小的控件作为目标控件。
  11. 根据权利要求1-10任一所述的方法，其特征在于，所述方法还包括：
    若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有一个，将所述第一控件标识对应的控件作为目标控件。
  12. 根据权利要求1-11任一所述的方法，其特征在于，所述方法还包括：
    若与所述第二控件标识所对应的控件有两个及以上，且所述第一控件标识对应的控件有两个及以上，则通过询问用户确定目标控件。
  13. 根据权利要求1所述的方法，其特征在于，所述从获取的语音控制指令中获取第一控件标识和第二控件标识之前还包括：
    若获取到指定语音内容，则开始获取语音控制指令。
  14. 根据权利要求13所述的方法，其特征在于，所述指定语音内容由用户进行配置。
  15. 根据权利要求1-14任一所述的方法，其特征在于，所述从获取的语音控制指令中获取第一控件标识和第二控件标识，包括：
    将获取的语音控制指令转换为对应的文本内容；
    基于语义提取规则从所述文本内容中进行标识获取，以获取第一控件标识和第二控件标识。
  16. 根据权利要求1-15任一所述的方法，其特征在于，所述方法还包括：
    在接收到所述语音控制指令后，同步开始对目标界面进行识别以获取到所述目标界面中所包括的控件。
  17. 根据权利要求16所述的方法，其特征在于，对目标界面进行识别的方式包括：
    通过代码解析方式对所述目标界面进行识别；
    通过图文识别的方式对所述目标界面进行识别；或者
    通过图标识别的方式对所述目标界面进行识别。
  18. 一种语音控制装置，其特征在于，所述装置包括：
    标识获取单元，用于从获取的语音控制指令中获取第一控件标识和第二控件标识；
    控件确定单元，用于若目标界面中包括有与所述第一控件标识对应的控件以及与第二控件标识对应的控件，且与所述第一控件标识对应的控件有多个，基于所述第二控件标识对应的控件从多个与所述第一控件标识对应的控件中确定目标控件，所述目标界面为获取到所述语音控制指令时所显示的界面，其中，所述第一控件标识对应的控件为与所述语音控制指令对应的待确定控件，所述第二控件标识对应的控件用于从所述待确定控件中确定表征用户实际控制目标的控件作为所述目标控件；
    控制单元，用于执行与所述目标控件对应的控制操作。
  19. 一种电子设备，其特征在于，包括一个或多个处理器以及存储器；
    一个或多个程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或 多个程序配置用于执行权利要求1-17任一所述的方法。
  20. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有程序代码，其中，在所述程序代码运行时执行权利要求1-17任一所述的方法。
PCT/CN2022/136341 2021-12-09 2022-12-02 语音控制方法、装置、电子设备及存储介质 WO2023103917A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111500093.0A CN114121012A (zh) 2021-12-09 2021-12-09 语音控制方法、装置、电子设备及存储介质
CN202111500093.0 2021-12-09

Publications (1)

Publication Number Publication Date
WO2023103917A1 true WO2023103917A1 (zh) 2023-06-15

Family

ID=80364063

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/136341 WO2023103917A1 (zh) 2021-12-09 2022-12-02 语音控制方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN114121012A (zh)
WO (1) WO2023103917A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114121012A (zh) * 2021-12-09 2022-03-01 杭州逗酷软件科技有限公司 语音控制方法、装置、电子设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010057434A1 (zh) * 2008-11-20 2010-05-27 腾讯科技（深圳）有限公司 一种生成控件对象库的方法和装置
US20140181865A1 (en) * 2012-12-25 2014-06-26 Panasonic Corporation Speech recognition apparatus, speech recognition method, and television set
CN108538291A (zh) * 2018-04-11 2018-09-14 百度在线网络技术（北京）有限公司 语音控制方法、终端设备、云端服务器及系统
CN109582311A (zh) * 2018-11-30 2019-04-05 网易（杭州）网络有限公司 一种游戏中ui编辑的方法及装置、电子设备、存储介质
CN110136718A (zh) * 2019-05-31 2019-08-16 深圳市语芯维电子有限公司 语音控制的方法和装置
CN113476848A (zh) * 2021-07-08 2021-10-08 网易（杭州）网络有限公司 树状链式地图的生成方法及装置、存储介质、电子设备
CN114121012A (zh) * 2021-12-09 2022-03-01 杭州逗酷软件科技有限公司 语音控制方法、装置、电子设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010057434A1 (zh) * 2008-11-20 2010-05-27 腾讯科技（深圳）有限公司 一种生成控件对象库的方法和装置
US20140181865A1 (en) * 2012-12-25 2014-06-26 Panasonic Corporation Speech recognition apparatus, speech recognition method, and television set
CN108538291A (zh) * 2018-04-11 2018-09-14 百度在线网络技术（北京）有限公司 语音控制方法、终端设备、云端服务器及系统
CN109582311A (zh) * 2018-11-30 2019-04-05 网易（杭州）网络有限公司 一种游戏中ui编辑的方法及装置、电子设备、存储介质
CN110136718A (zh) * 2019-05-31 2019-08-16 深圳市语芯维电子有限公司 语音控制的方法和装置
CN113476848A (zh) * 2021-07-08 2021-10-08 网易（杭州）网络有限公司 树状链式地图的生成方法及装置、存储介质、电子设备
CN114121012A (zh) * 2021-12-09 2022-03-01 杭州逗酷软件科技有限公司 语音控制方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN114121012A (zh) 2022-03-01

Similar Documents

Publication Publication Date Title
US10820295B2 (en) Method, terminal device and computer-readable storage medium for wireless connection
CN112543910A (zh) 用于确认用户的意图的电子装置的反馈方法和设备
EP3633947B1 (en) Electronic device and control method therefor
WO2023082703A1 (zh) 语音控制方法、装置、电子设备及可读存储介质
CN110556127B (zh) 语音识别结果的检测方法、装置、设备及介质
CN109947650B (zh) 脚本步骤处理方法、装置和系统
CN110457214B (zh) 应用测试方法及装置、电子设备
JP7252327B2 (ja) 人間とコンピュータとの相互作用方法および電子デバイス
CN111177180A (zh) 一种数据查询方法、装置以及电子设备
US20220020358A1 (en) Electronic device for processing user utterance and operation method therefor
WO2023103917A1 (zh) 语音控制方法、装置、电子设备及存储介质
WO2023077878A1 (zh) 语音控制方法、装置、电子设备以及可读存储介质
CN114333774B (zh) 语音识别方法、装置、计算机设备及存储介质
CN111835621A (zh) 会话消息处理方法、装置、计算机设备及可读存储介质
CN109947988B (zh) 一种信息处理方法、装置、终端设备及服务器
US20210405767A1 (en) Input Method Candidate Content Recommendation Method and Electronic Device
CN109063076B (zh) 一种图片生成方法及移动终端
WO2023103918A1 (zh) 语音控制方法、装置、电子设备及存储介质
JP7236551B2 (ja) キャラクタ推薦方法、キャラクタ推薦装置、コンピュータ装置およびプログラム
WO2023093280A1 (zh) 语音控制方法、装置、电子设备及存储介质
CN112165627A (zh) 信息处理方法、装置、存储介质、终端及系统
CN113742460A (zh) 生成虚拟角色的方法及装置
JP2017211430A (ja) 情報処理装置および情報処理方法
CN112219235A (zh) 包括处理用户语音的电子设备和控制电子设备上语音识别的方法的系统
CN114970562A (zh) 语义理解方法、装置、介质及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22903339

Country of ref document: EP

Kind code of ref document: A1