WO2023040692A1 - Speech control method, apparatus and device, and medium - Google Patents

Abstract

A speech control method, apparatus and device, and a medium. The method comprises: displaying a target interaction interface (S110); loading a target control instruction set corresponding to the target interaction interface, wherein the target control instruction set comprises control instructions that are generated according to interface control data of the target interaction interface (S120); when user control speech is received, querying the target control instruction set for a target control instruction that matches the user control speech (S130); and if the target control instruction is found, executing a target control operation corresponding to the target control instruction (S140).

Description

Voice control method, device, equipment and medium

Cross References to Related Applications

This application is based on a Chinese patent application with application number 202111084298.5 and a filing date of September 14, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated by reference into this application.

technical field

The present disclosure relates to the technical field of voice recognition, and in particular, to a voice control method, device, equipment and medium.

Background technique

With the continuous advancement of technology, more and more application scenarios have introduced electronic devices with voice control functions, such as introducing a vehicle-mounted terminal with voice control functions in vehicles.

Generally, some control instructions corresponding to each interactive interface are pre-stored in the electronic device, and the user can control the operation of each interactive interface of the electronic device by uttering these control instructions. However, due to the limited number of pre-stored control instructions, the user cannot fully realize all voice control of each interactive interface based on these control instructions.

Contents of the invention

In order to solve the above technical problem or at least partly solve the above technical problem, the present disclosure provides a voice control method, device, equipment and medium.

In a first aspect, the present disclosure provides a voice control method, including:

Display the target interface;

Loading the target control instruction set corresponding to the target interactive interface, the target control instruction set includes control instructions generated according to the interface control data of the target interactive interface;

In response to receiving the user control voice, querying the target control command set for the target control command matching the user control voice;

In response to querying the target control instruction, the target control operation corresponding to the target control instruction is executed.

In a second aspect, the present disclosure provides a voice control device, including:

an interface display module configured to display a target interactive interface;

The instruction loading module is configured to load the target control instruction set corresponding to the target interactive interface, and the target control instruction set includes control instructions generated according to the interface control data of the target interactive interface;

The instruction matching module is configured to, in response to receiving the user control speech, query the target control instruction set that matches the user control speech;

The instruction execution module is configured to execute the target control operation corresponding to the target control instruction in response to querying the target control instruction.

In a third aspect, the present disclosure provides a voice control device, comprising:

processor;

memory for storing executable instructions;

Wherein, the processor is used to read executable instructions from the memory, and execute the executable instructions to implement the voice control method described in the first aspect.

In a fourth aspect, the present disclosure provides a computer-readable storage medium, the storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the voice control method described in the first aspect.

In a fifth aspect, the present disclosure provides a computer program product, including a computer program or an instruction. When the computer program or instruction is executed by a processor, the voice control method described in the first aspect is implemented. The voice control method, device, device, and medium of the embodiments of the present disclosure can load the target control instruction set corresponding to the target interactive interface after displaying the target interactive interface, and then when receiving the user's control voice, the target control instruction set Query the target control instruction that matches the received user control voice, and execute the queried target control instruction, so as to realize the user's voice control of the target interactive interface. Since the loaded target control instruction set includes the interface according to the target interactive interface The control command generated by the control data, the interface control data can cover all the interface controls in the target interactive interface, therefore, can fully realize all the voice control of the target interactive interface, and then achieve the visibility of the target interactive interface (Display Can be Said, DCS) effect.

Description of drawings

The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

FIG. 1 is a schematic flowchart of a voice control method provided by an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a main interface of a vehicle-mounted terminal provided by an embodiment of the present disclosure;

FIG. 3A is a schematic diagram of an application interface of a vehicle-mounted terminal provided by an embodiment of the present disclosure;

FIG. 3B is a schematic diagram of an application interface of another vehicle-mounted terminal provided by an embodiment of the present disclosure;

FIG. 4 is a schematic flowchart of a processing process of dynamic control data provided by an embodiment of the present disclosure;

FIG. 5 is a schematic flowchart of another voice control method provided by an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a voice control device provided by an embodiment of the present disclosure;

Fig. 7 is a schematic structural diagram of a voice control device provided by an embodiment of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein; A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the protection scope of the present disclosure.

It should be understood that the various steps described in the method implementations of the present disclosure may be executed in different orders, and/or executed in parallel. Additionally, method embodiments may include additional steps and/or omit performing illustrated steps. The scope of the present disclosure is not limited in this regard.

As used herein, the term "comprise" and its variations are open-ended, ie "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the sequence of functions performed by these devices, modules or units or interdependence.

It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more" multiple".

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only, and are not used to limit the scope of these messages or information.

As technology continues to advance, more and more application scenarios have introduced electronic devices with voice control functions.

Generally, some control instructions corresponding to each interactive interface are pre-stored in the electronic device, and the user can control the operation of each interactive interface of the electronic device by uttering these control instructions.

For example, in wake-up based voice control, the control command may be a wake-up word, and each interactive interface may be registered with a fixed number of wake-up words, and the user may satisfy voice control requirements for each interactive interface by speaking these wake-up words.

However, the control instructions need to be set in advance, which means that the wake-up words on each interactive interface need to be designed in advance. If the content on the interactive interface is dynamically loaded, due to the limited number of pre-stored control instructions, users will not be able to control based on these Instructions fully realize all voice control of each interactive interface.

In addition, the wake-up models based on wake-up words are often small, and it is impossible to support multiple complex and large numbers of wake-up words in one scene. If too many wake-up words are registered on an interactive interface, it will also cause errors in the wake-up model. Wake up problem.

In order to solve the above problems, the embodiments of the present disclosure provide a voice control method, device, device, and medium capable of realizing "see-to-talk".

The voice control method provided by the embodiment of the present disclosure will first be described below with reference to FIG. 1 to FIG. 5 .

In the embodiment of the present disclosure, the voice control method may be executed by an electronic device. Wherein, the electronic device may include a mobile phone, a tablet computer, a desktop computer, a notebook computer, a vehicle terminal, a wearable electronic device, a smart home device, and other devices with a voice control function.

Fig. 1 shows a schematic flowchart of a voice control method provided by an embodiment of the present disclosure.

As shown in FIG. 1, the voice control method may include the following steps S110 to S140.

S110, displaying a target interaction interface.

In the embodiment of the present disclosure, the target interaction interface may be an interface visually displayed through a display screen of the electronic device.

In some embodiments, at least one interface control may be displayed in the target interaction interface. The interface controls may be controls that can be manipulated by the user, such as buttons, options, icons, or links in the interface.

In some embodiments, the target interaction interface may include a main interface displayed when the electronic device is turned on or in a standby state.

In some embodiments, the interface control displayed on the target interaction interface may be an icon of an application program.

Fig. 2 shows a schematic diagram of a main interface of a vehicle-mounted terminal provided by an embodiment of the present disclosure.

As shown in FIG. 2, the vehicle-mounted terminal can display a main interface 201, and multiple interface controls can be displayed in the main interface 201, such as a "setting application" icon 202, a "file application" icon 203, and a "browser application" icon. 204 and the “Music Application” icon 205 .

In some other embodiments, the target interaction interface may include an application interface of any application program installed in the electronic device.

In some embodiments, the interface controls displayed in the target interaction interface may be buttons, options, icons or links in the application interface.

Fig. 3A shows a schematic diagram of an application interface of a vehicle-mounted terminal provided by an embodiment of the present disclosure.

As shown in FIG. 3A, the vehicle-mounted terminal may display a main application interface 301 for setting an application program, and multiple interface controls may be displayed on the application interface 301, such as a "My Device" button 302, a "Control Center" button 303, and a " More Settings" button 304 .

Fig. 3B shows a schematic diagram of another application interface of a vehicle-mounted terminal provided by an embodiment of the present disclosure.

As shown in Figure 3B, the vehicle-mounted terminal can display a main application interface 305 of a music application program, and multiple interface controls can be displayed in the application interface 305, such as "daily recommendation" option 306, "song list" option 307, "Local Music" option 308, "Settings" option 309, various playback control buttons 310, various playlist links 311 and a "Back" button 312.

S120. Load the target control instruction set corresponding to the target interactive interface. The target control instruction set includes control instructions generated according to the interface control data of the target interactive interface.

In the embodiment of the present disclosure, after the electronic device displays the target interaction interface, the target control instruction set corresponding to the target interaction interface can be loaded, and each control command in the target control instruction set is generated according to the interface control data of the target interaction interface , the interface control data may include control data corresponding to all interface controls, that is, all interface controls in the target interactive interface have corresponding control instructions.

In some embodiments, the target control instruction set may be a set of control instructions corresponding to various interface controls in the target interactive interface.

In some embodiments, the control instruction may include a first control instruction generated according to static control data in the interface control data. The first control instruction is a control instruction of the static control.

The static control data may be control data corresponding to static controls in the target interactive interface. The static control may be an interface control that is always fixed and displayed, that is, the static control will not change according to user preferences or settings.

Taking the target interactive interface as the main interface as an example, the static controls can be interface controls that come with the device when it leaves the factory, and cannot be dynamically updated or changed by the user.

Continuing to refer to FIG. 2 , since the setting application, file application and browser application are built-in application programs in the main interface 201 when the device leaves the factory, the corresponding "setting application" icon 202, "file application" icon 203 and The "browser application" icon 204 belongs to the static control of the main interface 201, and the control instruction generated based on the control data corresponding to the "setting application" icon 202, the "file application" icon 203 and the "browser application" icon 204 is the main interface 201 corresponding to the first control command.

Taking the target interactive interface as the application interface as an example, static controls can be interface controls that are fixedly displayed within the interface frame and will not change according to user preferences, such as built-in resources preset in the interface project, and these built-in resources can be displayed in the application interface The interface content is perceived in advance before being pushed to the user.

Continuing to refer to FIG. 3A, since my device, control center and more settings are fixed setting functions in the application main interface 301 of the setting application program, the corresponding "my device" button 302 and "control center" of these setting functions The button 303 and the "more settings" button 304 belong to the static controls of the application main interface 301, and the control instructions are generated based on the control data corresponding to the "my device" button 302, the "control center" button 303 and the "more settings" button 304 It is the first control instruction corresponding to the application main interface 301 .

Continue referring to FIG. 3B , since daily recommendation, ranking, local music, and settings are fixed module functions in the application main interface 305 of the music application program, and return and various playback controls are fixed in the application main interface 301 of the music application program. Interface function, therefore, the "daily recommendation" option 306, "song list" option 307, "local music" option 308, "settings" option 309, various playback control buttons 310 and "return" button 312 corresponding to these functions belong to The static controls of the application main interface 305 are based on the "daily recommendation" option 306, the "ranking" option 307, the "local music" option 308, the "settings" option 309, various playback control buttons 310 and the "return" button 312 corresponding The control instruction generated by the control data is the first control instruction corresponding to the main application interface 305 .

In some other embodiments, the control instruction may further include a second control instruction generated according to the dynamic control data in the interface control data. The second control instruction is a control instruction of the dynamic control.

Wherein, the dynamic control data may be control data corresponding to the dynamic controls in the target interactive interface. The dynamic control may be an interface control that can be dynamically updated or changed with user preferences or settings.

Taking the target interactive interface as the main interface as an example, the dynamic control may be an interface control added by the user.

Continuing to refer to FIG. 2, since the music application is an application program that the user can independently download, the "music application" icon 205 corresponding to the music application belongs to the dynamic control of the main interface 201, and is generated based on the control data corresponding to the "music application" icon 205. The control instruction is the second control instruction corresponding to the main interface 201 .

Taking the target interactive interface as an example of an application interface, the dynamic control may be an interface control that will be dynamically updated within the interface framework, such as a resource that is filled after retrieval based on a network source.

Continuing to refer to FIG. 3B, since the titles of each playlist displayed in the tab of the "songlist" option 307 can be dynamically updated, therefore, the playlist links 311 corresponding to these playlist names belong to the dynamic control of the application main interface 305, based on each The control instruction generated by the control data corresponding to the song list link 311 is the second control instruction corresponding to the application main interface 305 .

It should be noted that, in the case that the target interaction interface is an application interface, the dynamic control may also be an interface control within the interface frame that changes according to the user's preference, and details are not described here.

In the embodiment of the present disclosure, the target control instruction set corresponding to the loaded target interaction interface can be used to control static controls in the target interaction interface, and can also be used to control dynamic controls in the target interaction interface. Thus, the electronic device can load the target control instruction set corresponding to the target interaction interface, so that each interface control in the target interaction interface has a corresponding control instruction, so that the user can perform voice control on the target interaction interface more comprehensively.

S130. In response to receiving the user control voice, query the target control command set for the target control command matching the user control voice.

In the embodiment of the present disclosure, after the electronic device loads the target control instruction set, it can monitor the user control voice, and after receiving the user control voice, search for information related to the user control voice in each control instruction of the target control instruction set. Matching target control instructions.

In some embodiments, S130 may specifically include: converting the user's control voice into the target voice text; querying the target control instruction set for a target control instruction that matches the target voice text.

In some embodiments, the electronic device can input the user's control voice into an automatic speech recognition (Automatic Speech Recognition, ASR) engine set offline to obtain the target voice text output by the ASR engine, and then in each control command of the target control command set Query for target control commands that match the target speech text.

The matching of the target control instruction and the target speech text may be that the target speech text contains any verb in the target control instruction and any control text participle, or that the verb in the target speech text is the same as any verb in the target control instruction and The similarity between nouns in the target speech text and any control text word in the target control instruction is greater than or equal to a preset similarity threshold.

Thus, the electronic device can determine the user's voice control intention by querying the target control command matching the user's control voice in the target control command set.

S140. In response to finding the target control instruction, execute the target control operation corresponding to the target control instruction.

In the embodiment of the present disclosure, if the electronic device determines that it finds a target control instruction that matches the user control voice, it may execute the target control operation corresponding to the target control command; If there is a matching target control instruction, the detection of the user's voice can be continued, waiting for the next user to control the voice.

In some embodiments, S140 may specifically include: performing a target control operation for the target interface control involved in the target control instruction.

Since each control instruction is generated according to the control data of the corresponding interface control, each control instruction can be used to trigger the execution of the target control operation on the target interface control involved in the control instruction, that is, each control instruction can be used to trigger The target interface control to which the control data generating the control instruction belongs performs the target control operation.

Further, the target control operation may be a control operation implemented in a target control manner indicated by the target control instruction.

In some embodiments, after querying the target control instruction, the electronic device may perform a control operation on the target interface control to which the control data generating the control instruction belongs according to the target control mode indicated by the target control instruction.

In the embodiment of the present disclosure, after S140, the electronic device may enter a new interaction interface, or may remain in the target interaction interface.

In some embodiments, when the electronic device remains in the target interaction interface, the electronic device does not need to reload the control instruction set, and can continue to implement the user's voice control of the target interaction interface based on the target control instruction set.

In other embodiments, when the electronic device enters a new interactive interface, the electronic device needs to reload the control instruction set corresponding to the new interactive interface, so as to implement the user's voice on the target interactive interface based on the reloaded control instruction set control.

Continuing to refer to FIG. 2, when the user controls the electronic device to perform the control operation of "opening the music application", the electronic device can enter the application main interface of the music application. Therefore, after jumping from the main interface 201 to the application main interface of the music application After the interface, it is necessary to reacquire the control instruction set corresponding to the application main interface of the music application program, so as to realize the voice control of the user on the application main interface of the music application program based on the reloaded control instruction set.

Continuing to refer to FIG. 3B, when the user controls the electronic device to perform the control operation of "play daily recommendation", the electronic device can directly play the daily recommended song in the application main interface 305 of the music application without jumping to other Therefore, without reloading the control instruction set, the user can continue to implement voice control of the music application main application interface 305 based on the control instruction set corresponding to the music application main application interface 305 .

In the embodiment of the present disclosure, after the target interaction interface is displayed, the target control instruction set corresponding to the target interaction interface can be loaded, and then when the user control voice is received, the target control command set can be queried with the received user control voice match the target control instruction, and execute the queried target control instruction, thereby realizing the voice control of the user on the target interactive interface. Since the loaded target control instruction set includes control instructions generated according to the interface control data of the target interactive interface, the interface control data can cover all interface controls in the target interactive interface, therefore, all voice control of the target interactive interface can be fully realized, Furthermore, the effect of DCS on the target interactive interface is achieved, and the user experience is improved.

In another implementation manner of the present disclosure, when the control instruction includes the first control instruction generated according to the static control data in the interface control data, the electronic device may directly acquire the pre-generated first control instruction.

In some embodiments, S120 may specifically include: determining the target application to which the target interactive interface belongs; querying the control instruction set corresponding to the target application among multiple pre-stored preset control instruction sets; in the control instruction set corresponding to the target application, A first control instruction is extracted.

In the embodiment of the present disclosure, multiple preset control instruction sets may be pre-stored in the electronic device, and each preset control instruction set may correspond to an application program, that is, each preset control instruction set may contain Control instructions for all static controls involved.

Further, the target application may be an application program to which the target interactive interface belongs. The electronic device may use the application program that needs to be run when displaying the target interaction interface as the target application to which the target interaction interface belongs.

In some embodiments, the electronic device may use the application program that needs to be run when displaying the target interactive interface as the target application, then query the control instruction set corresponding to the target application in multiple preset control instruction sets, and finally control the The first control instruction corresponding to the target interactive interface is extracted from the instruction set, and the first control instruction may include control instructions of all static controls involved in the target interactive interface.

In the embodiment of the present disclosure, before S110, the electronic device may first receive the preset control instruction set sent by the server.

In some embodiments, for each application program, the server may receive the control instructions of all static controls corresponding to each interactive interface of the application program and the control mode corresponding to each control instruction input by the developer. The control instruction of each static control includes a verb set and a control text word segmentation set corresponding to the static control. A set of word segmentation extracted from the text, the static control text may be the control name of the static control that can be seen by the user, and the verb set in the control instruction includes multiple verbs with similar semantics.

In other embodiments, for each static control in each interactive interface of each application, the server may extract the control text word segmentation set from the control data of the static control, that is, the static control text in the static control data, and then Multiple control instructions of the static control are obtained by combining different pre-set verb sets and control text segmentation sets, and the verb sets in each control command include multiple verbs with similar semantics. For each control instruction of each static control, the server can also use the control instruction corresponding to the verb set in the control instruction and the control function of the static control corresponding to the control text participle set to determine the control mode corresponding to the control instruction.

Wherein, each participle in the control text participle set can be connected by "|", and each verb in the verb set can also be connected by "|".

Thus, in the embodiment of the present disclosure, the participle set content conforming to the Extended Backus-Naur Form (EBNF) grammatical paradigm can be obtained, so that the first control instruction can be loaded into the language model of the grammar (Grammar) engine .

Continuing to refer to FIG. 2 , taking the control instruction corresponding to the "music application" icon 205 as an example, if a control instruction is used to open a music application, the verb set can be "open|open|enter|enter|click|click", and the control text The participle set may be "music|music application|music|music icon".

In the embodiment of the present disclosure, since the static controls in the target interaction interface may be updated due to reasons such as version upgrades, the preset control instruction set stored in the electronic device also needs to be updated to ensure that the user can control the updated target All static controls in the interactive interface are controlled by voice.

In some embodiments, before extracting the first control instruction from the control instruction set corresponding to the target application, the voice control method may further include: detecting an instruction set version of the control instruction set corresponding to the target application.

In some embodiments, the electronic device may detect the instruction set version of the control instruction set corresponding to the target application, and obtain the version number of the control instruction set corresponding to the target application.

Correspondingly, extracting the first control instruction from the control instruction set corresponding to the target application may specifically include: in response to detecting that the version of the instruction set is the latest version, extracting the first control instruction from the control instruction set corresponding to the target application.

Specifically, the electronic device can determine whether the instruction set version of the control instruction set corresponding to the target application is the latest version by judging whether the detected version number is the latest version number. If the electronic device determines that the version number is the latest version number, it can It is determined that the version of the instruction set is the latest version. At this time, there is no need to update the control instruction set corresponding to the target application, and the first control instruction corresponding to the target interactive interface can be directly extracted from the control instruction set corresponding to the target application.

In some other embodiments, after detecting the instruction set version of the control instruction set corresponding to the target application, the voice control method may further include: in response to detecting that the instruction set version is not the latest version, downloading from the server the to-be-updated version corresponding to the target application. The control instruction set; the control instruction set corresponding to the target application is replaced by the control instruction set to be updated; the first control instruction is extracted from the control instruction set to be updated.

In some embodiments, if the electronic device determines that the version number is not the latest version number, it can determine that the instruction set version is not the latest version. At this time, the control instruction set corresponding to the target application needs to be updated, and the electronic device can send the control instruction set for the target application to the server. An instruction set update request, so that the server responds to receiving the control instruction set update request, and feeds back to the electronic device the latest version of the control instruction set corresponding to the target application, that is, the control instruction set corresponding to the target application to be updated, so as to download the corresponding version of the target application from the server. The control instruction set to be updated, and then use the control instruction set to be updated to replace the control instruction set corresponding to the target application, that is, the control instruction set to be updated is used as the new control instruction set corresponding to the target application, and the target control instruction set that is not the latest version is deleted, Then extract the first control instruction corresponding to the target interaction interface from the control instruction set to be updated, that is, the new control instruction set corresponding to the target application.

Therefore, in the embodiment of the present disclosure, the control instructions of all static controls can be sorted out in advance for each interactive interface of each application program, and then the control instructions of all static controls corresponding to all installed application programs can be pre-stored in the electronic device. instructions, and these control instructions are used as static preset content to realize fast loading of the first control instructions of the target interactive interface.

In yet another implementation manner of the present disclosure, when the control instruction includes a second control instruction generated according to the dynamic control data in the interface control data, the electronic device may generate the second control instruction according to the dynamic control data.

In some embodiments, loading the target control instruction set corresponding to the target interaction interface may specifically include: processing the dynamic control data to generate the second control instruction.

In some embodiments, the dynamic control is a control formed by filling control data in a dynamic content reserved field.

The processing process of the dynamic control data provided by the embodiment of the present disclosure will be described in detail below with reference to FIG. 4 .

Fig. 4 shows a schematic flowchart of a processing procedure of dynamic control data provided by an embodiment of the present disclosure.

As shown in FIG. 4, the processing process of the dynamic control data may include the following steps S410 to S430.

S410. Extract dynamic control text from the dynamic control data.

In the embodiment of the present disclosure, whether it is static control data or dynamic control data, it may include control text of the control, display parameters of the control, and the like. Each static control data can belong to a static control, and each dynamic control data can belong to a dynamic control.

In some embodiments, the electronic device may extract the dynamic control text of the dynamic control to which the dynamic control data belongs from the dynamic control data corresponding to the target interaction interface, and the dynamic control text may be the name of the control that the dynamic control can be seen by the user. .

Continuing to refer to FIG. 3B , the song list link 311 belongs to the dynamic control of the application main interface 305. Taking the song list link 311 of "Ambient Piano Music as Soul and Endless Void" as an example, its dynamic control text is "Ambient Piano Music as Soul and Endless Void" Endless Void Dialogue".

S420. Perform word segmentation processing on the dynamic control text to obtain a word segment set corresponding to the dynamic control text.

In the embodiment of the present disclosure, after extracting the dynamic control text of any dynamic control, the electronic device may perform word segmentation processing on the dynamic control text to obtain a word segment set corresponding to the dynamic control text, that is, a control text word segment set of the dynamic control.

In some embodiments, the electronic device can use any word segmentation processing algorithm to split the dynamic control text into multiple control text word segments, then combine any number of adjacent control text word segments to obtain multiple word segment combinations, and finally A word segment set corresponding to the dynamic control text including multiple control text word segments and combinations of multiple word segments is obtained.

In some embodiments, the method of combining multiple control text word segments and multiple word segment combinations to obtain a word segment set may include connecting multiple control text word segments and multiple word segment combinations with "|" to obtain a word segment set.

Therefore, in the embodiment of the present disclosure, the content of the participle set conforming to the EBNF grammatical paradigm can be obtained, so that the generated second control instruction can be loaded into the language model of the Grammar engine.

S430. Generate a second control instruction according to the word segmentation set.

In the embodiment of the present disclosure, after obtaining the word segmentation set corresponding to the dynamic control text, the electronic device may generate the second control instruction according to the word segmentation set based on a preset control instruction generation manner.

In some embodiments, S430 may specifically include: generating a second control instruction according to a preset verb set and participle set.

The electronic device can combine different pre-set verb sets and word segmentation sets to obtain multiple control instructions of the dynamic control, and the verb set in each control instruction includes multiple verbs with similar semantics. For each control command of each dynamic control, the electronic device can also use the control command corresponding to the verb set in the control command and the control function of the dynamic control corresponding to the participle set to determine the control mode corresponding to the control command.

In some embodiments of the present disclosure, before S420, the voice control method may further include: preprocessing the dynamic control text.

In the embodiment of the present disclosure, after the electronic device extracts the dynamic control text, before performing word segmentation processing on the dynamic control text to obtain the word segmentation set corresponding to the dynamic control text, it can also preprocess the dynamic control text to obtain Dynamic control text for text processing.

In some embodiments, pre-processing may include symbol removal processing, digital conversion processing.

Symbol elimination processing can be used to eliminate symbols in the dynamic control text, such as punctuation marks, special symbols, mathematical symbols and any other symbols that do not have semantic meaning.

Numeral conversion processing can be used to convert Arabic numerals in dynamic control text to Chinese numerals. Among them, if the Arabic numerals have more than two digits, the entire Arabic numerals can be converted into a Chinese numeral, or each numeral can be converted into a numeral respectively.

In some embodiments, after extracting the dynamic control text, the electronic device may first remove symbols in the dynamic control text to obtain the dynamic control text after removing symbols. Then, the electronic device can convert the Arabic numerals in the dynamic control text after removing symbols into Chinese numerals. Taking the dynamic control text "200 idiom song titles privately collected by the representative of the Chinese class" as an example, the Arabic numeral "200" can be converted into the Chinese numeral "two hundred", and it can also be converted into the Chinese numeral "200" to obtain the digital conversion The final dynamic control text is "The Chinese class represents the private collection of 200 idiom song names in the cheat sheet | The Chinese class represents the private collection of 200 idiom song titles in the private copy", the electronic device can perform word segmentation and word segmentation combination on the converted dynamic control text, Get the participle collection "Chinese class representative's private collection cheat sheet 200 idiom song titles | Chinese class representative private collection cheat sheet 200 idiom song titles | 200 idiom song titles | 200 idiom song titles | idiom song titles| Representative of the Chinese class|Privately kept a cheat sheet".

Therefore, in the embodiment of the present disclosure, after the electronic device displays the target interaction interface, it can generate control instructions for all the dynamic controls in the target interaction interface based on the data of each dynamic control in the target interaction interface, that is, the second control instruction, and then These control instructions are used as dynamic loading content to realize reliable and efficient loading of the second control instructions of the target interactive interface.

An example is used below to describe the voice control method provided by the embodiment of the present disclosure in detail.

Fig. 5 shows a schematic flowchart of another voice control method provided by an embodiment of the present disclosure.

As shown in FIG. 5, the voice control method may include the following steps S510 to S560.

S510. Displaying a target interaction interface.

In an embodiment of the present disclosure, an electronic device with a voice control function may display a target interaction interface, so that the user may perform voice control on the target interaction interface.

S520. Load the target control instruction set corresponding to the target interaction interface.

In the embodiment of the present disclosure, after displaying the target interaction interface, the electronic device may initialize the ASR engine, and load the language model whose instruction content is empty. Then, load the target control instruction set corresponding to the target interactive interface into the language model. During the start-up and initialization of the ASR engine and the loading of the target control instruction set, the electronic device does not receive user voice.

Wherein, the electronic device needs to load the first control instruction and the second control instruction in the target control instruction set into the language model.

The electronic device can first determine whether the application to which the preloaded language model belongs is the target application to which the target interactive interface belongs, and if so, load the target control instruction set into the language model; if not, reload the empty program corresponding to the target application. language model, and then load the target control instruction set into the reloaded language model.

In some embodiments, the electronic device may first determine the target application to which the target interactive interface belongs, and then query the control instruction set corresponding to the target application in multiple preset control instruction sets, and then determine the instruction set version of the control instruction set corresponding to the target application Whether it is the latest version, if it is determined that the instruction set version is the latest version, there is no need to update the control instruction set corresponding to the target application at this time, and the first control instruction corresponding to the target interactive interface can be directly extracted from the control instruction set corresponding to the target application; The set version is not the latest version. In this case, the control instruction set corresponding to the target application needs to be updated. You can download the control instruction set corresponding to the target application to be updated from the server, and use the control instruction set to be updated to replace the control instruction set corresponding to the target application. The first control instruction corresponding to the target interactive interface is extracted from the set of control instructions to be updated. After the first control instruction is acquired by the electronic device, the first control instruction can be loaded into the language model.

In some other embodiments, the electronic device may acquire dynamic control data corresponding to all dynamic controls in the target interaction interface. For the dynamic control data corresponding to each dynamic control, the electronic device can extract the dynamic control text of the dynamic control from the dynamic control data, then remove the symbols in the dynamic control text and convert the Arabic numerals in the dynamic control text into Chinese numerals, The preprocessed dynamic control text is obtained, and then word segmentation is performed on the dynamic control text to obtain a word segmentation set corresponding to the dynamic control text, and finally a second control command is generated according to the preset verb set and word segmentation set. After the second control instruction is acquired by the electronic device, the second control instruction may be loaded into the language model.

Among them, before generating the second control instruction according to the preset verb set and participle set, then adding different grammatical content to the code position corresponding to the language model, and finally compiling the updated language model into a binary language model resource file, and sent to the audio recognition model.

Further, the electronic device may also convert the first control instruction and the second control instruction into binary codes, and load them into the speech model.

S530. Receive user control voice.

In the embodiment of the present disclosure, after the electronic device finishes loading the target control instruction set, it may wait for the user to input a voice. If the start of human voice is recognized based on Voice Activity Detection (VAD), the recording will continue. If the end of the human voice is recognized based on the VAD, the recording is stopped. The electronic device can use the recorded audio as the user control voice. The electronic device can then input the user control voice into the ASR engine to obtain the target voice text corresponding to the user control voice.

S540. According to the target voice text corresponding to the user control voice, search for a target control command matching the user control voice in the target control command set.

In the embodiment of the present disclosure, the electronic device may search the target control command set for the target control command matching the target voice text.

S550. Determine whether the target control instruction is found.

In the embodiment of the present disclosure, the electronic device may determine whether the target control instruction is found, and if the target control instruction is found, execute S560, otherwise return to execute S530.

S560. Execute the target control operation corresponding to the target control instruction.

In the embodiment of the present disclosure, the electronic device may perform a control operation on the target interface control to which the control data generating the control command belongs according to the target control mode indicated by the target control command.

To sum up, in the embodiment of the present disclosure, since the verb set and the participle set are used to generate control instructions, it can support the voice control of thousands of levels of speech in a single interface, and the control instructions include the generation of static controls based on the interface. The static control instructions and the dynamic control instructions generated based on the dynamic controls in the interactive interface, so on the basis of being able to support a sufficiently large grammatical level, the control instructions can also be arbitrarily expanded to achieve an effect that is visible to the interactive interface. . In addition, the loading process of the control instruction and the recognition process of the user's control voice are independent of each other and do not interfere with each other, which can improve the accuracy of recognition.

Furthermore, in the embodiment of the present disclosure, both the ASR engine and the Grammar engine are offline engines, which can run on the terminal side (that is, run in the electronic device) without relying on the network. Moreover, the model of the engine is small enough and requires low computing power, so that the control commands that need to be supported in the interactive interface can be responded at a faster speed (on average, it is about 1.2s faster than the cloud recognition result, and faster than the offline general recognition result. 500ms), which can bring higher benefits in vehicle application scenarios.

Fig. 6 shows a schematic structural diagram of a voice control device provided by an embodiment of the present disclosure.

In some embodiments of the present disclosure, the apparatus shown in FIG. 6 may be applied to electronic equipment. Wherein, the electronic device may include a mobile phone, a tablet computer, a desktop computer, a notebook computer, a vehicle terminal, a wearable electronic device, a smart home device, and other devices with a voice control function.

As shown in FIG. 6 , the voice control device 600 may include an interface display module 610 , an instruction loading module 620 , an instruction matching module 630 and an instruction execution module 640 .

The interface display module 610 may be configured to display a target interaction interface.

The instruction loading module 620 may be configured to load a target control instruction set corresponding to the target interactive interface, where the target control instruction set includes control instructions generated according to interface control data of the target interactive interface.

The instruction matching module 630 may be configured to, in response to receiving the user control speech, query the target control instruction set for the target control instruction matching the user control speech.

The instruction executing module 640 may be configured to execute the target control operation corresponding to the target control instruction in response to querying the target control instruction.

In the embodiment of the present disclosure, after the target interaction interface is displayed, the target control instruction set corresponding to the target interaction interface can be loaded, and then when the user control voice is received, the target control command set can be queried with the received user control voice match the target control instruction, and execute the queried target control instruction, and then realize the user's voice control of the target interactive interface. Since the loaded target control instruction set includes control instructions generated according to the interface control data of the target interactive interface, the The interface control data can cover all the interface controls in the target interactive interface. Therefore, all the voice control of the target interactive interface can be fully realized, thereby achieving the effect of being visible and talking about the target interactive interface, and improving the user experience.

In some embodiments of the present disclosure, the control instruction may include a first control instruction generated according to static control data in the interface control data.

In some embodiments of the present disclosure, the interface display module 620 may further include an application determination unit, a first query unit, and a first extraction unit.

The application determining unit may be configured to determine the target application to which the target interactive interface belongs.

The first query unit may be configured to query the target control command set corresponding to the target application among the multiple preset control command sets stored in advance.

The first extracting unit may also be configured to extract the first control instruction from the control instruction set corresponding to the target application.

In some embodiments of the present disclosure, the interface display module 620 may further include a version detection unit, which may be configured to detect the instruction set version of the control instruction set corresponding to the target application before extracting the first control instruction from the target control instruction set .

The first extraction unit may be further configured to extract the first control instruction from the target control instruction set when the version detection unit detects that the instruction set version is the latest version.

In some embodiments of the present disclosure, the interface display module 620 may further include an instruction set downloading unit, a first processing unit, and a second extracting unit.

The instruction set downloading unit may be configured to download the control instruction set to be updated corresponding to the target application from the server if it detects that the instruction set version is not the latest version after detecting the instruction set version of the control instruction set corresponding to the target application.

The first processing unit may be configured to replace the target control instruction set with the control instruction set to be updated.

The second extracting unit may be configured to extract the first control instruction from the set of control instructions to be updated.

In some embodiments of the present disclosure, the control instruction may include a second control instruction generated according to the dynamic control data in the interface control data.

In some embodiments of the present disclosure, the interface display module 620 may further include a third extracting unit, a second processing unit, and an instruction generating unit.

The third extracting unit may be configured to extract dynamic control text from the dynamic control data.

The second processing unit may be configured to perform word segmentation processing on the dynamic control text to obtain a word segmentation set corresponding to the dynamic control text.

The instruction generation unit may be configured to generate a second control instruction according to the word segmentation set.

In some embodiments of the present disclosure, the interface display module 620 may further include a third processing unit, and the third processing unit may be configured to perform word segmentation processing on the dynamic control text to obtain the word segmentation set corresponding to the dynamic control text, The text is pre-processed, wherein the pre-processing includes symbol elimination processing and digital conversion processing.

In some embodiments of the present disclosure, the instruction generation unit may be further configured to generate a second control instruction according to a preset verb set and participle set.

In some embodiments of the present disclosure, the interface display module 630 may include a text conversion unit and a second query unit.

The text converting unit may be configured to convert the user control speech into target speech text.

The second query unit may be configured to query the target control command set that matches the target voice text.

In some embodiments of the present disclosure, the instruction executing module 640 may be further configured to execute a target control operation for the target interface control involved in the target control command.

It should be noted that the voice control device 600 shown in FIG. 6 can execute each step in the method embodiment shown in FIG. 1 to FIG. 5 , and realize each process and effects, which will not be described here.

Fig. 7 shows a schematic structural diagram of a voice control device provided by an embodiment of the present disclosure.

In some embodiments of the present disclosure, the voice control device shown in FIG. 7 may be an electronic device.

Wherein, the electronic device may include a mobile phone, a tablet computer, a desktop computer, a notebook computer, a vehicle terminal, a wearable electronic device, a smart home device, and other devices with a voice control function.

As shown in Figure 7, the voice control device may include a processor 701 and a memory 702 storing computer program instructions.

Specifically, the processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured to implement one or more integrated circuits in the embodiments of the present application.

Memory 702 may include mass storage for information or instructions. By way of example and not limitation, memory 702 may include a hard disk drive (Hard Disk Drive, HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (Universal Serial Bus, USB) drive or two or more thereof. A combination of the above. Storage 602 may include removable or non-removable (or fixed) media, where appropriate. Memory 602 may be internal or external to the integrated gateway device, where appropriate. In a particular embodiment, memory 602 is a non-volatile solid-state memory. In a particular embodiment, the memory 702 includes a read-only memory (Read-Only Memory, ROM). Where appropriate, the ROM can be a mask programmed ROM, a programmable ROM (Programmable ROM, PROM), an erasable PROM (Electrical Programmable ROM, EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM) ), electrically rewritable ROM (Electrically Alterable ROM, EAROM) or flash memory, or a combination of two or more of these.

The processor 701 reads and executes the computer program instructions stored in the memory 702 to execute the steps of the voice control method provided by the embodiments of the present disclosure.

In an example, the voice control device may further include a transceiver 703 and a bus 704 . Wherein, as shown in FIG. 7 , a processor 701 , a memory 702 and a transceiver 703 are connected through a bus 704 and complete mutual communication.

Bus 704 includes hardware, software, or both. By way of example and not limitation, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Super Transmission (Hyper Transport, HT) interconnection, Industrial Standard Architecture (Industrial Standard Architecture, ISA) bus, Infinity Bandwidth interconnection, Low Pin Count (Low Pin Count, LPC) bus, memory bus, Micro Channel Architecture (Micro Channel Architecture) , MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (Serial Advanced Technology Attachment, SATA) bus, Video Electronics Standards Association local (Video Electronics Standards Association Local Bus, VLB) bus or other suitable bus or a combination of two or more of these. Bus 704 may comprise one or more buses, where appropriate. Although the embodiments of this application describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

The embodiment of the present disclosure also provides a computer-readable storage medium, the storage medium can store a computer program, and when the computer program is executed by the processor, the processor implements the voice control method provided by the embodiment of the present disclosure.

The above-mentioned storage medium may include, for example, a memory 702 of computer program instructions, and the above-mentioned instructions can be executed by the processor 701 of the voice control device to complete the voice control method provided by the embodiment of the present disclosure. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be ROM, random access memory (Random Access Memory, RAM), compact disc read-only memory (Compact Disc ROM) , CD-ROM), tapes, floppy disks and optical data storage devices, etc.

An embodiment of the present disclosure also provides a computer program product, including a computer program or an instruction. When the computer program or instruction is executed by a processor, the voice control method described in the first aspect is implemented.

It should be noted that in this article, relative terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these No such actual relationship or order exists between entities or operations. Moreover, the term "comprising" is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed, or is also included as such. An element inherent in a process, method, article, or device.

The above descriptions are only specific implementation manners of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (15)

1. A voice control method, comprising:

   Display the target interface;

   Loading the target control instruction set corresponding to the target interaction interface, wherein the target control instruction set includes control instructions generated according to the interface control data of the target interaction interface;

   In response to receiving the user control voice, querying the target control command set for a target control command that matches the user control voice;

   In response to finding the target control instruction, execute the target control operation corresponding to the target control instruction.
2. The method according to claim 1, wherein the control instruction comprises a first control instruction generated according to static control data in the interface control data.
3. The method according to claim 2, wherein said loading the target control instruction set corresponding to the target interactive interface comprises:

   determining the target application to which the target interactive interface belongs;

   Querying the control instruction set corresponding to the target application among the plurality of pre-stored preset control instruction sets;

   Extract the first control instruction from the control instruction set corresponding to the target application.
4. The method according to claim 3, wherein, before extracting the first control instruction from the control instruction set corresponding to the target application, the method further comprises:

   Detecting the instruction set version of the control instruction set corresponding to the target application;

   Wherein, extracting the first control instruction from the control instruction set corresponding to the target application includes:

   In response to detecting that the instruction set version is the latest version, the first control instruction is extracted from the control instruction set corresponding to the target application.
5. The method according to claim 4, wherein, after detecting the instruction set version of the control instruction set corresponding to the target application, the method further comprises:

   In response to detecting that the version of the instruction set is not the latest version, downloading the control instruction set to be updated corresponding to the target application from the server;

   Using the control instruction set to be updated to replace the control instruction set corresponding to the target application;

   In the set of control instructions to be updated, extract the first control instruction.
6. The method according to claim 1, wherein the control instruction comprises a second control instruction generated according to the dynamic control data in the interface control data.
7. The method according to claim 6, wherein said loading the target control instruction set corresponding to the target interactive interface comprises:

   Extracting dynamic control text from the dynamic control data;

   performing word segmentation processing on the dynamic control text to obtain a word segmentation set corresponding to the dynamic control text;

   The second control instruction is generated according to the word segmentation set.
8. The method according to claim 7, wherein, before performing word segmentation processing on the dynamic control text to obtain the word segmentation set corresponding to the dynamic control text, the method further includes:

   Preprocessing the dynamic control text;

   Wherein, the preprocessing includes symbol elimination processing and digital conversion processing.
9. The method according to claim 7, wherein said generating said second control instruction according to said word segmentation set comprises:

   The second control instruction is generated according to the preset verb set and the participle set.
10. The method according to claim 1, wherein the querying the target control command set for the target control command matching the user control voice comprises:

    Converting the user-controlled voice into target voice text;

    The target control instruction matching the target speech text is searched in the target control instruction set.
11. The method according to claim 1, wherein the executing the target control operation corresponding to the target control instruction comprises:

    The target control operation is executed for the target interface control involved in the target control instruction.
12. A voice control device comprising:

    an interface display module configured to display a target interactive interface;

    The instruction loading module is configured to load the target control instruction set corresponding to the target interaction interface, wherein the target control instruction set includes control instructions generated according to the interface control data of the target interaction interface;

    An instruction matching module, configured to, in response to receiving a user control voice, query the target control instruction set for a target control instruction that matches the user control voice;

    The instruction execution module is configured to execute the target control operation corresponding to the target control instruction in response to querying the target control instruction.
13. A voice control device comprising:

    processor;

    memory for storing executable instructions;

    Wherein, the processor is configured to read the executable instructions from the memory, and execute the executable instructions to implement the voice control method according to any one of claims 1 to 11.
14. A computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the voice control method according to any one of claims 1 to 11 .
15. A computer program product, including computer programs or instructions, when the computer programs or instructions are executed by a processor, the voice control method according to any one of claims 1 to 11 is realized.

Images