WO2023212993A1

WO2023212993A1 - Appliance control method, storage medium, and electronic device

Info

Publication number: WO2023212993A1
Application number: PCT/CN2022/096401
Authority: WO
Inventors: 刘建国; 王迪; 朱毅
Original assignee: 青岛海尔科技有限公司; 海尔智家股份有限公司
Priority date: 2022-05-05
Filing date: 2022-05-31
Publication date: 2023-11-09
Also published as: CN117059083A

Abstract

An appliance control method, a storage medium, and an electronic device. The appliance control method comprises: if a target audio is obtained, recognizing the target audio to obtain a statement to be recognized corresponding to the target audio, wherein the target audio is used for requesting to control at least one controlled appliance (S202); inputting the statement to be recognized into a target statement recognition model to obtain a target intention and a target word (S204); and sending a control instruction to a target controlled appliance according to the target intention and the target word, wherein the at least one controlled appliance comprises the target controlled appliance (S206). The method solves the technical problem of large pressure on a server caused by the requirement of two models for the recognition of a voice control instruction.

Description

Equipment control method, storage medium and electronic device

Technical field

The present application relates to the field of communications, specifically, to an equipment control method, a storage medium and an electronic device.

Background technique

Currently, with the development of smart homes, more and more smart homes can be controlled through voice. In task-based conversations, intent recognition and slot filling are issues that the semantic understanding module (Nartul Language Understanding, or NLU for short) cannot avoid.

In related technologies, traditional task-based dialogue requires at least two models to identify intent and slot respectively. In other words, the user's voice control instructions need to be input into the two models. As the number of smart devices increases, servers will be put under excessive pressure.

In related technologies, no effective solution has yet been proposed for the technical problem that requires two models to recognize voice control instructions, which causes great pressure on the server.

Contents of the invention

Embodiments of the present application provide a device control method, a storage medium, and an electronic device to at least solve the technical problem in related technologies that requires two models to recognize voice control instructions, resulting in high pressure on the server.

According to an embodiment of the present application, a device control method is provided, including: when the target audio is obtained, identifying the target audio, and obtaining a sentence to be recognized corresponding to the target audio, wherein the target audio is In response to a request to control at least one controlled device; input the above-mentioned sentence to be recognized into the target sentence recognition model to obtain the target meaning and the target word, wherein the above-mentioned target sentence recognition model includes a first recognition substructure and a second recognition substructure, and the above-mentioned third recognition substructure An identification substructure is used to identify the ideographic direction of the sentence, and the above-mentioned second recognition result is used to identify the word data in the sentence. The above-mentioned target sentence recognition model uses multiple sample sentence data to compare the above-mentioned first recognition substructure and the above-mentioned second recognition. A model for identifying the ideographic direction and word data of a sentence obtained through joint training of substructures; sending control instructions to the target controlled device according to the target ideogram and the target word, wherein the at least one controlled device includes the target controlled device. control equipment.

In an exemplary embodiment, sending the control instruction to the target controlled device according to the target expression and the target word includes: obtaining the control instruction and instruction sending information according to the target expression and the target word, wherein the instruction sending The information is used to instruct the above-mentioned control instruction to be sent to the above-mentioned target controlled device in the above-mentioned at least one controlled device.

In an exemplary embodiment, the above-mentioned obtaining the above-mentioned control instruction and the above-mentioned instruction sending information based on the above-mentioned target meaning and the above-mentioned target word includes: determining the execution authority of the above-mentioned control instruction based on the above-mentioned target meaning; and determining the above-mentioned instruction sending based on the above-mentioned target word. information and the instruction content of the above control instructions.

In an exemplary embodiment, before acquiring the target audio, the method includes: acquiring multiple sample sentence data; marking the sentence data in each of the above sample sentence data to obtain the marked multiple sample sentence data, Wherein, each marked sample sentence data includes a marked ideographic identifier and a word identifier, the above-mentioned ideographic identifier is used to mark the ideographic direction of the above-mentioned sentence data, and the above-mentioned word identifier is used to mark at least one word data in the above-mentioned sentence data; from The current sample sentence data is determined from the marked plurality of sample sentence data, and an initial sentence recognition model is determined, wherein the above-mentioned initial sentence recognition model includes a first recognition substructure and a second recognition substructure, and the above-mentioned first recognition substructure The above-mentioned second recognition result is used to identify the word data in the above-mentioned sentence data; the above-mentioned current sample sentence data is input into the above-mentioned first recognition sub-structure and the above-mentioned second recognition sub-structure respectively to obtain the above-mentioned The first recognition result output by the first recognition substructure, and the second recognition result output by the above-mentioned second recognition substructure; according to the above-mentioned first recognition result and the ideographic identification of the above-mentioned current sample sentence data, determine the above-mentioned first recognition substructure a first training loss value; and, based on the above-mentioned second recognition result and the word identifier of the above-mentioned current sample sentence data, determine a second training loss value of the above-mentioned second recognition substructure; based on the above-mentioned first training loss value and the above-mentioned second training loss value to obtain the current training loss value, where the above-mentioned current training loss value is used to determine the training status of the above-mentioned initial sentence recognition model; when the above-mentioned current training loss value does not reach the recognition convergence condition, the above-mentioned multiple marked Obtain the next sample sentence data from the sample sentence data and input it into the above-mentioned initial sentence recognition model; when the above-mentioned current training loss value reaches the above-mentioned convergence condition for recognition, the above-mentioned initial sentence recognition model is determined to be the above-mentioned target sentence recognition model.

In an exemplary embodiment, obtaining the current training loss value based on the above-mentioned first training loss value and the above-mentioned second training loss value includes: performing the above-mentioned first training loss value and the above-mentioned second training loss value according to the first The target weight value and the second target weight value are integrated to obtain the above-mentioned current training loss value, wherein the above-mentioned first target weight value is the weight value of the above-mentioned first training loss value, and the above-mentioned second target weight value is the above-mentioned second training loss value. The weight value of the value.

In an exemplary embodiment, the method further includes: when the word identifier of the current sample sentence data includes a target hidden identifier, the third recognition result output according to the second recognition substructure and the word identifier of the current sample sentence data. , determine the third training loss value of the above-mentioned target hidden identifier; the above-mentioned first training loss value, the above-mentioned second training loss value and the above-mentioned third training loss value are respectively based on the first target weight value, the second target weight value and the third The target weight values are integrated to obtain the above-mentioned current training loss value, wherein the above-mentioned first target weight value is the weight value of the above-mentioned first training loss value, the above-mentioned second target weight value is the weight value of the above-mentioned second training loss value, and the above-mentioned The third target weight value is the weight value of the above-mentioned third training loss value.

In an exemplary embodiment, when the current training loss value does not reach the recognition convergence condition, obtaining the next sample sentence data from the marked plurality of sample sentence data and inputting it into the above initial sentence recognition model includes: When the current training loss value does not reach the recognition convergence condition, the parameters of the first recognition substructure and the second recognition substructure are adjusted to obtain an adjusted initial sentence recognition model; from the marked multiple Obtain the next sample sentence data from the sample sentence data and input it into the above-adjusted initial sentence recognition model.

According to another embodiment of the present application, a device control device is also provided, including: an identification module configured to identify the target audio when the target audio is obtained, and obtain the target audio corresponding to the target audio to be identified. statement, wherein the above-mentioned target audio is used to request the control of at least one controlled device; the input module is configured to input the above-mentioned sentence to be recognized into the target sentence recognition model to obtain the target ideogram and the target word, wherein the above-mentioned target sentence recognition model includes A first recognition substructure and a second recognition substructure. The first recognition substructure is used to identify the ideographic direction of a sentence. The second recognition result is used to identify word data in the sentence. The target sentence recognition model uses multiple samples. The sentence data is a model for identifying the ideographic direction and word data of the sentence, which is obtained by jointly training the above-mentioned first recognition substructure and the above-mentioned second recognition substructure; the sending module is set to be based on the above-mentioned target ideogram and the above-mentioned target word direction. The target controlled device sends a control instruction, wherein the at least one controlled device includes the target controlled device.

According to yet another aspect of the embodiment of the present application, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the above device control method when running. .

According to another aspect of the embodiment of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above device through the computer program Control Method.

In the embodiment of the present application, when the target audio is obtained, the target audio is identified and a sentence to be recognized corresponding to the target audio is obtained, where the target audio is used to request control of at least one controlled device; the sentence to be recognized is input to the target The sentence recognition model obtains the target meaning and the target word. The target sentence recognition model includes a first recognition substructure and a second recognition substructure. The first recognition substructure is used to identify the ideographic direction of the sentence, and the second recognition result is used to identify The word data in the sentence, the target sentence recognition model is a model obtained by jointly training the first recognition substructure and the second recognition substructure using multiple sample sentence data, and is used to recognize the ideographic direction of the sentence and the word data; according to the target The ideographic and target words send control instructions to the target controlled device, wherein at least one controlled device includes the target controlled device. Using the above technical solution, two substructures are set up in the target sentence recognition model to identify the ideographic direction and word data respectively, so that one model can be used to identify the intention and slot of the sentence at the same time, thus reducing the pressure on the server and solving the problem. Technical issues that put a lot of pressure on the server due to the need for two models to recognize voice control commands.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings needed to describe the embodiments or related technologies. Obviously, for those of ordinary skill in the art, Other drawings can also be obtained based on these drawings without incurring any creative effort.

Figure 1 is a hardware structure block diagram of a computer terminal according to an optional device control method according to an embodiment of the present application;

Figure 2 is a flow chart of an optional device control method according to an embodiment of the present application;

Figure 3 is a schematic diagram of an optional device control method according to an embodiment of the present application;

Figure 4 is a schematic diagram of another optional device control method according to an embodiment of the present application;

Figure 5 is a structural block diagram of an optional equipment control device according to an embodiment of the present application.

Detailed ways

In order to enable those in the technical field to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only These are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of protection of this application.

It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that encompasses a series of steps or units and need not be limited to those explicitly listed. Those steps or elements may instead include other steps or elements not expressly listed or inherent to the process, method, product or apparatus.

According to an aspect of an embodiment of the present application, a device control method is provided. This device control method is widely used in whole-house intelligent digital control application scenarios such as Smart Home, Smart Home, Smart Home Equipment Ecology, and Intelligence House Ecology. Optionally, in this embodiment, the above device control method can be applied to the hardware environment composed of the terminal device 102 and the server 104 as shown in FIG. 1 . As shown in Figure 1, the server 104 is connected to the terminal device 102 through the network and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal. A database can be set up on the server or independently from the server. To provide data storage services for the server 104, cloud computing and/or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

The above-mentioned network may include but is not limited to at least one of the following: wired network, wireless network. The above-mentioned wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, and local area network. The above-mentioned wireless network may include at least one of the following: WIFI (Wireless Fidelity, Wireless Fidelity), Bluetooth. The terminal device 102 may be, but is not limited to, a PC, a mobile phone, a tablet, a smart air conditioner, a smart hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing equipment, a smart dishwasher, or a smart projection device. , smart TV, smart clothes drying rack, smart curtains, smart audio and video, smart sockets, smart audio, smart speakers, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart sweeping robot, smart window cleaning robot, smart mopping robot, Smart air purification equipment, smart steamers, smart microwave ovens, smart kitchen appliances, smart purifiers, smart water dispensers, smart door locks, etc.

This embodiment provides a device control method. Figure 2 is a flow chart of the device control method according to the embodiment of the present application. The process includes the following steps:

Step S202: When the target audio is obtained, identify the target audio and obtain the sentence to be recognized corresponding to the target audio, where the target audio is used to request control of at least one controlled device;

Step S204: Input the sentence to be recognized into the target sentence recognition model to obtain the target meaning and the target word. The target sentence recognition model includes a first recognition substructure and a second recognition substructure. The first recognition substructure is used to recognize the sentence. Indicative direction, the second recognition result is used to identify word data in the sentence, and the target sentence recognition model is a model for recognizing sentences obtained by jointly training the first recognition substructure and the second recognition substructure using multiple sample sentence data. Models of ideographic pointing and word data;

Step S206: Send a control instruction to the target controlled device according to the target meaning and the target word, where at least one controlled device includes the target controlled device.

Optionally, in this embodiment, the user may, but is not limited to, control the controlled device through voice control instructions; the target audio may include, but is not limited to, the audio used to control the controlled device.

Optionally, in this embodiment, the target sentence recognition model may include, but is not limited to, two recognition substructures, respectively used to identify the ideographic direction and word data of the sentence. The two recognition substructures may, but are not limited to, be trained through joint training. get.

Optionally, in this embodiment, the target controlled device and the control instruction sent to the target controlled device can be determined according to the target meaning and the target word.

Through the solution provided by the embodiment of the present application, when the target audio is obtained, the target audio is recognized and the sentence to be recognized corresponding to the target audio is obtained, where the target audio is used to request control of at least one controlled device; the sentence to be recognized is input Go to the target sentence recognition model to obtain the target meaning and target words. The target sentence recognition model includes a first recognition substructure and a second recognition substructure. The first recognition substructure is used to identify the ideographic direction of the sentence, and the second recognition result is For identifying the word data in the sentence, the target sentence recognition model is a model obtained by jointly training the first recognition substructure and the second recognition substructure using multiple sample sentence data, and is used to identify the ideographic direction of the sentence and the word data; Send a control instruction to the target controlled device according to the target meaning and the target word, wherein at least one controlled device includes the target controlled device. Using the above technical solution, two substructures are set up in the target sentence recognition model to identify the ideographic direction and word data respectively, so that one model can be used to identify the intention and slot of the sentence at the same time, thus reducing the pressure on the server and solving the problem. Technical issues that put a lot of pressure on the server due to the need for two models to recognize voice control commands.

In an exemplary embodiment, sending a control instruction to the target controlled device according to the target meaning and the target word includes: obtaining the control instruction and command sending information according to the target meaning and the target word, wherein the command sending information is used to indicate that the control instruction will be Sent to the target controlled device in at least one controlled device.

Optionally, in this embodiment, the intention of the target audio can be determined based on the target meaning, and the target controlled device and the instruction content of the control instruction can be determined based on the target words. For example, if the target audio is "turn on the TV in the living room", then the target meaning is "home appliance control", the target controlled device is "the TV in the living room", and the command content of the control instruction is "turn on the TV in the living room".

Through the solution provided by the embodiments of this application, the controlled device is controlled through voice interaction, which improves the user experience.

In an exemplary embodiment, obtaining the control instruction and instruction sending information based on the target meaning and the target word includes: determining the execution authority of the control instruction based on the target meaning; and determining the instruction sending information and the instruction content of the control instruction based on the target word.

Optionally, in this embodiment, the intention of the target audio can be determined based on the target meaning, and the target controlled device and the instruction content of the control instruction can be determined based on the target words. For example, if the target audio is "turn on the air conditioner in the living room", then the target meaning is "home appliance control", the target controlled device is "the air conditioner in the living room", and the command content of the control instruction is "turn on the air conditioner in the living room".

In an exemplary embodiment, before acquiring the target audio, the method includes: acquiring multiple sample sentence data; marking the sentence data in each sample sentence data to obtain multiple marked sample sentence data, wherein, Each marked sample sentence data includes a marked ideographic identifier and a word identifier. The ideographic identifier is used to mark the ideographic direction of the sentence data, and the word identifier is used to mark at least one word data in the sentence data; from multiple marked samples The current sample sentence data is determined in the sentence data, and an initial sentence recognition model is determined, wherein the initial sentence recognition model includes a first recognition substructure and a second recognition substructure, and the first recognition substructure is used to identify the ideographic direction of the sentence data, The second recognition result is used to identify word data in the sentence data; input the current sample sentence data into the first recognition substructure and the second recognition substructure respectively to obtain the first recognition result output by the first recognition substructure and the second recognition result. The second recognition result output by the substructure; determining the first training loss value of the first recognition substructure according to the first recognition result and the ideographic identification of the current sample sentence data; and, according to the second recognition result and the words of the current sample sentence data identify, determine the second training loss value of the second recognition substructure; obtain the current training loss value according to the first training loss value and the second training loss value, where the current training loss value is used to determine the training status of the initial sentence recognition model ; When the current training loss value does not reach the recognition convergence condition, obtain the next sample sentence data from the marked multiple sample sentence data and input it into the initial sentence recognition model; when the current training loss value reaches the recognition convergence condition, Determine the initial sentence recognition model as the target sentence recognition model.

Optionally, in this embodiment, before training the sentence recognition model, multiple sample sentence data can be obtained in advance from a local database or server, where the types of the multiple sample sentence data can be the same or different, No limitation is made here.

Optionally, in this embodiment, marking the statement data in each sample statement data may include adding an ideographic identifier in front of each sample statement data. For example, the ideographic identifier may be the identifier CLS, which is not limited here. .

Optionally, in this embodiment, the first identification substructure and the second identification substructure may be different substructures, or may be partially the same and partially different substructures, which are not limited here.

Optionally, in this embodiment, after the first recognition substructure outputs the first recognition result, the first recognition substructure of the first recognition substructure may be calculated based on the first recognition result and the real or expected ideographic identification of the current sample sentence data. The training loss value is the difference between the first recognition result and the real or expected ideographic identification of the current sample sentence data; after the second recognition substructure outputs the second recognition result, the second recognition result and the current sample sentence data can be The real or expected word identifier is used to calculate the second training loss value of the second recognition substructure, that is, the difference between the second recognition result and the real or expected word identifier of the current sample sentence data.

Optionally, in this embodiment, different weight values can be used to integrate the first training loss value and the second training loss value respectively to obtain the training loss value of the entire sentence recognition model, which is not limited here.

Optionally, in this embodiment, the convergence condition of the sentence recognition model can be set in advance. After obtaining the training loss value of the entire sentence recognition model, the overall training loss value is compared with the convergence condition to determine the overall training loss value. Whether the loss value meets the convergence condition; if not, continue training the sentence recognition model; if it is satisfied, the target sentence recognition model can be obtained.

Through the solution provided by the embodiments of this application, by adding ideographic identifiers to the sample sentence data, a training model can be used to simultaneously identify the intention and slot of the sentence, thereby reducing the training cost.

In an exemplary embodiment, obtaining the current training loss value according to the first training loss value and the second training loss value includes: adjusting the first training loss value and the second training loss value according to the first target weight value and the second training loss value respectively. The two target weight values are integrated to obtain the current training loss value, where the first target weight value is the weight value of the first training loss value, and the second target weight value is the weight value of the second training loss value.

For example, assuming α is the weight value of the first training loss value Loss ₁ and β is the weight value of the second training loss value Loss ₂ , then the current training loss value is Loss _total = αLoss ₁ + βLoss ₂ . Among them, α and β can be adjusted differently for different training tasks.

Through the solution provided by the embodiments of this application, by using weight values to integrate the training loss values, the accuracy of the overall training loss value of the sentence recognition model can be ensured.

In an exemplary embodiment, the method further includes: when the word identifier of the current sample sentence data includes the target hidden identifier, determining the target according to the third recognition result output by the second recognition substructure and the word identifier of the current sample sentence data. Hidden the third training loss value of the logo; integrate the first training loss value, the second training loss value and the third training loss value according to the first target weight value, the second target weight value and the third target weight value respectively to obtain The current training loss value, where the first target weight value is the weight value of the first training loss value, the second target weight value is the weight value of the second training loss value, and the third target weight value is the weight value of the third training loss value value.

It should be noted that since the sentence data will include various types of words, some words, such as unregistered words, may not be recognized by the sentence recognition model. Therefore, we use hidden identifiers for words that cannot be recognized by the sentence recognition model. Instead, the sentence recognition model is used to predict the word identifier corresponding to the hidden identifier, and the third training loss value of the hidden identifier can be calculated based on the recognition result of the sentence recognition model and the real or expected word identifier corresponding to the hidden identifier. ; Furthermore, different weight values can be used to integrate the first training loss value, the second training loss value and the third training loss value respectively to obtain the overall training loss value of the sentence recognition model, which is not limited here.

For example, let α be the weight value of the first training loss value Loss ₁ , β be the weight value of the second training loss value Loss ₂ , and γ be the weight value of the second training loss value Loss _3. Then, the current training loss value is Loss _total =αLoss ₁ +βLoss ₂ +γLoss ₃ . Among them, α+100β+γ=1, α, β and γ can be adjusted differently for different training tasks.

In an exemplary embodiment, when the current training loss value does not reach the recognition convergence condition, the next sample sentence data is obtained from the marked multiple sample sentence data and input into the initial sentence recognition model, including: at the current training loss If the value does not reach the recognition convergence condition, adjust the parameters of the first recognition substructure and the second recognition substructure to obtain the adjusted initial sentence recognition model; obtain the next sample from the marked multiple sample sentence data Sentence data is input into the adjusted initial sentence recognition model.

Optionally, in this embodiment, when the current training loss value does not reach the recognition convergence condition, the parameters of the first recognition substructure and the second recognition substructure can be adjusted so that the current training loss value reaches the recognition convergence condition as soon as possible. Identify the convergence condition, wherein the parameters of the first identification substructure and the second identification substructure can be adjusted simultaneously, or the parameters of the first identification substructure and the second identification substructure can be adjusted individually. For example, in a parameter adjustment, the parameters of the first identification substructure and the second identification substructure can be adjusted simultaneously, or the parameters of the first identification substructure and the second identification substructure can be adjusted sequentially, that is, the first identification substructure is currently adjusted. The parameters of the structure, the parameters of the second identification substructure will be adjusted next time, and are not limited here.

Through the solutions provided by the embodiments of this application, by adjusting the parameters of the sentence recognition model, the convergence speed of the model can be improved.

In an exemplary embodiment, after obtaining multiple sample sentence data, the method includes: performing a data preprocessing operation on the multiple sample sentence data to obtain processed multiple sample sentence data, wherein the data preprocessing operation includes at least the following: One: Convert full-width to half-width, convert uppercase numbers to lowercase numbers, convert uppercase letters to lowercase letters, remove emoticons, word segmentation, stop word filtering.

Optionally, in this embodiment, after obtaining multiple sample sentence data, you can first perform data preprocessing operations on the sample sentence data, which mainly includes: converting full-width to half-width, converting uppercase numbers to lowercase numbers, converting uppercase letters to lowercase letters, Emoji removal, word segmentation, stop word filtering. Among them, word segmentation is the process of recombining continuous word sequences into word sequences according to certain specifications. In English writing, spaces are used as natural delimiters between words. Each word can also be regarded as part of a sequence, which is not limited here.

Through the solution provided by the embodiments of this application, by performing data preprocessing operations on the sample sentence data, the training speed and training accuracy of the sentence recognition model can be improved.

In order to better understand the process of the training method of the above sentence recognition model, the implementation method of training the above sentence recognition model will be described below with reference to optional embodiments, but this is not intended to limit the technical solutions of the embodiments of the present application.

This embodiment provides a model structure of a sentence recognition model, as shown in Figure 3.

1) Input layer

Since each word is encoded with One-hot, and One-hot encoding is a discrete encoding, for the sentence recognition model, sparse vectors are very unfavorable for deep model learning and waste storage space, so word embedding ( Word Embedding), mapping sparse One-hot encoding into dense vectors.

Since the sentence recognition model uses a two-way auto-encoding structure, all the words in a sentence are input to the sentence recognition model at the same time. The sentence recognition model needs to know the position information between words, so position encoding (Position Embedding) is added. The formula is:

PE _t,2i =sin(t/1000 ^2i/d )

PE _t,2i+1 =cos(t/1000 ^2i/d )

Among them, t is the absolute position of the text, d represents the vector dimension of each text, and i represents the index of the dimension.

Add word embedding and position embedding as input to the model.

2) Feature extraction layer

The feature extraction layer is shown in Figure 4. It uses the self-attention (Self-Attention) mechanism to define three matrices K, Q, and V respectively. The input vector is multiplied by the three vectors K, Q, and V respectively to obtain three Corresponding eigenvectors K ₁ , Q ₁ , V ₁ , bring the obtained three eigenvectors into the formula

Among them, T represents the transpose of the matrix, that is, we can get the attention weight we need.

Taking three matrices as a group, because features may have multiple situations, multiple groups can be defined. For example, 12 groups, that is, 12 K, 12 Q, and 12 V, can be defined to better extract features.

This step is repeated once, called one layer. This model has four such structures, that is, four layers.

Between each layer, a residual network is also added to make up for the problem of information loss, that is, the input of the previous layer is used and added to the output of this layer.

Before inputting into the model, a batch of data needs to be normalized to unify all sentences into one length. Parts with insufficient length need to be filled with zeros, and parts that are too long need to be deleted. This causes errors in the batch normalization (BatchNormalize) stage. Therefore, layer normalization (LayerNormalize) is introduced in this application. Different from BatchNormalize, LayerNormalize focuses on each data rather than a batch of data, which can greatly reduce data errors caused by unifying sentence lengths.

3)Ideographic recognition

For each statement, an identifier CLS is added to represent the sentence. The features output by the feature extraction layer in the statement recognition model corresponding to CLS are brought into a layer of feedforward neural network (FeedforWard), and then connected to multiple ( Number of categories) SigMoid layer, put the final output into the cross entropy loss function (Cross Entropy Loss function), and obtain LOSS ₁ .

4) Word recognition

The features output by the feature extraction layer in the sentence recognition model corresponding to each word in the sentence are brought into the conditional random field (CRF), and the highest-scoring sequence prediction value and LOSS ₂ are obtained.

5)Mask mask

In practical applications, you will encounter a large number of out-of-vocab (OOV) problems. For example, "5:1" in "The red team beat the blue team 5:1" is an out-of-vocabulary word. , In order to solve this problem, this application introduces [MASK] to randomly replace some words. In the output of the final sentence recognition model, the features corresponding to [MASK] are taken out, and this feature is used to predict the covered words, and LOSS ₃ is obtained.

6)Loss function

Since LOSS ₁ and LOSS ₃ are values between 0 and 1, and CRF is the accumulation of sequence losses, the difference between the two is too large, so some hyperparameters are needed to adjust, so the loss function of this model is Loss _total = αLoss ₁ +βLoss ₂ +γLoss ₃ , where α, β, and γ are hyperparameters that need to be adjusted, and different adjustments need to be made for different tasks.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is Better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or that contributes to related technologies. The computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk). ), includes several instructions to cause a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of the present application.

Figure 5 is a structural block diagram of an equipment control device according to an embodiment of the present application; as shown in Figure 5, it includes:

The identification module 501 is configured to, when the target audio is obtained, identify the target audio and obtain the sentence to be recognized corresponding to the target audio, where the target audio is used to request control of at least one controlled device;

The input module 502 is configured to input the sentence to be recognized into the target sentence recognition model to obtain the target meaning and the target word. The target sentence recognition model includes a first recognition substructure and a second recognition substructure. The first recognition substructure is In order to identify the ideographic direction of the sentence, the second recognition result is used to identify the word data in the sentence. The target sentence recognition model is obtained by jointly training the first recognition substructure and the second recognition substructure using multiple sample sentence data. A model for identifying the ideographic direction of sentences and word data;

The sending module 503 is configured to send control instructions to the target controlled device according to the target meaning and the target word, wherein at least one controlled device includes the target controlled device.

Embodiments of the present application also provide a storage medium that includes a stored program, wherein any of the above methods is executed when the program is run.

Optionally, in this embodiment, the above-mentioned storage medium may be configured to store program codes for performing the following steps:

S1, when the target audio is obtained, identify the target audio and obtain the sentence to be recognized corresponding to the target audio, where the target audio is used to request control of at least one controlled device;

S2. Input the sentence to be recognized into the target sentence recognition model to obtain the target meaning and the target word. The target sentence recognition model includes a first recognition substructure and a second recognition substructure. The first recognition substructure is used to recognize the meaning of the sentence. Point to, the second recognition result is used to identify the word data in the sentence, and the target sentence recognition model is the ideographic meaning of the sentence obtained by jointly training the first recognition substructure and the second recognition substructure using multiple sample sentence data. Models for pointing and word data;

S3: Send a control instruction to the target controlled device according to the target meaning and the target word, where at least one controlled device includes the target controlled device.

An embodiment of the present application also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

Optionally, the above-mentioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.

Optionally, in this embodiment, the above-mentioned processor may be configured to perform the following steps through a computer program:

Optionally, in this embodiment, the above storage medium may include but is not limited to: U disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), Various media that can store program code, such as mobile hard drives, magnetic disks, or optical disks.

Optionally, for specific examples in this embodiment, reference can be made to the examples described in the above-mentioned embodiments and optional implementations, and details will not be described again in this embodiment.

Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented using general-purpose computing devices, and they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices. , optionally, they may be implemented in program code executable by a computing device, such that they may be stored in a storage device for execution by the computing device, and in some cases, may be in a sequence different from that herein. The steps shown or described are performed either individually as individual integrated circuit modules, or as multiple modules or steps among them as a single integrated circuit module. As such, the application is not limited to any specific combination of hardware and software.

The above are only the preferred embodiments of the present application. It should be pointed out that for those of ordinary skill in the art, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications can also be made. should be regarded as the scope of protection of this application.

Industrial applicability

When the target audio is obtained, identify the target audio and obtain the sentence to be recognized corresponding to the target audio, where the target audio is used to request control of at least one controlled device; input the sentence to be recognized into the target sentence recognition model to obtain the target ideogram and the target word, wherein the target sentence recognition model includes a first recognition substructure and a second recognition substructure. The first recognition substructure is used to identify the ideographic direction of the sentence, and the second recognition result is used to identify the word data in the sentence. The target The sentence recognition model is a model obtained by jointly training the first recognition substructure and the second recognition substructure using multiple sample sentence data, and is used to recognize the ideographic direction and word data of the sentence; according to the target ideographic meaning and the target word, the target recipient is The control device sends a control instruction, wherein at least one controlled device includes a target controlled device. Using the above technical solution, two substructures are set up in the target sentence recognition model to identify the ideographic direction and word data respectively, so that one model can be used to identify the intention and slot of the sentence at the same time, thus reducing the pressure on the server and solving the problem. Technical issues that put a lot of pressure on the server due to the need for two models to recognize voice control commands.

Claims

A device control method including:

When the target audio is obtained, identify the target audio and obtain the sentence to be recognized corresponding to the target audio, wherein the target audio is used to request control of at least one controlled device;

The sentence to be recognized is input into the target sentence recognition model to obtain the target meaning and the target word, wherein the target sentence recognition model includes a first recognition substructure and a second recognition substructure, and the first recognition substructure is used to Identify the ideographic direction of the sentence, the second recognition result is used to identify word data in the sentence, and the target sentence recognition model uses multiple sample sentence data to compare the first recognition substructure and the second recognition substructure. A model obtained through joint training for identifying the ideographic orientation of sentences and word data;

Send a control instruction to a target controlled device according to the target meaning and the target word, wherein the at least one controlled device includes the target controlled device.
The method according to claim 1, wherein sending a control instruction to a target controlled device according to the target meaning and the target word includes:

The control instruction and instruction sending information are obtained according to the target meaning and the target word, wherein the instruction sending information is used to indicate sending the control instruction to the target subject in the at least one controlled device. control equipment.
The method according to claim 2, wherein said obtaining the control instruction and instruction sending information according to the target ideogram and the target word includes:

The execution authority of the control instruction is determined according to the target meaning; and the instruction sending information and the instruction content of the control instruction are determined according to the target word.
The method according to claim 1, wherein before obtaining the target audio, comprising:

Obtain multiple sample statement data;

Mark the sentence data in each of the sample sentence data to obtain the plurality of marked sample sentence data, wherein each marked sample sentence data includes a marked ideographic identifier and a word identifier, and the ideographic identifier The identifier is used to mark the ideographic direction of the sentence data, and the word identifier is used to mark at least one word data in the sentence data;

Determine the current sample sentence data from the marked plurality of sample sentence data, and determine an initial sentence recognition model, wherein the initial sentence recognition model includes a first recognition substructure and a second recognition substructure, and the third recognition substructure An identification substructure is used to identify the ideographic direction of the sentence data, and the second recognition result is used to identify word data in the sentence data;

The current sample sentence data is input into the first recognition substructure and the second recognition substructure respectively, and the first recognition result output by the first recognition substructure and the first recognition result output by the second recognition substructure are obtained. second identification result;

Determine the first training loss value of the first recognition substructure according to the first recognition result and the ideographic identification of the current sample sentence data; and, according to the second recognition result and the current sample sentence data Word identification, determining the second training loss value of the second recognition substructure;

According to the first training loss value and the second training loss value, a current training loss value is obtained, wherein the current training loss value is used to determine the training status of the initial sentence recognition model;

When the current training loss value does not reach the recognition convergence condition, obtain the next sample sentence data from the marked plurality of sample sentence data and input it into the initial sentence recognition model;

When the current training loss value reaches the convergence condition for identifying, the initial sentence recognition model is determined to be the target sentence recognition model.
The method according to claim 4, wherein obtaining the current training loss value according to the first training loss value and the second training loss value includes:

The first training loss value and the second training loss value are integrated according to the first target weight value and the second target weight value respectively to obtain the current training loss value, wherein the first target weight value is The weight value of the first training loss value, and the second target weight value is the weight value of the second training loss value.
The method of claim 4, further comprising:

When the word identifier of the current sample sentence data includes a target hidden identifier, the target hidden identifier is determined based on the third recognition result output by the second recognition substructure and the word identifier of the current sample sentence data. The third training loss value;

The first training loss value, the second training loss value and the third training loss value are integrated according to the first target weight value, the second target weight value and the third target weight value respectively to obtain the current Training loss value, wherein the first target weight value is the weight value of the first training loss value, the second target weight value is the weight value of the second training loss value, and the third target weight value The value is the weight value of the third training loss value.
The method according to claim 4, wherein when the current training loss value does not reach the recognition convergence condition, the next sample sentence data is obtained from the marked plurality of sample sentence data and input into the Initial sentence recognition model, including:

When the current training loss value does not reach the recognition convergence condition, adjust the parameters of the first recognition substructure and the second recognition substructure to obtain an adjusted initial sentence recognition model;

The next sample sentence data is obtained from the marked plurality of sample sentence data and input into the adjusted initial sentence recognition model.
An equipment control device including:

An identification module, configured to, when the target audio is obtained, identify the target audio and obtain the sentence to be recognized corresponding to the target audio, wherein the target audio is used to request control of at least one controlled device;

The input module is configured to input the sentence to be recognized into a target sentence recognition model to obtain the target meaning and the target word, wherein the target sentence recognition model includes a first recognition substructure and a second recognition substructure, and the third recognition substructure An identification substructure is used to identify the ideographic direction of a sentence, the second identification result is used to identify word data in the sentence, and the target sentence identification model uses multiple sample sentence data to identify the first identification substructure and the A model for identifying the ideographic direction of sentences and word data obtained through joint training of the second recognition substructure;

The sending module is configured to send a control instruction to a target controlled device according to the target meaning and the target word, wherein the at least one controlled device includes the target controlled device.
A computer-readable storage medium includes a stored program, wherein the method of any one of claims 1 to 7 is executed when the program is run.
An electronic device includes a memory and a processor, a computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.