CN108520746B

CN108520746B - Method and device for controlling intelligent equipment through voice and storage medium

Info

Publication number: CN108520746B
Application number: CN201810239465.0A
Authority: CN
Inventors: 陈维扬; 常洋
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2018-03-22
Filing date: 2018-03-22
Publication date: 2022-04-01
Anticipated expiration: 2038-03-22
Also published as: CN108520746A

Abstract

The disclosure relates to a method and a device for controlling intelligent equipment by voice and a storage medium, and belongs to the field of intelligent home furnishing. The method comprises the following steps: when the awakening words aiming at the control terminal are detected, voice information is collected, the target intelligent equipment needing to be controlled at present is determined according to the collected voice information, and control information is sent to the target intelligent equipment. That is, in the present disclosure, a wakeup word is set in advance for the control terminal, and when the control terminal detects the wakeup word, the control terminal collects voice information, then determines the target intelligent device according to the voice information, and sends a control message to the target intelligent device, so as to implement voice control on the target intelligent device. Therefore, in the present disclosure, voice control over the smart devices in the smart home system can be achieved without the smart devices in the smart home system having a microphone acquisition function, and flexibility of controlling the smart devices in the smart home system by voice is improved.

Description

Method and device for controlling intelligent equipment through voice and storage medium

Technical Field

The present disclosure relates to the field of smart home, and in particular, to a method and an apparatus for controlling a smart device by voice, and a storage medium.

Background

At present, an intelligent home system generally includes various intelligent devices, such as an intelligent television, an intelligent refrigerator, an intelligent air conditioner, and the like, and a user can control the intelligent devices through a control terminal. Because of the convenience of voice control, intelligent devices are increasingly favored by users through voice control.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides a method, an apparatus, and a storage medium for voice-controlling an intelligent device.

According to a first aspect of the embodiments of the present disclosure, a method for controlling an intelligent device by voice is provided, which is applied to a control terminal in an intelligent home system, and the method includes:

when a wake-up word aiming at the control terminal is detected, voice information is collected;

determining target intelligent equipment which needs to be controlled currently according to the acquired voice information, and establishing network connection with the target intelligent equipment, wherein the target intelligent equipment is any intelligent equipment in a plurality of intelligent equipment included in the intelligent home system;

and sending a control message to the target intelligent equipment through the network connection, and executing corresponding operation by the target intelligent equipment according to the control message.

Optionally, the determining, according to the collected voice information, a target intelligent device that needs to be controlled currently, and establishing a network connection with the target intelligent device includes:

sending the voice information to a server, and performing text recognition on the voice information by the server to obtain a recognition result, wherein the recognition result comprises the equipment identification of the target intelligent equipment;

when an identification result sent by the server is received, establishing network connection with the target intelligent equipment according to the equipment identification of the target intelligent equipment included in the identification result;

accordingly, the sending a control message to the target smart device over the network connection includes:

and sending a control message carrying the identification result to the target intelligent equipment through the network connection.

Optionally, after establishing a network connection with the target intelligent device according to the device identifier of the target intelligent device included in the recognition result, the method further includes:

if the network connection is failed to be established, determining the Bluetooth physical address of the target intelligent device according to the corresponding relation between the stored Bluetooth physical address and the device identification of the target intelligent device;

broadcasting a wake-up message, wherein the wake-up message carries the Bluetooth physical address of the target intelligent device;

and when receiving a wakeup confirmation message returned by the target intelligent equipment according to the wakeup message, re-executing the step of establishing network connection with the target intelligent equipment according to the equipment identification of the target intelligent equipment, which is included in the identification result.

Optionally, before determining the bluetooth physical address of the target smart device according to the stored correspondence between the bluetooth physical address and the device identifier of the target smart device, the method further includes:

broadcasting a query message through a UPnP (Universal Plug and Play) protocol, wherein the query message is used for querying an intelligent device supporting the UPnP protocol;

receiving a response message sent by each intelligent device in at least one intelligent device, wherein the response message is used for indicating that the corresponding intelligent device supports the UPnP Protocol, and the response message carries a device identifier and an IP (Internet Protocol) address of the corresponding intelligent device;

determining function description information of each intelligent device through the IP address of each intelligent device, wherein the function description information comprises whether the corresponding intelligent device supports a voice function and the Bluetooth physical address of the corresponding intelligent device;

and determining the intelligent equipment needing voice control from the at least one intelligent equipment according to the function description information of each intelligent equipment, and storing the corresponding relation between the determined Bluetooth physical address of the intelligent equipment and the equipment identification.

Optionally, the establishing a network connection with the target intelligent device according to the device identifier of the target intelligent device included in the recognition result includes:

determining the IP address of the target intelligent equipment according to the corresponding relation between the stored IP address and the equipment identification of the target intelligent equipment;

and establishing network connection with the target intelligent equipment according to the IP address of the target intelligent equipment.

Optionally, before determining the IP address of the target intelligent device according to the correspondence between the stored IP address and the device identifier of the target intelligent device, the method further includes:

broadcasting a query message through a UPnP protocol, wherein the query message is used for querying intelligent equipment supporting the UPnP protocol;

receiving a response message sent by each intelligent device in at least one intelligent device, wherein the response message is used for indicating that the corresponding intelligent device supports the UPnP protocol and carries a device identifier and an IP address of the corresponding intelligent device;

determining function description information of each intelligent device through the IP address of each intelligent device, wherein the function description information comprises whether the corresponding intelligent device supports a voice function or not;

and determining the intelligent equipment needing voice control from the at least one intelligent equipment according to the function description information of each intelligent equipment, and storing the corresponding relation between the IP address of the determined intelligent equipment and the equipment identification.

Optionally, the determining, according to the function description information of each intelligent device, an intelligent device that needs to be controlled by voice from the at least one intelligent device includes:

according to the function description information of each intelligent device, the intelligent device supporting the voice function in the at least one intelligent device is determined;

when at least two intelligent devices with the same device type exist in the determined intelligent devices, dividing at least one group of intelligent devices from the determined intelligent devices according to the device types of the determined intelligent devices, wherein each group of intelligent devices comprises at least two intelligent devices with the same device type;

and selecting one intelligent device from each group of intelligent devices, and determining the selected intelligent device and intelligent devices which are not divided from the determined intelligent devices as the intelligent devices needing voice control.

Optionally, the control terminal is a microphone array deployed in the smart home system.

According to a second aspect of the embodiments of the present disclosure, there is provided a method for controlling a smart device by voice, which is applied to a target smart device, where the target smart device is any one of a plurality of smart devices included in a smart home system, and the method includes:

receiving a control message sent by a control terminal in the intelligent home system through network connection;

the network connection is established by the control terminal according to the collected voice information after the voice information is collected, and the voice information is collected by the control terminal after the control terminal detects a wake-up word aiming at the control terminal;

and executing corresponding operation according to the control message.

Optionally, the control message carries an identification result, and the identification result is obtained by performing text identification on the voice message by the server;

the executing corresponding operation according to the control message comprises:

determining state information of the intelligent device in a preset time period which is before the current time and is closest to the current time, wherein the state information comprises operation or displayed content executed by the target intelligent device in the preset time period which is before the current time and is closest to the current time;

sending the determined state information and the recognition result to the server, and correcting the recognition result by the server according to the state information to obtain a semantic result;

and receiving the semantic result sent by the server, and executing the operation corresponding to the semantic result.

According to a third aspect of the embodiments of the present disclosure, there is provided an apparatus for controlling an intelligent device by voice, which is applied to a control terminal in an intelligent home system, the apparatus including:

the acquisition module is used for acquiring voice information when the awakening words aiming at the control terminal are detected;

the system comprises an establishing module, a control module and a control module, wherein the establishing module is used for determining target intelligent equipment needing to be controlled currently according to acquired voice information and establishing network connection with the target intelligent equipment, and the target intelligent equipment is any intelligent equipment in a plurality of intelligent equipment included in the intelligent home system;

and the sending module is used for sending a control message to the target intelligent equipment through the network connection, and the target intelligent equipment executes corresponding operation according to the control message.

Optionally, the establishing module is specifically configured to:

correspondingly, the sending module is specifically configured to:

Optionally, the establishing module is further specifically configured to:

According to a fourth aspect of the embodiments of the present disclosure, there is provided an apparatus for controlling an intelligent device by voice, which is applied to a target intelligent device, where the target intelligent device is any one of a plurality of intelligent devices included in an intelligent home system, and the apparatus includes:

the receiving module is used for receiving a control message sent by a control terminal in the intelligent home system through network connection;

and the execution module is used for executing corresponding operation according to the control message.

the execution module is specifically configured to:

According to a fifth aspect of the embodiments of the present disclosure, there is provided an apparatus for controlling an intelligent device by voice, which is applied to a control terminal in an intelligent home system, the apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of any of the methods of the first aspect described above.

According to a sixth aspect of the embodiments of the present disclosure, there is provided an apparatus for controlling an intelligent device by voice, which is applied to a target intelligent device, where the target intelligent device is any one of a plurality of intelligent devices included in an intelligent home system, and the apparatus includes:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of any of the methods of the second aspect described above.

According to a seventh aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of any one of the methods of the first aspect.

According to an eighth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of any one of the methods of the second aspect.

According to a ninth aspect of embodiments of the present disclosure, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the method of voice controlling a smart device according to the first aspect described above.

According to a tenth aspect of embodiments of the present disclosure, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the method of voice controlling a smart device according to the second aspect described above.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

in the embodiment of the disclosure, when the control terminal detects a wake-up word for itself, the control terminal collects voice information, then determines a target intelligent device according to the collected voice information, establishes a network connection with the target intelligent device, and sends a control message to the target intelligent device, so as to implement voice control on the target intelligent device. Therefore, according to the method for controlling the intelligent device by voice provided by the embodiment of the disclosure, the intelligent device in the intelligent home system can be controlled by voice without the intelligent device in the intelligent home system having a microphone acquisition function, and the flexibility of controlling the intelligent device in the intelligent home system by voice is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a schematic diagram of an intelligent home system provided in an embodiment of the present disclosure;

fig. 2 is a flowchart of a method for controlling an intelligent device through voice according to an embodiment of the present disclosure;

FIG. 3 is a flow chart of another method for controlling an intelligent device by voice according to an embodiment of the present disclosure;

FIG. 4 is a flow chart of another method for controlling an intelligent device by voice according to an embodiment of the present disclosure;

fig. 5 is a block diagram of an apparatus for controlling an intelligent device through voice according to an embodiment of the present disclosure;

FIG. 6 is a block diagram of an apparatus for controlling an intelligent device by voice according to another embodiment of the present disclosure;

fig. 7 is a block diagram of an apparatus for controlling an intelligent device through voice according to another embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

Before explaining the embodiments of the present disclosure in detail, an application scenario of the embodiments of the present disclosure will be described.

With the development of the smart home systems, more and more smart devices are included in the smart home systems, which leads to higher and higher requirements of users on flexibility of controlling the smart devices, and therefore, how to control a plurality of smart devices in the smart home systems has become a current research hotspot. The method for controlling the intelligent device through the voice provided by the embodiment of the disclosure is applied to a scene of controlling the intelligent device in the intelligent home system.

In the embodiment of the disclosure, when a wake-up word for a control terminal is detected, voice information is collected, a target intelligent device which needs to be controlled currently is determined according to the collected voice information, a network connection with the target intelligent device is established, then a control message is sent to the target intelligent device through the network connection, and the target intelligent device executes corresponding operation according to the control message. That is, in the embodiment of the present disclosure, a wakeup word is set in advance for the control terminal, when the control terminal detects the wakeup word, the control terminal collects voice information, then determines the target intelligent device according to the collected voice information, and sends a control message to the target intelligent device, so as to implement voice control on the target intelligent device. Therefore, according to the method for controlling the intelligent device by voice provided by the embodiment of the disclosure, the intelligent device in the intelligent home system can be controlled by voice without the intelligent device in the intelligent home system having a microphone acquisition function, and the flexibility of controlling the intelligent device in the intelligent home system by voice is improved.

Fig. 1 is a schematic diagram of an intelligent home system provided by an embodiment of the present disclosure, and as shown in fig. 1, the system 100 includes a plurality of intelligent devices 101, a control terminal 102, and a server 103, where each intelligent device 101 and the control terminal 102 are connected in a wireless or wired manner for communication. Each intelligent device 101 is connected with the server 103 in a wireless or wired manner for communication, and the control terminal 102 is also connected with the server in a wireless or wired manner for communication.

The control terminal 102 is configured to send an instruction to the intelligent device 101 according to a requirement of a user, and the intelligent device 101 is configured to receive the instruction sent by the control terminal 102 and execute a corresponding operation according to the instruction sent by the control terminal 102. The control terminal 102 or the intelligent device 101 can also perform information interaction with the server 103.

The intelligent device 101 may be a smart television, a smart refrigerator, a smart air conditioner, a smart lighting lamp, and the like, and the control terminal may be a desktop computer, a tablet computer, a mobile phone, or a smart watch, and the like.

Fig. 2 is a flowchart of a method for controlling an intelligent device by voice according to an embodiment of the present disclosure, and is applied to a control terminal in an intelligent home system, as shown in fig. 2, the method includes the following steps:

in step 201, when a wake-up word for the control terminal is detected, voice information is collected.

In step 202, a target smart device that needs to be controlled currently is determined according to the collected voice information, and a network connection with the target smart device is established, where the target smart device is any one of a plurality of smart devices included in the smart home system.

In step 203, a control message is sent to the target smart device through the network connection, and the target smart device performs a corresponding operation according to the control message.

Optionally, determining a target intelligent device currently required to be controlled according to the collected voice information, and establishing a network connection with the target intelligent device, including:

when receiving the recognition result sent by the server, establishing network connection with the target intelligent equipment according to the equipment identification of the target intelligent equipment included in the recognition result;

accordingly, the sending of the control message to the target smart device over the network connection includes:

if the network connection is failed to be established, determining the Bluetooth physical address of the target intelligent device according to the corresponding relation between the stored Bluetooth physical address and the device identifier of the target intelligent device;

and when receiving a wake-up confirmation message returned by the target intelligent device according to the wake-up message, re-executing the step of establishing network connection with the target intelligent device according to the device identification of the target intelligent device included in the identification result.

receiving a response message sent by each intelligent device in at least one intelligent device, wherein the response message is used for indicating that the corresponding intelligent device supports the UPnP protocol and carries the device identification and the IP address of the corresponding intelligent device;

Optionally, establishing a network connection with the target intelligent device according to the device identifier of the target intelligent device included in the recognition result includes:

determining the IP address of the target intelligent device according to the corresponding relation between the stored IP address and the device identification of the target intelligent device;

Optionally, before determining the IP address of the target intelligent device according to the stored correspondence between the IP address and the device identifier of the target intelligent device, the method further includes:

Optionally, determining, from the at least one smart device, a smart device that needs to be controlled by voice according to the function description information of each smart device includes:

when at least two intelligent devices with the same device type exist in the determined intelligent devices, at least one group of intelligent devices are divided from the determined intelligent devices according to the device type of the determined intelligent devices, wherein each group of intelligent devices comprises at least two intelligent devices with the same device type;

All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present disclosure, and the embodiments of the present disclosure are not described in detail again.

Fig. 3 is a flowchart of a method for controlling an intelligent device by voice according to an embodiment of the present disclosure, which is applied to a target intelligent device, where the target intelligent device is any one of a plurality of intelligent devices included in an intelligent home system, and as shown in fig. 3, the method includes the following steps:

in step 301, a control message sent by a control terminal in the smart home system through a network connection is received, where the network connection is established by the control terminal according to collected voice information after the control terminal collects the voice information, and the voice information is collected by the control terminal after detecting a wakeup word for the control terminal.

In step 302, corresponding operations are performed according to the control message.

In the embodiment of the disclosure, when the control terminal detects the wake-up word for itself, the control terminal collects the voice information, then establishes a network connection with the target intelligent device according to the collected voice information, and sends a control message to the target intelligent device, so as to implement voice control on the target intelligent device. Therefore, according to the method for controlling the intelligent device by voice provided by the embodiment of the disclosure, the intelligent device in the intelligent home system can be controlled by voice without the intelligent device in the intelligent home system having a microphone acquisition function, and the flexibility of controlling the intelligent device in the intelligent home system by voice is improved.

executing corresponding operations according to the control message, including:

determining state information of the target intelligent device in a preset time period which is before the current time and is closest to the current time, wherein the state information comprises operation or display content executed by the target intelligent device in the preset time period which is before the current time and is closest to the current time;

Fig. 4 is a flowchart of a method for controlling an intelligent device by voice according to an embodiment of the present disclosure, and is applied to the intelligent home system shown in fig. 1, as shown in fig. 4, the method includes the following steps:

in step 401, when the control terminal detects a wakeup word for itself, voice information is collected.

In the embodiment of the present disclosure, a corresponding wake-up word is set in advance for a control terminal of an intelligent home system, where the wake-up word is used to indicate that a user is about to perform voice control on a certain intelligent device in the intelligent home system. That is, for the user in the smart home system, the user only needs to remember the awakening word corresponding to the control terminal to be connected, when the user needs to control a certain smart device, the voice including the awakening word corresponding to the control terminal can be sent out, so that the control terminal collects the voice sent by the user when detecting the awakening word, the smart device which needs to be controlled by the user is controlled according to the collected voice information, and the flexibility of the voice control smart device is further improved.

In addition, in order to facilitate the control terminal to accurately collect the voice sent by the user, in the embodiment of the disclosure, the control terminal may be a microphone array, the microphone array includes a plurality of microphones and a microphone control center, and the plurality of microphones are distributed in each place of the home space corresponding to the smart home system.

In this case, the implementation manner of step 401 may be: the microphones are always in a state of collecting voice information and send the voice information collected in real time to the microphone control center, and the microphone control center identifies the voice information collected by the microphones and determines whether the voice information collected by the microphones is awakened aiming at the control terminal in advance. When the microphone control center determines that the voice information acquired by the microphones contains the awakening word preset for the control terminal, the microphone control center can determine the current direction of the user according to the voice information corresponding to the awakening word acquired by each microphone, so as to acquire the voice information according to the current direction of the user, and thus the strength of the voice information acquired in the determined direction is enhanced.

The microphone control center determines the current direction of the user according to the voice information corresponding to the awakening word acquired by each microphone, that is, the microphone control center determines the signal strength of the voice information corresponding to the awakening word acquired by each microphone, and determines the current direction of the user according to the difference between the signal strengths of the voice information acquired by each microphone.

For example, the awakening word preset for the control terminal is a sound box, when the user sends a voice "sound box", the control terminal detects the awakening word and starts to collect voice information, that is, collects the voice continuously sent by the user after the user sends the voice "sound box", if the voice continuously sent by the user after the user sends the voice "sound box" is "i want to see the video of liudelhi", the voice information collected by the control terminal is "i want to see the video of liudelhi".

In step 402, the control terminal determines a target smart device that needs to be controlled currently according to the collected voice information, and establishes a network connection with the target smart device, where the target smart device is any one of a plurality of smart devices included in the smart home system.

The implementation manner of step 402 is: and the control terminal sends the voice information to a server, the server performs text recognition on the voice information to obtain a recognition result, and the recognition result is sent to the control terminal, wherein the recognition result comprises the equipment identification of the target intelligent equipment. And when the control terminal receives the identification result sent by the server, establishing network connection with the target intelligent equipment according to the equipment identification of the target intelligent equipment included in the identification result.

That is, in the embodiment of the present disclosure, when the control terminal acquires the voice information after detecting the wakeup word, the control terminal sends the voice information to the server, and the server identifies the voice information to determine the target intelligent device that the user currently needs to control, and sends the identification result to the control terminal. And when the control terminal receives the recognition result, determining the target intelligent equipment which needs to be controlled currently by the user according to the recognition result, and establishing network connection with the target intelligent equipment so as to realize voice control on the target intelligent equipment through the network connection.

The device identifier of the target intelligent device is used for uniquely identifying the target intelligent device, and is the name of the target intelligent device.

For example, the voice information acquired by the control terminal is "i want to watch videos of liu de hua", and when the server receives the voice information sent by the control terminal, the server identifies the voice information and determines that the user needs to watch videos at present, that is, determines that the target intelligent device is a television. After determining the target intelligent device, the server sends a recognition result to the control terminal, wherein the recognition result comprises two parts of contents, one part is a text aiming at the voice information of "i want to watch videos of Liudebua", the other part is a device identifier of the target intelligent device, and the device identifier of the target intelligent device can be a television.

When the control terminal receives the recognition result sent by the server, the target intelligent device needing to be controlled at present is determined to be the television according to the device identification of the target intelligent device included in the recognition result, and at the moment, the control terminal can establish network connection with the television so as to carry out voice control on the television through the network connection.

The implementation manner of establishing the network connection between the control terminal and the target intelligent device may be as follows: and determining the IP address of the target intelligent device according to the corresponding relation between the stored IP address and the device identification of the target intelligent device, and establishing network connection with the target intelligent device according to the IP address of the target intelligent device.

It should be noted that the control terminal stores therein a correspondence between the IP address and the device identifier, which is predetermined and stored by the control terminal. The implementation manner of determining and storing the corresponding relationship by the control terminal may be: and broadcasting a query message through the UPnP protocol, wherein the query message is used for querying the intelligent equipment supporting the UPnP protocol. And when the intelligent equipment in the intelligent home system receives the query message, sending a response message to the control terminal, wherein the response message is used for indicating that the corresponding intelligent equipment supports the UPnP protocol, and the response message carries the equipment identifier and the IP address of the corresponding intelligent equipment. The control terminal receives a response message sent by each intelligent device in at least one intelligent device, determines function description information of each intelligent device according to the IP address of each intelligent device, wherein the function description information comprises whether the corresponding intelligent device supports a voice function, then determines the intelligent device needing voice control from the at least one intelligent device according to the function description information of each intelligent device, and stores the corresponding relation between the IP address of the determined intelligent device and the device identification.

The implementation mode of determining the function description information of each intelligent device through the IP address of each intelligent device is as follows: and accessing an HTTP (Hypertext Transfer Protocol) service interface of each intelligent device through the IP address of each intelligent device to acquire the function description information of each intelligent device.

In addition, the implementation mode of determining the intelligent device needing voice control from the at least one intelligent device according to the function description information of each intelligent device is as follows: the method comprises the steps of determining intelligent equipment supporting a voice function in at least one intelligent equipment according to function description information of each intelligent equipment, when at least two intelligent equipment with the same equipment type exist in the determined intelligent equipment, dividing at least one group of intelligent equipment from the determined intelligent equipment according to the equipment type of the determined intelligent equipment, wherein each group of intelligent equipment comprises at least two intelligent equipment with the same equipment type, selecting one intelligent equipment from each group of intelligent equipment, and determining the selected intelligent equipment and intelligent equipment which is not divided in the determined intelligent equipment as intelligent equipment needing voice control.

The implementation mode of selecting one intelligent device from each group of intelligent devices is as follows: and the control terminal sends a selection message to a terminal currently used by the user, wherein the selection message carries the equipment identifier of each intelligent equipment in at least two intelligent equipment included in each group of intelligent equipment. When the terminal currently used by the user receives the selection message, each intelligent device in at least two intelligent devices included in each group of intelligent devices is displayed according to the selection message, options for each intelligent device are displayed, when the terminal currently used by the user detects the selection operation of the options for a certain intelligent device, the intelligent device is determined as the intelligent device selected by the user, and the intelligent device selected by the user is sent to the control terminal, so that the control terminal determines the intelligent device as the selected intelligent device.

The terminal currently used by the user can be a mobile phone, a tablet computer, a desktop computer or an intelligent watch and the like. The selection operation may be a click operation, a slide operation, a voice operation, or the like.

In addition, it should be noted that the control terminal establishes a network connection between the IP address of the target intelligent device and the target intelligent device, and the premise that the network connection between the control terminal and the target intelligent device is established according to the IP address of the target intelligent device is that the target intelligent device is in an awake state. Therefore, if the target smart device is not currently in the awake state, the network connection with the target smart device cannot be established through the IP address of the target smart device. When the target smart device is not in the wake-up state, the target smart device may send and receive information through the bluetooth physical address, and therefore, after the control terminal establishes a network connection between the target smart devices corresponding to the target wake-up word, there may be the following operations:

if the network connection is failed to be established, determining the Bluetooth physical address of the target intelligent device according to the corresponding relation between the stored Bluetooth physical address and the device identifier of the target intelligent device; broadcasting a wake-up message, wherein the wake-up message carries the Bluetooth physical address of the target intelligent device; and when a wake-up confirmation message returned by the target intelligent device according to the wake-up message is received, re-executing the step of establishing the network connection between the target intelligent devices corresponding to the target wake-up word.

That is, after the control terminal establishes the network connection with the target intelligent device through the IP address of the target intelligent device, if the network connection is failed to be established, it indicates that the target intelligent device may not be in the wake-up state currently, at this time, the control terminal may wake up the target intelligent device through the bluetooth physical address of the target intelligent device, and after waking up the target intelligent device, establish the network connection with the target intelligent device again according to the IP address of the target intelligent device.

The control terminal determines the bluetooth physical address of the target intelligent device according to the stored correspondence between the bluetooth physical address and the device identifier of the target intelligent device, that is, the correspondence between the bluetooth physical address and the device identifier is stored in the control terminal in advance. The implementation manner of predetermining and storing the corresponding relationship between the bluetooth physical address and the device identifier in the control terminal may be as follows: the control terminal broadcasts an inquiry message through a UPnP protocol, wherein the inquiry message is used for inquiring the intelligent equipment supporting the UPnP protocol; receiving a response message sent by each intelligent device in at least one intelligent device, wherein the response message is used for indicating that the corresponding intelligent device supports the UPnP protocol and carries the device identification and the IP address of the corresponding intelligent device; determining function description information of each intelligent device through the IP address of each intelligent device, wherein the function description information comprises whether the corresponding intelligent device supports a voice function and the Bluetooth physical address of the corresponding intelligent device; and determining the intelligent equipment needing voice control from the at least one intelligent equipment according to the function description information of each intelligent equipment, and storing the corresponding relation between the determined Bluetooth physical address of the intelligent equipment and the equipment identification.

The implementation manner of determining, by the control terminal, the intelligent device to be voice-controlled from the at least one intelligent device according to the function description information of each intelligent device is introduced in determining the correspondence between the IP address and the device identifier, and is not elaborated herein.

It should be noted that, the determining and storing of the correspondence between the bluetooth physical address and the device identifier by the control terminal may be performed after the network connection is failed to be established, or may be performed before the control terminal establishes the network connection with the target intelligent device.

Optionally, in this embodiment of the present disclosure, a correspondence between the device identifier, the IP address, and the bluetooth physical address of the intelligent device that needs to be subjected to voice control may be directly determined in advance, so that the correspondence may be directly obtained from the stored correspondence between the device identifier, the IP address, and the bluetooth physical address when the IP address or the bluetooth physical address is subsequently needed.

That is, before the control terminal establishes a network connection with the target intelligent device, the control terminal broadcasts an inquiry message through the UPnP protocol, where the inquiry message is used to inquire the intelligent device supporting the UPnP protocol; receiving a response message sent by each intelligent device in at least one intelligent device, wherein the response message is used for indicating that the corresponding intelligent device supports the UPnP protocol and carries the device identification and the IP address of the corresponding intelligent device; determining function description information of each intelligent device through the IP address of each intelligent device, wherein the function description information comprises whether the corresponding intelligent device supports a voice function and the Bluetooth physical address of the corresponding intelligent device; and determining the intelligent equipment needing voice control from the at least one intelligent equipment according to the function description information of each intelligent equipment, and storing the corresponding relation among the IP address, the Bluetooth physical address and the equipment identification of the determined intelligent equipment.

In step 403, the control terminal sends a control message to the target smart device through the network connection.

Step 402 shows that the control terminal determines the target intelligent device that the user currently needs to control according to the recognition result sent by the server, and establishes a network connection with the target intelligent device, and the recognition result further includes a text for the voice information collected by the control terminal, so that the control terminal can directly send a control message carrying the recognition result to the target intelligent device through the network connection.

The Control message is obtained by encoding the recognition result, and the Control terminal sends the Control message to the target smart device through a Transmission Control Protocol (TCP).

Table 1 below is a schematic structural diagram of a control message provided in an embodiment of the present disclosure.

TABLE 1

As shown in table 1, the control message is composed of at least 9 bytes (byte), and the first byte [0] is used to indicate a front-end type, which is used to indicate the current type of the control terminal, for example, when the control terminal is a microphone array, the front-end type may indicate that the control terminal is a microphone array. The second byte [1] is used to indicate the type of hardware currently employed by the control terminal, for example, when the control terminal is a microphone array, the hardware type refers to the type of microphone in the microphone array. The third byte [2] is used to indicate the front-end software version, which refers to the version of the corresponding application on the control terminal. The fourth byte [4] to the eighth byte [7] are bytes in a reserved state. The bytes after the ninth byte [8] to the ninth byte are obtained by encoding the recognition result according to a Payload (Payload) encoding technique.

In step 404, the target smart device receives a control message sent by the control terminal over the network connection.

After the control terminal sends the control message carrying the identification result through step 403, the target smart device will accordingly receive the control message carrying the identification result.

In step 405, the target smart device performs a corresponding operation according to the control message.

In the embodiment of the present disclosure, if an operation that the target smart device needs to perform is determined directly from the voice information, the operation is likely not to be an operation that the user wants, and thus, the operation that the target smart device needs to perform may be determined from the context state of the target smart device and the voice information.

Thus, the implementation of step 405 may be: determining state information of the target intelligent device in a preset time period which is before the current time and is closest to the current time, wherein the state information comprises operation or display content executed by the target intelligent device in the preset time period which is before the current time and is closest to the current time; sending the determined state information and the recognition result to the server, and correcting the recognition result by the server according to the state information to obtain a semantic result; and receiving the semantic result sent by the server, and executing the operation corresponding to the semantic result.

The preset time period is a preset time period, and the preset time period may be 1s, 2s, 3s, or the like.

That is, in the embodiment of the present disclosure, the server may correct the recognition result corresponding to the voice information according to the state information of the target smart device to obtain a semantic result, so that the target smart device executes an operation corresponding to the semantic result. The operation executed by the target intelligent equipment is more in line with the requirement of the user.

For example, the identification result carried in the control message is "i want to watch the video of liu de hua", and if the state information of the target intelligent device is "movie page", the server may correct the identification result "i want to watch the video of liu de hua" to obtain a semantic result "the movie of liu de hua", and then perform an operation corresponding to the semantic result at this time, that is, perform an operation of "searching for the movie of liu de hua".

Fig. 5 is a block diagram of an apparatus for controlling an intelligent device by using voice according to an embodiment of the present disclosure, and is applied to a control terminal in an intelligent home system, as shown in fig. 5, the apparatus 500 includes an acquisition module 501, a setup module 502, and a sending module 503:

an acquisition module 501, configured to acquire voice information when a wakeup word for the control terminal is detected;

an establishing module 502, configured to determine a target intelligent device that needs to be controlled currently according to the acquired voice information, and establish a network connection with the target intelligent device, where the target intelligent device is any one of a plurality of intelligent devices included in the intelligent home system;

a sending module 503, configured to send a control message to the target smart device through the network connection, where the target smart device executes a corresponding operation according to the control message.

Optionally, the establishing module 502 is specifically configured to:

accordingly, the sending module 503 is specifically configured to:

Optionally, the establishing module is further specifically configured to:

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 6 is a block diagram of an apparatus for controlling an intelligent device by voice, which is provided in an embodiment of the present disclosure and is applied to a target intelligent device, where the target intelligent device is any one of a plurality of intelligent devices included in an intelligent home system, and as shown in fig. 6, the apparatus 600 includes a receiving module 601 and an executing module 602:

a receiving module 601, configured to receive a control message sent by a control terminal in the smart home system through network connection;

the network connection is established by the control terminal according to the collected voice information after the voice information is collected, and the voice information is collected by the control terminal after the control terminal detects the awakening word aiming at the control terminal;

an executing module 602, configured to execute a corresponding operation according to the control message.

the execution module is specifically configured to:

Fig. 7 is a block diagram of an apparatus for controlling an intelligent device through voice according to an embodiment of the present disclosure. The control terminal or smart device of fig. 1 may be implemented by the apparatus shown in fig. 7, for example, the apparatus 700 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, an exercise device, etc.

Referring to fig. 7, apparatus 700 may include one or more of the following components: a processing component 702, a memory 704, a power component 706, a multimedia component 708, an audio component 710, an input/output (I/O) interface 712, a sensor component 714, and a communication component 716.

The processing component 702 generally controls overall operation of the device 700, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 702 may include one or more processors 720 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 702 may include one or more modules that facilitate interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate interaction between the multimedia component 708 and the processing component 702.

The memory 704 is configured to store various types of data to support operations at the apparatus 700. Examples of such data include instructions for any application or method operating on device 700, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 704 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 706 provides power to the various components of the device 700. The power components 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power supplies for the apparatus 700.

The multimedia component 708 includes a screen that provides an output interface between the device 700 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 700 is in an operation mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 710 is configured to output and/or input audio signals. For example, audio component 710 includes a Microphone (MIC) configured to receive external audio signals when apparatus 700 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 704 or transmitted via the communication component 716. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

The I/O interface 712 provides an interface between the processing component 702 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 714 includes one or more sensors for providing status assessment of various aspects of the apparatus 700. For example, sensor assembly 714 may detect an open/closed state of device 700, the relative positioning of components, such as a display and keypad of device 700, sensor assembly 714 may also detect a change in position of device 700 or a component of device 700, the presence or absence of user contact with device 700, orientation or acceleration/deceleration of device 700, and a change in temperature of device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 716 is configured to facilitate wired or wireless communication between the apparatus 700 and other devices. The apparatus 700 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer readable storage medium comprising instructions, such as the memory 704 comprising instructions, executable by the processor 720 of the device 700 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

A non-transitory computer readable storage medium, wherein instructions of the storage medium, when executed by a processor of a terminal, enable the terminal to perform a method of voice controlling a smart device provided by the above embodiments.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A method for controlling intelligent equipment by voice is applied to a control terminal in an intelligent home system, the control terminal is a microphone array deployed in the intelligent home system, the microphone array comprises a plurality of microphones and a microphone control center, the microphones are distributed in all places of a home space corresponding to the intelligent home system and are used for acquiring voice information in real time, and the microphone control center is used for identifying the voice information acquired by the microphones in real time, and the method is characterized by comprising the following steps:

when the microphone control center determines that the voice information acquired by the microphones in real time contains a wake-up word aiming at the control terminal, determining the signal intensity of the voice information corresponding to the wake-up word acquired by each microphone of the microphones, and determining the current direction of the user according to the difference value between the signal intensities of the voice information acquired by each microphone so as to acquire the voice information according to the current direction of the user;

sending the voice information to a server, and performing text recognition on the voice information by the server to obtain a recognition result, wherein the recognition result comprises an equipment identifier of the target intelligent equipment; when an identification result sent by the server is received, establishing network connection with a target intelligent device according to a device identifier of the target intelligent device included in the identification result, wherein the target intelligent device is any one of a plurality of intelligent devices included in the intelligent home system;

sending a control message carrying the identification result to the target intelligent equipment through the network connection, and executing corresponding operation by the target intelligent equipment according to the control message;

if the network connection is failed to be established, the target intelligent device is not in an awakening state currently, and the Bluetooth physical address of the target intelligent device is determined according to the corresponding relation between the stored Bluetooth physical address and the device identification of the target intelligent device; broadcasting a wake-up message, wherein the wake-up message carries the Bluetooth physical address of the target intelligent device; when receiving a wakeup confirmation message returned by the target intelligent device according to the wakeup message, re-executing the step of establishing network connection with the target intelligent device according to the device identification of the target intelligent device included in the identification result;

2. The method of claim 1, wherein before determining the bluetooth physical address of the target smart device according to the stored correspondence between the bluetooth physical address and the device identifier of the target smart device, the method further comprises:

broadcasting a query message through a universal plug and play (UPnP) protocol, wherein the query message is used for querying intelligent equipment supporting the UPnP protocol;

receiving a response message sent by each intelligent device in at least one intelligent device, wherein the response message is used for indicating that the corresponding intelligent device supports the UPnP protocol and carries a device identifier and an Internet Protocol (IP) address of the corresponding intelligent device;

3. The method according to claim 1, wherein the establishing a network connection with the target smart device according to the device identifier of the target smart device included in the recognition result comprises:

4. The method of claim 3, wherein before determining the IP address of the target intelligent device according to the stored correspondence between the IP address and the device identifier of the target intelligent device, the method further comprises:

5. The method according to claim 2 or 4, wherein the determining the intelligent device needing voice control from the at least one intelligent device according to the function description information of each intelligent device comprises:

6. A method for controlling intelligent equipment by voice is applied to target intelligent equipment, wherein the target intelligent equipment is any intelligent equipment in a plurality of intelligent equipment included in an intelligent home system, and the method comprises the following steps:

receiving a control message sent by a control terminal in the intelligent home system through network connection, wherein the control terminal is a microphone array deployed in the intelligent home system, the microphone array comprises a plurality of microphones and a microphone control center, the plurality of microphones are distributed in each place of a home space corresponding to the intelligent home system and collect voice information in real time, the microphone control center is used for identifying the voice information collected by the plurality of microphones in real time, the control message carries an identification result, the identification result is obtained by a server through text identification on the voice information, and the identification result comprises an equipment identifier of the target intelligent equipment;

the network connection is established by the control terminal according to the collected voice information after the voice information is collected by the control terminal, the voice information is collected by the microphones according to the current direction of the user, and the current direction of the user is determined by the microphone control center according to the difference between the signal strengths of the voice information corresponding to the awakening words collected by each microphone in the microphones when the awakening words aiming at the control terminal are determined in the voice information collected by the microphones in real time;

executing corresponding operation according to the control message;

the control terminal is used for establishing network connection with the target intelligent equipment according to the equipment identification of the target intelligent equipment, which is included in the identification result, when the identification result sent by the server is received;

the control terminal is further configured to determine a bluetooth physical address of the target intelligent device according to a correspondence between a stored bluetooth physical address and a device identifier and the device identifier of the target intelligent device if the network connection is failed to be established, which indicates that the target intelligent device is not currently in an awake state; broadcasting a wake-up message, wherein the wake-up message carries the Bluetooth physical address of the target intelligent device; when receiving a wakeup confirmation message returned by the target intelligent device according to the wakeup message, re-executing the step of establishing network connection with the target intelligent device according to the device identification of the target intelligent device included in the identification result;

7. The utility model provides a device of speech control smart machine, is applied to the control terminal among the intelligent home systems, control terminal for dispose in microphone array among the intelligent home systems, microphone array includes a plurality of microphones and microphone control center, a plurality of microphones distribute in each place of the home space that intelligent home systems corresponds to gather speech information in real time, microphone control center is used for right the speech information of a plurality of microphones real-time collection discerns, its characterized in that, the device includes:

the acquisition module is used for determining the signal intensity of the voice information corresponding to the awakening words acquired by each microphone in the microphones when the microphone control center determines that the voice information acquired by the microphones in real time contains the awakening words aiming at the control terminal, and determining the current direction of the user according to the difference value between the signal intensities of the voice information acquired by each microphone so as to acquire the voice information according to the current direction of the user;

a sending module, configured to send a control message to the target intelligent device through the network connection, where the target intelligent device executes a corresponding operation according to the control message;

the establishing module is specifically configured to:

correspondingly, the sending module is specifically configured to:

sending a control message carrying the identification result to the target intelligent device through the network connection;

the establishing module is further specifically configured to:

if the network connection is failed to be established, the target intelligent device is not in an awakening state currently, and the Bluetooth physical address of the target intelligent device is determined according to the corresponding relation between the stored Bluetooth physical address and the device identification of the target intelligent device;

when receiving a wakeup confirmation message returned by the target intelligent device according to the wakeup message, re-executing the step of establishing network connection with the target intelligent device according to the device identification of the target intelligent device included in the identification result;

8. The apparatus of claim 7, wherein the establishing module is further specifically configured to:

9. The apparatus of claim 7, wherein the establishing module is further specifically configured to:

10. The apparatus of claim 9, wherein the establishing module is further specifically configured to:

11. The apparatus according to claim 8 or 10, wherein the establishing module is further specifically configured to:

12. The utility model provides a device of speech control smart machine, is applied to target smart machine, arbitrary smart machine in a plurality of smart machines that target smart machine includes for intelligent home systems, its characterized in that, the device includes:

the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a control message sent by a control terminal in the intelligent home system through network connection, the control terminal is a microphone array deployed in the intelligent home system, the microphone array comprises a plurality of microphones and a microphone control center, the microphones are distributed in all places of a home space corresponding to the intelligent home system and collect voice information in real time, the microphone control center is used for identifying the voice information collected by the microphones in real time, the control message carries an identification result, the identification result is obtained by a server through text identification on the voice information, and the identification result comprises an equipment identifier of a target intelligent equipment;

the execution module is used for executing corresponding operation according to the control message;

the execution module is specifically configured to:

13. The utility model provides a device of speech control smart machine, is applied to the control terminal among the intelligent home systems, its characterized in that, the device includes:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of the method of any one of claims 1-5.

14. The utility model provides a device of speech control smart machine, is applied to target smart machine, arbitrary smart machine in a plurality of smart machines that target smart machine includes for intelligent home systems, its characterized in that, the device includes:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of the method of claim 6.

15. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of the method of any of claims 1-5.

16. A computer readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of the method of claim 6.