CN109817211B - Electric appliance control method and device, storage medium and electric appliance - Google Patents
Electric appliance control method and device, storage medium and electric appliance Download PDFInfo
- Publication number
- CN109817211B CN109817211B CN201910114386.1A CN201910114386A CN109817211B CN 109817211 B CN109817211 B CN 109817211B CN 201910114386 A CN201910114386 A CN 201910114386A CN 109817211 B CN109817211 B CN 109817211B
- Authority
- CN
- China
- Prior art keywords
- user
- image
- voice
- electric appliance
- control command
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides an electric appliance control method, an electric appliance control device, a storage medium and an electric appliance, wherein the method comprises the following steps: when a preset voice awakening command is received, acquiring a first image of a user in the environment where the electric appliance is located when the voice awakening command is sent; identifying the first image to identify a target user who sends the preset voice awakening command; recognizing a voice control command sent by the target user; and controlling the electric appliance to execute corresponding operation according to the voice control command. The scheme provided by the invention can combine image recognition and voice recognition, and improves the accuracy of voice recognition.
Description
Technical Field
The invention relates to the field of control, in particular to an electric appliance control method, an electric appliance control device, a storage medium and an electric appliance.
Background
As voice and image technologies mature, more and more electric appliances are used for voice and image. For example, an intelligent air conditioner with voice or image or an intelligent air conditioner with voice and image. At present, when only a single user issues a voice command to the air conditioner, the air conditioner can quickly recognize and correspondingly make a more accurate action. However, when the users get up, the intelligent air conditioner is not satisfactory in receiving and recognizing the voice commands of the users. For example, a group of people meet in a room, many people speak at the same time, and a user wants to operate an air conditioner through voice, and at this time, the accuracy of voice recognition may be reduced by simply relying on techniques such as echo cancellation, and the like, which may possibly cause a phenomenon that recognition cannot be performed or recognition errors occur.
Disclosure of Invention
The main purpose of the present invention is to overcome the above-mentioned defects in the prior art, and to provide an electrical appliance control method, device, storage medium and electrical appliance, so as to solve the problem in the prior art that the accuracy of speech recognition will be reduced when performing speech control on an electrical appliance in a scenario where multiple users speak simultaneously.
The invention provides an electric appliance control method on one hand, which comprises the following steps: when a preset voice awakening command is received, acquiring a first image of a user in the environment where the electric appliance is located when the voice awakening command is sent; identifying the first image to identify a target user who sends the preset voice awakening command; recognizing a voice control command sent by the target user; and controlling the electric appliance to execute corresponding operation according to the voice control command.
Optionally, the method further comprises: when the preset voice awakening command is received, judging whether a plurality of users exist in the environment where the electric appliance is located; and under the condition that a plurality of users exist, acquiring a first image of the user in the environment where the electric appliance is located when the voice wake-up command is sent out.
Optionally, recognizing the first image to recognize a target user who issues the preset voice wake-up command includes: identifying whether each user in the first image speaks or not by using a preset image identification model; and determining the identified talking user in the first image as the target user.
Optionally, recognizing whether each user in the first image is speaking by using a preset image recognition model, including: recognizing the lip state of each user in the first image by using a preset image recognition model so as to determine whether each user in the first image is speaking; wherein the lip state comprises: whether the lips are closed and/or whether the lips are moving; when the lip states of any user within the preset time are identified to be closed states, the user is identified as not speaking; and/or when the lip states of any user within the preset time are recognized to be motion states, recognizing that the user is speaking.
Optionally, recognizing the voice control command issued by the target user includes: acquiring a second image of the target user and the voice in the current environment; identifying the second image to identify the start point and the stop point of the speaking of the target user; and extracting a voice control command sent by the target user according to the identified start point and the stop point.
Optionally, controlling the electrical appliance to perform corresponding operations according to the voice control command, including: carrying out voice recognition on the voice control command so as to recognize an electric appliance control command corresponding to the voice control command; and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command.
Another aspect of the present invention provides an electric appliance control apparatus, including: the device comprises an acquisition unit, a display unit and a control unit, wherein the acquisition unit is used for acquiring a first image of a user in the environment where the electric appliance is located when a preset voice awakening command is received; the user identification unit is used for identifying the first image so as to identify a target user sending the preset voice awakening command; the command recognition unit is used for recognizing the voice control command sent by the target user; and the control unit is used for controlling the electric appliance to execute corresponding operation according to the voice control command.
Optionally, the method further comprises: the judging unit is used for judging whether a plurality of users exist in the environment where the electric appliance is located or not when the preset voice awakening command is received; the acquisition unit acquires a first image of a user in the environment where the electric appliance is located when the voice wake-up command is sent out under the condition that a plurality of users exist.
Optionally, the user identification unit includes: the identification subunit is used for identifying whether each user in the first image speaks or not by using a preset image identification model; and the determining subunit is used for determining the identified speaking user in the first image as the target user.
Optionally, the identifying subunit identifies, by using a preset image identification model, whether each user in the first image is speaking, including: recognizing the lip state of each user in the first image by using a preset image recognition model so as to determine whether each user in the first image is speaking; wherein the lip state comprises: whether the lips are closed and/or whether the lips are moving; when the lip states of any user within the preset time are identified to be closed states, the user is identified as not speaking; and/or when the lip states of any user within the preset time are recognized to be motion states, recognizing that the user is speaking.
Optionally, the command recognition unit includes: the image acquisition subunit is used for acquiring a second image of the target user and the voice in the current environment; the image identification subunit is used for identifying the second image so as to identify the start point and the stop point of the speaking of the target user; and the command extraction subunit is used for extracting the voice control command sent by the target user according to the identified start point and the identified stop point.
Optionally, the controlling unit controls the electrical appliance to perform corresponding operations according to the voice control command, including: carrying out voice recognition on the voice control command so as to recognize an electric appliance control command corresponding to the voice control command; and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command.
A further aspect of the invention provides a storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of any of the methods described above.
Yet another aspect of the present invention provides an appliance comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any of the methods described above when executing the program.
In another aspect, the invention provides an electrical appliance comprising the electrical appliance control device described in any one of the above.
According to the technical scheme of the invention, the image of the user in the environment where the electric appliance is located when the voice wake-up command is sent is obtained, the target user of the voice wake-up command is sent out through image recognition, so that the voice control command sent by the target user is recognized, the electric appliance is controlled according to the voice control command, the image recognition and the voice recognition are combined, the accuracy of the voice recognition is improved, especially in a multi-user scene, the user sending the voice control command in a plurality of users can be recognized, the user experience is improved, the voice control command is extracted through the image recognition of the user who is carrying out the voice control and the starting and stopping point of the speaking of the user, and the accuracy of the voice recognition is further improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 is a method diagram of an embodiment of an appliance control method provided by the present invention;
FIG. 2 is a flowchart illustrating an embodiment of the steps of recognizing the first image to identify a target user who issues the preset voice wake-up command;
FIG. 3 is a flowchart diagram of one embodiment of the step of identifying a voice control command issued by the target user, in accordance with an embodiment of the present invention;
FIG. 4 is a method diagram of another embodiment of an appliance control method provided by the present invention;
FIG. 5 is a schematic structural diagram of an embodiment of an appliance control device provided in the present invention;
FIG. 6 is a block diagram illustrating an embodiment of a subscriber identity unit, according to an embodiment of the present invention;
FIG. 7 is a block diagram illustrating an exemplary implementation of a command recognition unit according to an embodiment of the invention;
fig. 8 is a schematic structural diagram of another embodiment of the electrical appliance control device provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the specific embodiments of the present invention and the accompanying drawings. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Fig. 1 is a schematic method diagram of an embodiment of an appliance control method provided by the present invention.
As shown in fig. 1, according to an embodiment of the present invention, the appliance control method includes at least step S110, step S120, step S130, and step S140.
Step S110, when a preset voice awakening command is received, acquiring a first image of a user in the environment where the electric appliance is located when the voice awakening command is sent.
Specifically, a first image of a user in an environment where the electrical appliance is located may be acquired by a camera on the electrical appliance. For example, the electric appliance is an intelligent air conditioner, a camera of the intelligent air conditioner can be always in an open state, and user images in the environment where the intelligent air conditioner is located are collected in real time. When a preset voice awakening command is received, a first image of a user in the environment where the electric appliance is located when the voice awakening command is sent is acquired, so that the user of the voice awakening command can be identified through the first image.
Step S120, the first image is recognized to recognize a target user who sends the preset voice awakening command.
Fig. 2 is a flowchart illustrating a specific implementation of the step of recognizing the first image to identify the target user who issues the preset voice wake-up command according to an embodiment of the present invention. As shown in fig. 2, in a specific embodiment, step S120 includes step S121 and step S122.
Step S121, identifying whether the user in the first image is speaking by using a preset image identification model.
Step S122, determining the speaking user in the identified first image as the target user.
Specifically, the lip states of the users in the image are identified by using a preset image identification model so as to determine whether the users in the image are speaking. The image recognition model may be, for example, a pre-trained convolutional neural network model, and the lip states include: whether the lips are closed and/or whether the lips are moving; when the lip states of any user within the preset time are identified to be closed states, the user is identified as not speaking; and/or recognizing that the user is speaking when recognizing that the lip states of any user within the preset time are motion states. That is, the lips of a user are closed for a period of time, i.e., the user is not speaking at that time; a user's lips are in motion for a period of time, e.g., regular opening and closing of the lips), the user is considered speaking at that time. And if a certain user is recognized to speak when the voice awakening command is sent out, determining the recognized speaking user as a target user, namely the user sending the voice awakening command.
Step S130, recognizing the voice control command sent by the target user.
FIG. 3 is a flowchart illustrating an embodiment of the step of recognizing the voice control command issued by the target user according to the present invention. As shown in fig. 3, in one embodiment, step S130 includes step S131, step S132, and step S133.
Step S131, collecting a second image of the target user and the voice in the current environment.
Step S132, recognizing the second image to recognize a start point and a stop point of the target user speaking.
And step S133, extracting a voice control command sent by the target user according to the identified start point and the identified stop point.
Specifically, after a target user who starts a voice wake-up command is identified, an image (a second image) of the target user and a voice in the current environment are collected, the collected image is identified to identify a start point and a stop point of the speaking of the target user, the user generally makes a subconscious pause when the user finishes speaking a sentence, a phenomenon detected on the face of the user is that the lips are closed, and in a period of time, in the collected second image (preferably, the collected facial image of the target user), the lips of the user are in a closed state, that is, the user is considered not speaking at the moment. In a period of time, in the second image of the target user, collected lips of the user are regularly opened and closed, the user is considered to be speaking at the moment, the user state in the previous period of time is a speaking state, after a period of time, the lips of the user are not opened and closed, that is, the user is considered to finish speaking, and according to the identified start and stop points, the voice in the environment collected between the start and stop points is extracted (for example, according to the time period between the start and stop points, the voice in the environment of the time period is extracted), that is, the voice control command sent by the target user is obtained. Optionally, a front end point (i.e., a start point) and a rear end point (i.e., an end point) of the collected speech in the environment may be identified by combining with a speech end point detection technique, so as to extract an effective speech segment (i.e., a speech control command of the target user) in the collected speech in the environment, thereby improving accuracy of sentence break identification.
And step S140, controlling the electric appliance to execute corresponding operation according to the voice control command.
Specifically, voice recognition is carried out on the voice control command so as to recognize an electric appliance control command corresponding to the voice control command; and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command. More specifically, voice recognition is carried out on the voice control command so as to recognize the semantic meaning of the voice control command, and the electric appliance control command corresponding to the voice control command is determined according to the recognized semantic meaning. If the corresponding electric appliance control command can be determined according to the recognized semantics, the voice control command is an effective voice control command, and the electric appliance can be controlled to execute corresponding operation according to the recognized electric appliance control command.
In an actual application scenario, when a plurality of users are in a room and the users may chat together, at this time, the users want to control the air conditioner by voice, and need to send a voice wake-up command to the intelligent air conditioner first, when the intelligent air conditioner receives the voice wake-up command of the users, the images of the environment where the voice wake-up command is sent are acquired, the users speaking in the images are identified, the voice control command of the users is identified, and when the voice control command is identified to be an effective voice control command, the intelligent air conditioner is controlled to execute corresponding operation.
Fig. 4 is a schematic method diagram of another embodiment of the appliance control method provided by the present invention. As shown in fig. 4, according to another embodiment of the present invention, the appliance control method further includes a step S102.
And step S102, judging whether a plurality of users exist in the environment where the electric appliance is located or not when the preset voice awakening command is received.
In case there are multiple users, a first image of the user in the environment where the appliance was located at the time the voice wake-up command was issued is acquired (step S110).
In one embodiment, the determination of whether there are multiple users is made by capturing sound in the environment in which the appliance is located. For example, when the collected sound in the environment where the electric appliance is located is subjected to voiceprint recognition and more than two different voiceprints are recognized, it is determined that a plurality of users are in the environment where the electric appliance is located. In another embodiment, an image of the environment where the electrical appliance is located is acquired, and whether a plurality of users are in the environment where the electrical appliance is located is identified through the image. And under the condition that a plurality of users exist, acquiring a first image of the user in the environment where the electric appliance is located when the voice wake-up command is sent out.
Fig. 5 is a schematic structural diagram of an embodiment of an electrical appliance control device provided in the present invention. As shown in fig. 5, the appliance control device 100 includes an acquisition unit 110, a user recognition unit 120, a command recognition unit 130, and a control unit 140.
The acquiring unit 110 is configured to, when a preset voice wake-up command is received, acquire a first image of a user in an environment where the electrical appliance is located when the voice wake-up command is sent; the user identification unit 120 is configured to identify the first image to identify a target user who issues the preset voice wake-up command; the command recognition unit 130 is used for recognizing a voice control command issued by the target user; the control unit 140 is used for controlling the electric appliance to execute corresponding operations according to the voice control command.
When a preset voice wake-up command is received, the obtaining unit 110 obtains a first image of a user in an environment where the electrical appliance is located when the voice wake-up command is sent.
Specifically, the obtaining unit 110 may collect a first image of a user in an environment where the electrical appliance is located through a camera on the electrical appliance. For example, the electric appliance is an intelligent air conditioner, a camera of the intelligent air conditioner can be always in an open state, and user images in the environment where the intelligent air conditioner is located are collected in real time. When the smart air conditioner receives a preset voice wake-up command, the obtaining unit 110 obtains a first image of a user in an environment where the electrical appliance is located when the voice wake-up command is sent, so that the user who sends the voice wake-up command is identified through the first image.
The user recognition unit 120 recognizes the first image to recognize a target user who issues the preset voice wake-up command.
Fig. 6 is a schematic structural diagram of a specific implementation of a subscriber identity unit according to an embodiment of the present invention. As shown in fig. 6, in a specific embodiment, the user identifying unit 120 includes an identifying subunit 121 and a determining subunit 122.
The identifying subunit 121 is configured to identify whether each user in the first image is speaking by using a preset image identification model. The determining subunit 122 is configured to determine the identified speaking user in the first image as the target user.
Specifically, the identifying subunit 121 identifies the lip state of each user in the image by using a preset image identification model to determine whether each user in the image is speaking. The image recognition model may be, for example, a pre-trained convolutional neural network model, and the lip states include: whether the lips are closed and/or whether the lips are moving; when the lip states of any user within the preset time are identified to be closed states, the user is identified as not speaking; and/or recognizing that the user is speaking when recognizing that the lip states of any user within the preset time are motion states. That is, the lips of a user are closed for a period of time, i.e., the user is not speaking at that time; a user's lips are in motion for a period of time, e.g., regular opening and closing of the lips), the user is considered speaking at that time. If the identifying subunit 121 identifies that a certain user is speaking at the time of issuing the voice wakeup command, the determining subunit 122 determines the identified speaking user as the target user, i.e., the user issuing the voice wakeup command.
The command recognition unit 130 recognizes a voice control command issued by the target user.
Fig. 7 is a schematic structural diagram of an embodiment of a command recognition unit according to an embodiment of the present invention. As shown in fig. 7, in one embodiment, the command recognition unit 130 includes an image acquisition subunit 131, an image recognition subunit 132, and a command extraction subunit 133.
The image acquisition subunit 131 is configured to acquire a second image of the target user and a voice in the current environment; the image recognition subunit 132 is configured to recognize the second image collected by the image collection subunit 131, so as to recognize a start point and a stop point of the speaking of the target user; the command extracting subunit 133 is configured to extract a voice control command issued by the target user according to the start point and the end point identified by the image identifying subunit 132.
Specifically, after the user identification unit 120 identifies the target user who has sent the voice wakeup command, the image acquisition subunit 131 acquires an image (a second image) of the target user and the voice in the current environment, the image identification subunit 132 identifies the image acquired by the image acquisition subunit 131, identifies the start point and the end point of the speaking of the target user, when the user has spoken a sentence, the user usually makes an informed pause, the phenomenon detected on the face of the user is that the lips are closed, and in a period of time, the lips of the user are all in a closed state in the acquired second image (preferably, the image of the face of the target user is acquired), that is, the user is not speaking at this time. In a period of time, in the second image of the target user collected, if the lips of the user are regularly opened and closed, it is considered that the user is speaking at this time, the user state in the previous period of time is a speaking state, and after a period of time, the lips of the user are not opened and closed, that is, it is considered that the user is speaking finished, the command extracting subunit 133 extracts the voice in the environment collected between the start point and the stop point according to the start point and the stop point identified by the image identifying subunit 132 (for example, the voice in the environment of the period of time is extracted according to the period of time between the start point and the stop point), that is, the voice control command sent by the target. Optionally, the command extracting sub-unit 133 may further identify a front end point (i.e., a start point) and a rear end point (i.e., an end point) of the collected voice in the environment in combination with the voice end point detecting technology, so as to extract a valid voice segment (i.e., the voice control command of the target user) in the collected voice in the environment, thereby improving accuracy of sentence segmentation recognition.
The control unit 140 controls the electrical appliance to perform corresponding operations according to the voice control command.
Specifically, the control unit 140 performs voice recognition on the voice control command to recognize an electrical appliance control command corresponding to the voice control command; and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command. More specifically, the control unit 140 performs voice recognition on the voice control command to recognize the semantic meaning of the voice control command, and determines the electrical appliance control command corresponding to the voice control command according to the recognized semantic meaning. If the corresponding electrical appliance control command can be determined according to the recognized semantic meaning, and the voice control command is an effective voice control command, the control unit 140 may control the electrical appliance to execute a corresponding operation according to the recognized electrical appliance control command.
Fig. 8 is a schematic structural diagram of another embodiment of the electrical appliance control device provided by the present invention. As shown in fig. 8, according to another embodiment of the present invention, the appliance control device further includes a determination unit 102.
The judging unit 102 is configured to judge whether there are multiple users in an environment where the electrical appliance is located when the preset voice wake-up command is received; the obtaining unit 110 obtains a first image of a user in an environment where the electric appliance is located when the voice wake-up command is issued, in case that there are a plurality of users.
In one embodiment, the determining unit 102 may determine whether there are multiple users by collecting sounds in the environment where the appliance is located. For example, when the collected sound in the environment where the electric appliance is located is subjected to voiceprint recognition and more than two different voiceprints are recognized, it is determined that a plurality of users are in the environment where the electric appliance is located. In another embodiment, capturing images of the environment in which the appliance is located identifies whether there are multiple users in the environment in which the appliance is located from the captured images. In the case where the judging unit 102 determines that there are a plurality of users, the acquiring unit 110 acquires a first image of the user in the environment where the electric appliance is located at the time when the voice wake-up command is issued.
The invention also provides a storage medium corresponding to the appliance control method, on which a computer program is stored, which program, when executed by a processor, carries out the steps of any of the methods described above.
The invention also provides an electric appliance corresponding to the electric appliance control method, which comprises a processor, a memory and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of any one of the methods.
The invention also provides an electric appliance corresponding to the electric appliance control device, which comprises any one of the electric appliance control devices.
Therefore, according to the scheme provided by the invention, the image of the user in the environment where the electric appliance is located when the voice wake-up command is sent out is obtained, the target user of the voice wake-up command is sent out through image recognition, so that the voice control command sent out by the target user is recognized, the image recognition and the voice recognition are combined, the accuracy of the voice recognition is improved, especially in a multi-user scene, the user sending the voice control command out of a plurality of users can be recognized, the user experience is improved, the voice control command is extracted through the image recognition of the user who is carrying out the voice control and the starting and stopping point of the speaking of the user, and the accuracy of the voice recognition is further improved.
The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Other examples and implementations are within the scope and spirit of the invention and the following claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hardwired, or a combination of any of these. In addition, each functional unit may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and the parts serving as the control device may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The above description is only an example of the present invention, and is not intended to limit the present invention, and it is obvious to those skilled in the art that various modifications and variations can be made in the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (13)
1. An appliance control method, comprising:
when a preset voice awakening command is received, acquiring a first image of a user in the environment where the electric appliance is located when the voice awakening command is sent;
identifying the first image to identify a target user who sends the preset voice awakening command;
recognizing a voice control command sent by the target user;
controlling the electric appliance to execute corresponding operation according to the voice control command;
identifying the first image to identify a target user who sends the preset voice awakening command, comprising:
identifying whether each user in the first image speaks or not by using a preset image identification model;
determining the identified talking user in the first image as the target user;
further comprising:
when the preset voice awakening command is received, judging whether a plurality of users exist in the environment where the electric appliance is located;
and under the condition that a plurality of users exist, acquiring a first image of the user in the environment where the electric appliance is located when the voice wake-up command is sent out.
2. The method of claim 1, wherein identifying whether each user in the first image is speaking using a preset image recognition model comprises:
recognizing the lip state of each user in the first image by using a preset image recognition model so as to determine whether each user in the first image is speaking; wherein the lip state comprises: whether the lips are closed and/or whether the lips are moving;
when the lip states of any user within the preset time are identified to be closed states, the user is identified as not speaking;
and/or the presence of a gas in the gas,
when the lip states of any user within the preset time are recognized to be action states, the user is recognized to speak.
3. The method of any of claims 1-2, wherein recognizing the voice control command issued by the target user comprises:
acquiring a second image of the target user and the voice in the current environment;
identifying the second image to identify the start point and the stop point of the speaking of the target user;
and extracting a voice control command sent by the target user according to the identified start point and the stop point.
4. The method according to any one of claims 1-2, wherein controlling the appliance to perform a corresponding operation according to the voice control command comprises:
carrying out voice recognition on the voice control command so as to recognize a corresponding electric appliance control command;
and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command.
5. The method of claim 3, wherein controlling the appliance to perform corresponding operations according to the voice control command comprises:
carrying out voice recognition on the voice control command so as to recognize a corresponding electric appliance control command;
and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command.
6. An appliance control device, comprising:
the device comprises an acquisition unit, a display unit and a control unit, wherein the acquisition unit is used for acquiring a first image of a user in the environment where the electric appliance is located when a preset voice awakening command is received;
the user identification unit is used for identifying the first image so as to identify a target user sending the preset voice awakening command;
the command recognition unit is used for recognizing the voice control command sent by the target user;
the control unit is used for controlling the electric appliance to execute corresponding operation according to the voice control command;
the user identification unit includes:
the identification subunit is used for identifying whether each user in the first image speaks or not by using a preset image identification model;
a determining subunit, configured to determine a speaking user in the identified first image as the target user;
further comprising:
the judging unit is used for judging whether a plurality of users exist in the environment where the electric appliance is located or not when the preset voice awakening command is received;
the acquisition unit acquires a first image of a user in the environment where the electric appliance is located when the voice wake-up command is sent out under the condition that a plurality of users exist.
7. The apparatus of claim 6, wherein the identifying subunit identifies whether each user in the first image is speaking by using a preset image recognition model, and comprises:
recognizing the lip state of each user in the first image by using a preset image recognition model so as to determine whether each user in the first image is speaking; wherein the lip state comprises: whether the lips are closed and/or whether the lips are moving;
when the lip states of any user within the preset time are identified to be closed states, the user is identified as not speaking;
and/or the presence of a gas in the gas,
when the lip states of any user within the preset time are recognized to be action states, the user is recognized to speak.
8. The apparatus according to any one of claims 6-7, wherein the command recognition unit comprises:
the image acquisition subunit is used for acquiring a second image of the target user and the voice in the current environment;
the image identification subunit is used for identifying the second image so as to identify the start point and the stop point of the speaking of the target user;
and the command extraction subunit is used for extracting the voice control command sent by the target user according to the identified start point and the identified stop point.
9. The device according to any one of claims 6-7, wherein the control unit controls the electric appliance to perform corresponding operations according to the voice control command, and comprises:
carrying out voice recognition on the voice control command so as to recognize an electric appliance control command corresponding to the voice control command;
and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command.
10. The apparatus of claim 8, wherein the control unit controls the electric appliance to perform corresponding operations according to the voice control command, and comprises:
carrying out voice recognition on the voice control command so as to recognize an electric appliance control command corresponding to the voice control command;
and controlling the electric appliance to execute corresponding operation according to the identified electric appliance control command.
11. A storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.
12. An electrical appliance comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of any one of claims 1 to 5 when executing the program.
13. An appliance comprising an appliance control device as claimed in any one of claims 6 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910114386.1A CN109817211B (en) | 2019-02-14 | 2019-02-14 | Electric appliance control method and device, storage medium and electric appliance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910114386.1A CN109817211B (en) | 2019-02-14 | 2019-02-14 | Electric appliance control method and device, storage medium and electric appliance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109817211A CN109817211A (en) | 2019-05-28 |
| CN109817211B true CN109817211B (en) | 2021-04-02 |
Family
ID=66606549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910114386.1A Active CN109817211B (en) | 2019-02-14 | 2019-02-14 | Electric appliance control method and device, storage medium and electric appliance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109817211B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110517685B (en) * | 2019-09-25 | 2021-10-08 | 深圳追一科技有限公司 | Voice recognition method and device, electronic equipment and storage medium |
| WO2021114224A1 (en) * | 2019-12-13 | 2021-06-17 | 华为技术有限公司 | Voice detection method, prediction model training method, apparatus, device, and medium |
| CN110910887B (en) * | 2019-12-30 | 2022-06-28 | 思必驰科技股份有限公司 | Voice wake-up method and device |
| CN111294706A (en) * | 2020-01-16 | 2020-06-16 | 珠海格力电器股份有限公司 | Voice electrical appliance control method and device, storage medium and voice electrical appliance |
| CN111276140B (en) * | 2020-01-19 | 2023-05-12 | 珠海格力电器股份有限公司 | Voice command recognition method, device, system and storage medium |
| CN112002316A (en) * | 2020-07-30 | 2020-11-27 | 珠海格力电器股份有限公司 | Electric appliance control method and device, storage medium and terminal |
| CN113450795A (en) * | 2021-06-28 | 2021-09-28 | 深圳七号家园信息技术有限公司 | Image recognition method and system with voice awakening function |
| CN115691485A (en) * | 2021-07-26 | 2023-02-03 | 华为技术有限公司 | Equipment awakening method, related device and communication system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682273A (en) * | 2011-03-18 | 2012-09-19 | 夏普株式会社 | Device and method for detecting lip movement |
| CN104049721A (en) * | 2013-03-11 | 2014-09-17 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN104166831A (en) * | 2013-05-15 | 2014-11-26 | 五邑大学 | ALBP and SRC algorithm-based fatigue detection method and system |
| CN104409075A (en) * | 2014-11-28 | 2015-03-11 | 深圳创维－Rgb电子有限公司 | Voice identification method and system |
| CN106257355A (en) * | 2015-06-18 | 2016-12-28 | 松下电器（美国）知识产权公司 | Apparatus control method and controller |
| CN107346661A (en) * | 2017-06-01 | 2017-11-14 | 李昕 | A kind of distant range iris tracking and acquisition method based on microphone array |
| CN107679506A (en) * | 2017-10-12 | 2018-02-09 | Tcl通力电子（惠州）有限公司 | Awakening method, intelligent artifact and the computer-readable recording medium of intelligent artifact |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9622289B2 (en) * | 2014-06-20 | 2017-04-11 | Apple Inc. | End-to-end delay adaptation with connected DRX in a cellular voice call |
-
2019
- 2019-02-14 CN CN201910114386.1A patent/CN109817211B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682273A (en) * | 2011-03-18 | 2012-09-19 | 夏普株式会社 | Device and method for detecting lip movement |
| CN104049721A (en) * | 2013-03-11 | 2014-09-17 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN104166831A (en) * | 2013-05-15 | 2014-11-26 | 五邑大学 | ALBP and SRC algorithm-based fatigue detection method and system |
| CN104409075A (en) * | 2014-11-28 | 2015-03-11 | 深圳创维－Rgb电子有限公司 | Voice identification method and system |
| CN106257355A (en) * | 2015-06-18 | 2016-12-28 | 松下电器（美国）知识产权公司 | Apparatus control method and controller |
| CN107346661A (en) * | 2017-06-01 | 2017-11-14 | 李昕 | A kind of distant range iris tracking and acquisition method based on microphone array |
| CN107679506A (en) * | 2017-10-12 | 2018-02-09 | Tcl通力电子（惠州）有限公司 | Awakening method, intelligent artifact and the computer-readable recording medium of intelligent artifact |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109817211A (en) | 2019-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109817211B (en) | Electric appliance control method and device, storage medium and electric appliance | |
| US11163356B2 (en) | Device-facing human-computer interaction method and system | |
| US11373646B2 (en) | Household appliance control method, device and system, and intelligent air conditioner by determining user sound source location based on analysis of mouth shape | |
| KR102611751B1 (en) | Augmentation of key phrase user recognition | |
| JP4675840B2 (en) | Remote controller and home appliance | |
| WO2019137066A1 (en) | Electric appliance control method and device | |
| CN105700363A (en) | Method and system for waking up smart home equipment voice control device | |
| CN108231079A (en) | For the method, apparatus, equipment and computer readable storage medium of control electronics | |
| CN109377995B (en) | Method and device for controlling equipment | |
| CN105278380B (en) | The control method and device of smart machine | |
| CN109448705B (en) | Voice segmentation method and device, computer device and readable storage medium | |
| CN105204628A (en) | Voice control method based on visual awakening | |
| CN108227903A (en) | A kind of virtual reality language interactive system and method | |
| CN109979474B (en) | Voice equipment and user speech rate correction method and device thereof and storage medium | |
| CN109373518B (en) | Air conditioner and voice control device and voice control method thereof | |
| KR101644015B1 (en) | Communication interface apparatus and method for multi-user and system | |
| CN112489648A (en) | Wake-up processing threshold adjustment method, voice home appliance, and storage medium | |
| CN111413877A (en) | Method and device for controlling household appliance | |
| CN110024027A (en) | Speaker Identification | |
| CN111341350A (en) | Man-machine interaction control method and system, intelligent robot and storage medium | |
| CN112286364A (en) | Man-machine interaction method and device | |
| CN112002316A (en) | Electric appliance control method and device, storage medium and terminal | |
| CN113299287A (en) | Multi-mode-based service robot interaction method, system and storage medium | |
| CN107452381B (en) | Multimedia voice recognition device and method | |
| CN109343481B (en) | Method and device for controlling device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200409 Address after: 519070 Guangdong city of Zhuhai Province Qianshan Applicant after: GREE ELECTRIC APPLIANCES Inc. OF ZHUHAI Applicant after: GREE GREEN REFRIGERATION TECHNOLOGY CENTER CO., LTD. OF ZHUHAI Address before: Xiangzhou District of Guangdong city in Zhuhai province 519070 Qianshan No. 789 Applicant before: GREE ELECTRIC APPLIANCES Inc. OF ZHUHAI |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |