CN112750437A - Control method, control device and electronic equipment - Google Patents
Control method, control device and electronic equipment Download PDFInfo
- Publication number
- CN112750437A CN112750437A CN202110001669.2A CN202110001669A CN112750437A CN 112750437 A CN112750437 A CN 112750437A CN 202110001669 A CN202110001669 A CN 202110001669A CN 112750437 A CN112750437 A CN 112750437A
- Authority
- CN
- China
- Prior art keywords
- image
- voice
- instruction
- user
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000001514 detection method Methods 0.000 claims abstract description 39
- 241000282414 Homo sapiens Species 0.000 claims abstract description 13
- 238000010801 machine learning Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 241001465382 Physalis alkekengi Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
- H05B47/10—Controlling the light source
- H05B47/105—Controlling the light source in response to determined parameters
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Social Psychology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Acoustics & Sound (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses a control method, a control device and an electronic device, wherein the control method comprises the following steps: dividing a specific space into at least one region; acquiring a frame of image in real time; the images comprise an environment image and a user image which are shot by a three-dimensional image sensor; acquiring an area where a three-dimensional image sensor corresponding to the image is located; intercepting a user image from the image; inputting the user image to a gesture detection model; and judging whether a first switch instruction can be acquired, if so, sending the first switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located. The invention supports the user to directly speak the natural language and is more in line with the human speaking logic. Meanwhile, the invention can accurately adjust the on-off state of the electric equipment according to the real-time posture of the user.
Description
Technical Field
The application relates to the technical field of smart home, in particular to a control method, a control device and an electronic device.
Background
With the continuous progress of science and technology and the gradual improvement of the living standard of people, smart home becomes an essential part in modern family fashion life.
In the prior art, electric equipment (such as intelligent lamps and lanterns) is controlled by online voice, but the method is inconvenient. On-line voice control needs the user to set up through application to the consumer of networking, and user experience is relatively poor, and on-line voice control device must awaken up the consumer through a fixed word of awakening up in addition, sends the instruction again to the consumer, does not conform to user's natural language custom. Because online voice control is based on the internet, when the network is delayed, the electric equipment cannot execute related instructions at the first time, and the user experience is very poor. The prior art also lacks a control technology for accurately controlling the electric equipment.
Therefore, there is a need to provide a new electric device control technology to overcome the above-mentioned drawbacks.
Disclosure of Invention
The invention aims to provide a control method, a control device and a storage medium, which are used for solving the problem that the electric equipment is easy to generate an abnormal instruction phenomenon in the prior art and solving the problem of complicated operation caused by the fact that the electric equipment needs to download an application program in advance.
In order to achieve the above object, the present invention provides a control method, comprising the steps of: dividing a specific space into at least one region; acquiring a frame of image in real time; the images comprise an environment image and a user image which are shot by a three-dimensional image sensor; acquiring an area where a three-dimensional image sensor corresponding to the image is located; intercepting a user image from the image; inputting the user image to a gesture detection model; and judging whether a first switch instruction can be acquired, if so, sending the first switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located.
Further, before the step of acquiring a frame of image in real time, the method comprises the following steps: an electric device, a three-dimensional image sensor and a voice recognition device are arranged in each area.
Further, the step of determining whether a first switch command can be obtained further includes the steps of: if not, intercepting an image of the arm of the user from the image; inputting the user arm image to a gesture detection model; and judging whether a second switch instruction can be acquired, if so, sending the second switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located.
Further, the step of determining whether a second switching command can be obtained further includes the steps of: if not, acquiring a voice instruction; the voice instruction is user voice collected by voice recognition equipment; acquiring the area where the voice recognition equipment collecting voice is located; and judging whether the voice command is consistent with at least one preset voice command, if so, sending a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located.
Further, the step of determining whether a first switch command can be obtained further includes the steps of: if not, acquiring a voice instruction; the voice instruction is user voice collected by voice recognition equipment; acquiring the area where the voice recognition equipment collecting voice is located; and judging whether the voice command is consistent with at least one preset voice command, if so, sending a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located.
Further, the step of determining whether a first switch command can be obtained further includes the steps of: judging whether a first switch instruction can be acquired or not, and if not, intercepting an image of the arm of the user from the image; inputting the user arm image to a gesture detection model; judging whether a second switching instruction can be acquired, if so, sending the second switching instruction to a voice recognition device; collecting a voice instruction; the voice instruction is user voice collected by voice recognition equipment; acquiring the area where the voice recognition equipment collecting voice is located; and judging whether the voice command is consistent with at least one preset voice command, if so, sending a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located.
Further, before the step of acquiring a frame of image in real time, the method comprises the following steps: collecting more than two groups of training samples, wherein each group of training samples comprises a plurality of frames of human body posture images, the training samples of the same group are marked as the same group labels, and the group labels comprise a starting instruction, a stopping instruction and an invalid instruction; inputting a plurality of groups of training samples into a machine learning model for training; and generating a gesture detection model, and judging the instruction type corresponding to the gesture according to the gesture of the user.
Further, before the step of acquiring a frame of image in real time, the method comprises the following steps: collecting more than two groups of training samples, wherein each group of training samples comprises multi-frame gesture images, the training samples in the same group are marked as the same group labels, and the group labels comprise a starting instruction, a shutdown instruction and an invalid instruction; inputting the training sample into a machine learning model for training; and generating a gesture detection model, and judging the instruction type corresponding to the gesture according to the gesture of the user.
Further, before the step of acquiring a frame of image in real time, the method comprises the following steps: and storing the serial numbers and the areas of the electric equipment, the three-dimensional image sensor and the voice recognition equipment to a database.
Further, the step of sending the first switch command to the electric device corresponding to the area where the three-dimensional image sensor is located specifically includes the following steps: acquiring the area of the three-dimensional image sensor according to the number of the three-dimensional image sensor of the acquired image; acquiring the electric equipment of the area according to the area of the three-dimensional image sensor; and sending the first switch instruction to the electric equipment.
Further, the step of acquiring a frame of image in real time specifically includes the following steps: acquiring a frame of image; processing the image with a data filtering algorithm; and judging whether the number of the difference pixel points of the two adjacent processed images is greater than or equal to a preset threshold value, if not, deleting the last acquired frame of image, and returning to the step of acquiring one frame of image.
The present invention also provides a control device comprising a data processing unit, the data processing unit comprising: the area dividing unit is used for dividing a specific space into at least one area; the image acquisition unit is used for acquiring a frame of image in real time; the images comprise an environment image and a user image which are shot by a three-dimensional image sensor; the area acquisition unit is used for acquiring an area where the three-dimensional image sensor corresponding to the image is located; an image intercepting unit configured to intercept a user image from the image; the image input unit is used for inputting the user image to a gesture detection model and judging whether a first switch instruction can be acquired or not; and the instruction sending unit is used for sending the first switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located.
Further, the control device further includes: a speech recognition device; connected to the data processing unit; a communication unit connected to the data processing unit; at least one powered device connected to the data processing unit; and at least one three-dimensional image sensor connected to the data processing unit.
The present invention also provides an electronic device, comprising: a processor; and a memory storing a computer program that, when executed by the processor, performs at least one of the above-described control methods.
The technical effect of the invention is that the invention provides a control method, a control device and an electronic device for an electric device, which solves the problem that voice control requires a wakeup word not conforming to the natural language of human beings in the prior art, and also solves the problem of complicated operation caused by the fact that a user can preset the electric device only by downloading a client. The invention is more in line with the language habit of human beings, and increases the experience of users. The invention does not need to download the application program in advance to carry out the preset setting on the electronic equipment, thereby reducing the complexity of the preset setting step of the electronic equipment. Meanwhile, the control method provided by the invention can realize accurate automatic control of the electric equipment, and reduce the complexity of user operation.
Drawings
The technical solution and other advantages of the present application will become apparent from the detailed description of the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a schematic structural diagram of a control device provided in embodiment 1 of the present application.
Fig. 2 is a schematic structural diagram of a data processing unit in embodiment 1 of the present application.
Fig. 3 is a first flowchart of a control method provided in embodiment 1 of the present application.
Fig. 4 is a second flowchart of the control method provided in embodiment 1 of the present application.
Fig. 5 is a flowchart of the model training described in embodiment 1 of the present application.
Fig. 6 is a flowchart of a step of determining whether a first switch command can be obtained by the data processing unit in embodiment 1 of the present application.
Fig. 7 is a flowchart of a step of determining whether a first switch command can be obtained by the data processing unit in embodiment 2 of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments.
Example 1
As shown in fig. 1, the present embodiment provides a control apparatus 100 for an electric device, which includes a data processing unit 110, wherein the data processing unit 110 is respectively connected to a three-dimensional image sensor 120, an electric device 130, a communication unit 140, and a voice recognition device 150. In this embodiment, the data processing unit 110 is preferably a processor, the communication unit 140 is preferably a bluetooth wireless sensor, the voice recognition device 150 is preferably a microphone, the three-dimensional image sensor 120 is preferably a TOF sensor, and the electric device is preferably a lamp. The electric equipment can be selected from a vehicle interior lighting lamp or a household lamp.
The voice recognition device 150 is used to collect voice commands. The communication unit 140 is configured to send a switch command to the electric device 130, the three-dimensional image sensor 120 is configured to obtain a frame of image in real time, and the electric device 130 is configured to execute the switch command.
As shown in fig. 2, the data processing unit 110 includes: an area dividing unit 111, an image acquiring unit 112, an area acquiring unit 113, an image cutting unit 114, an image entry unit 115, and an instruction transmitting unit 116.
In this embodiment, the first switch command is a switch command of the electric device obtained by an attitude detection model, and the third switch command is a switch command of the electric device obtained by the voice recognition device.
The region dividing unit 111 is used for dividing a specific space into at least one region. An image obtaining unit 112, configured to obtain a frame of image in real time through the three-dimensional image sensor 120; the images comprise an environment image and a user image which are shot by the three-dimensional image sensor. The area acquiring unit 113 is configured to acquire an area where the three-dimensional image sensor corresponding to the image is located. An image interception unit 114 to intercept the user image from the image. The image input unit 115 is configured to input the user image into a gesture detection model, and determine whether a first switch instruction can be obtained. The command sending unit 116 is configured to send the first switch command to the electric device corresponding to the area where the three-dimensional image sensor is located. Alternatively, the body posture may be set to a lying posture or a dining posture in training the posture detection model. For example: the three-dimensional image sensor 120 and the lamp are arranged in the living room, when the user posture in the picture acquired by the data processing unit 110 through the three-dimensional image sensor 120 is consistent with the user posture in the machine learning model, the data processing unit 110 obtains the position of the three-dimensional image sensor, namely the living room, and the data processing unit 110 sends a power on/off command corresponding to the user posture to the lamp in the living room, so that the automatic management of the lamp is realized.
The control device of the electric equipment solves the problem that voice control in the prior art requires that a wakeup word does not conform to the natural language of human beings, and simultaneously solves the problem of complex operation caused by the fact that a user needs to download a client to preset the electric equipment. The control device is more in line with the language habit of human beings, and the experience of the user is improved. The control device does not need to download the application program in advance to carry out the preset device, and reduces the complexity of the preset device steps. Meanwhile, the control method provided by the invention can realize automatic control of the electric equipment, and reduce the complexity of user operation.
As shown in fig. 3 and 4, based on the above-mentioned control apparatus for an electric device, the present embodiment further provides a control method for an electric device, where the control method includes steps S100 to S900. Wherein steps S100-S300 are pre-stage steps.
Step S100) model training. As shown in fig. 5, this step includes steps S110-S160: step S110) the data processor collects more than two sets of training samples. Each group of training samples comprises a plurality of frames of human body posture images, the training samples in the same group are marked as the same group label, and the group label comprises a starting instruction, a stopping instruction and an invalid instruction. Step S120) the data processing unit inputs a plurality of groups of training samples into a machine learning model for training. Step S130) the data processing unit generates a posture detection model. The gesture detection model can judge the instruction type corresponding to the gesture according to the gesture of the user. For example: an image with the sitting posture of the user is marked as being on, an image with the lying posture of the user is marked as being off, and a manufacturer records a large number of the two images into the machine learning model to finally obtain a posture detection model. The data processing unit 110 can obtain a light-on command through the posture detection model when the user is sitting, and can obtain a light-off command through the posture detection model when the user is lying down. The pose detection model is preferably trained from human skeletal point data or human 19 point cloud data.
Step S200) device setup. The method comprises the following steps: the data processing unit divides a specific space into at least one region. The specific area may be a house, a car, preferably a house. The area may be a living room, a bedroom, etc. of the above-mentioned house or a main driving area in the above-mentioned car, etc. The specific region may be divided into at least one region by a manufacturer of the electric device. And a user sets the electric equipment, the three-dimensional image sensor and the voice recognition equipment in each area. For example: the user needs to realize automatic management of the lamps in the living room, namely, the specific space is the house of the user, and the area is the living room. The user needs to locate the lamp, the three-dimensional image sensor and the voice recognition device in the living room. Preferably, the three-dimensional image sensor is arranged between the horizontal height of 0.8m-2m, and the camera of the three-dimensional image sensor is horizontally arranged on the sofa oppositely. The three-dimensional image sensor arranged in this way can identify an area of about 6 square meters at the maximum.
Step S300) database setting. And the user stores the serial numbers of each electric equipment, the three-dimensional image sensor and the voice recognition equipment and the areas where the serial numbers are located in the database. For example: the number of the lamp is 1, the position is a living room, the number of the three-dimensional image sensor for shooting the living room is 2, and the number of the voice recognition device for recording the sound of the living room is 3, namely, a manufacturer records the parameters into a database to establish the connection among the lamp, the three-dimensional image sensor and the voice recognition device, so that the subsequent operation of the corresponding lamp is facilitated.
Step S400) the data processing unit acquires a frame of image in real time. The images comprise an environment image and a user image which are shot by the three-dimensional image sensor. The method specifically comprises the following steps: the data processing unit acquires a frame of image; the data processing unit processes the image by adopting a data filtering algorithm; and the data processing unit judges whether the number of the difference pixel points of the two adjacent processed images is greater than or equal to a preset threshold value, if not, the last acquired frame of image is deleted, the step of acquiring the frame of image is returned, and if yes, the step S500) can be executed). The preset threshold can be accurately set according to the requirements of manufacturers, the smaller the preset threshold is, the more pictures sent to the posture detection model by the data processing unit are, the larger the preset threshold is, the fewer the pictures sent to the posture detection model by the data processing unit are, and the smaller the load of the data processor is. By deleting duplicate pictures, the computational burden on the data processing unit 110 can be reduced, greatly increasing the lifetime of the data processing unit 110. The data filtering algorithm is preferably a kalman filtering algorithm. The image obtained by the data processing unit 110 through the data filtering algorithm can greatly reduce the influence of image noise. The image in this step is preferably an image with point cloud data.
Step S500) the data processing unit acquires the area where the three-dimensional image sensor corresponding to the image is located. The data processing unit 110 queries the area where the current three-dimensional image sensor is located through the database, so as to control the electric equipment in the area subsequently.
Step S600) the data processing unit intercepts a user image from the image. Preferably, the data processing unit 110 compares the pixel difference points of several adjacent frames of images, makes the pixel difference points into a new image, and performs the subsequent steps with the new image, so as to effectively reduce the computation amount of the posture detection model and improve the computation speed of the posture detection model.
Step S700) the data processing unit inputs the user image into a posture detection model. The data processing unit 110 recognizes the posture of the user by the posture detection model described above.
Step S800) the data processing unit judges whether a first switch instruction can be acquired. If yes, the data processing unit executes the next step. If not, the data processing unit 110 executes steps S831-S834, as shown in FIG. 6. Step S831) the data processing unit collects the voice instruction. The voice instruction is user voice collected by voice recognition equipment. Step S832) the data processing unit acquires the area where the voice recognition device that has collected the voice is located. Step S833) the data processing unit determines whether the voice command is consistent with at least one preset voice command, and if so, executes step S834. Step 834) the data processing unit sends a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located. For example: the data processing unit does not recognize that the user is in the living room, at which time the user shouts "light" in the living room, and the data processing unit 110 recognizes that the voice recognition device is located, i.e., the living room. The data processing unit 110 determines that the user is in the living room and needs to turn on the light, and then inquires the number of the lamp corresponding to the living room and sends a light-on command to the lamp.
Step S900) the data processing unit sends a first switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located. The method specifically comprises the following steps: the data processing unit acquires the area of the three-dimensional image sensor according to the number of the three-dimensional image sensor which acquires the image; the data processing unit acquires the electric equipment in the area according to the area where the three-dimensional image sensor is located; and the data processing unit sends the first switch instruction to the electric equipment. For example: the user keeps sitting in the living room, the three-dimensional image sensor captures the image, and the data processing unit 110 determines that the posture of the user is watching television and needs to power on the lamps in the living room through the posture detection model. The data processing unit 110 queries the number corresponding to the lamp in the living room through the database, and accurately sends the power-on command to the electric equipment through the number.
According to the control method of the electric equipment provided by the embodiment, the electric equipment can be adjusted to the state desired by the user through the dual control of the user posture and the voice. The control method does not set a wake-up word for voice control, so that a user can directly speak a natural language and the control method is more in line with the human speaking logic. When the user uses the control method, the user does not need to download the application program, so that the operation difficulty of the user is reduced, and the experience of the user is improved.
The embodiment also provides an electronic device, comprising a processor; and a memory storing a computer program that, when executed by the processor, executes at least one step of the method for controlling the electric device.
The electronic equipment that this embodiment provided is applied to above-mentioned consumer's controlling means, assigns the method through different instructions, has increased consumer's sensitive degree for each instruction action of user can both send to consumer effectively, and consumer also can accurately carry out corresponding instruction.
Example 2
This embodiment is substantially the same as embodiment 1, and therefore, the same portions are not described herein again, and only different portions are described in detail herein.
In this embodiment, the second switch command is a switch command of the electric device obtained by a gesture detection model.
Based on the method for controlling the electric device provided in embodiment 1, step S100) of model training further includes the following steps: the data processor collects more than two sets of training samples. Each group of training samples comprises a plurality of frames of gesture images, the training samples in the same group are marked as the same group label, and the group label comprises a starting instruction, a stopping instruction and an invalid instruction. The data processing unit inputs a plurality of groups of training samples into a machine learning model for training. The data processing unit generates a gesture detection model. The gesture detection model can judge the instruction type corresponding to the gesture according to the gesture of the user. For example: and a picture shot with the left arm lifted by the user is marked as on, a picture shot with the right arm lifted by the user is marked as off, and a manufacturer records a large number of the two pictures into the machine learning model to finally obtain a gesture detection model. When the user lifts the left arm, the data processing unit 110 can obtain a light-on command through the gesture detection model, and when the user lifts the right arm, the data processing unit 110 can obtain a light-off command through the gesture detection model. The gesture detection model is preferably trained from human skeletal points or human 19-point cloud data.
As shown in fig. 7, in step S800), the data processing unit determines whether a first switch command can be acquired, and if not, the data processing unit executes steps S810-S840.
Step S810) the data processing unit intercepts the user' S arm image from the image.
Step S820) the data processing unit inputs the arm image of the user to a gesture detection model.
Step S830) the data processing unit determines whether a second switch command can be obtained, and if so, executes the next step. If not, the data processing unit 110 executes steps S831-S834, as shown in FIG. 6. Step S831) the data processing unit collects the voice instruction. The voice instruction is user voice collected by voice recognition equipment. Step S832) the data processing unit acquires the area where the voice recognition device that has collected the voice is located. Step S833) the data processing unit determines whether the voice command is consistent with at least one preset voice command, and if so, executes step S834. Step 834) the data processing unit sends a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located. In this embodiment, when the body posture and the gesture of the user cannot be detected, the voice recognition device starts to be started, and the data processing unit receives whether the user issues a voice command in real time. For example: the data processing unit does not recognize that the user is in the living room, at which time the user shouts "light" in the living room, and the data processing unit 110 recognizes that the voice recognition device is located, i.e., the living room. The data processing unit 110 determines that the user is in the living room and needs to turn on the light, and then inquires the number of the lamp corresponding to the living room and sends a light-on command to the lamp.
Step 840) the data processing unit sends the second switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located. For example: the user stretches the left arm in the living room, the three-dimensional image sensor shoots the image, the data processing unit 110 judges the gesture of the user to be an instruction for electrifying the lamp in the living room through the gesture detection model, the data processing unit 110 inquires a number corresponding to the lamp in the living room through the database, and the first switch instruction is accurately sent to the lamp in the living room through the number so as to ensure the accuracy of the instruction.
The control method for the electric equipment provided by the embodiment is based on triple judgment of the gesture, the gesture and the voice, and the on-off state of the electric equipment can be adjusted when any one instruction of a user reaches the standard, so that the on-off state of the electric equipment can better accord with the mind of the user, and the user experience is further improved.
Example 3
This embodiment is substantially the same as embodiment 2, and therefore, the same parts are not described herein again, and only different parts are described in detail herein.
The second switching instruction is a switching instruction of the voice recognition equipment obtained by a gesture detection model.
As shown in fig. 6, on the basis of the control method of an electric device proposed in embodiment 2, step S830) the data processing unit determines whether a second switching command can be acquired. If so, the data processing unit performs steps S831-S834. Step S831) the data processing unit collects voice commands; the voice instruction is user voice collected by voice recognition equipment. Step S832) the data processing unit acquires the area where the voice recognition device that has collected the voice is located. Step S833) the data processing unit determines whether the voice command is consistent with at least one preset voice command, and if so, executes step S834. Step 834) the data processing unit sends a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located. If not, the data processing unit returns to the step S400) and the data processing unit acquires a frame of image in real time.
In this embodiment, when the gesture of the user is detected, the voice recognition device starts to be started, and the data processing unit receives in real time whether the user issues a voice command. For example: the data processing unit recognizes a gesture of the user in the living room on behalf of the activation of the speech recognition device and the data processing unit 110 activates the speech recognition device. At this time, the user shouts "light on" in the living room, and the data processing unit 110 recognizes the location of the voice recognition device, i.e., the living room. The data processing unit 110 determines that the user is in the living room and needs to turn on the light, and then inquires the number of the lamp corresponding to the living room and sends a light-on command to the lamp.
The control method of the electric equipment provided by the embodiment starts the voice recognition equipment after judging the gesture of the user, so that the setting of the awakening word is avoided, the natural language of the user is more met, the user does not need to speak the awakening word irrelevant to the command when issuing the command at every time, and the user experience is greatly improved.
Claims (14)
1. A control method, characterized by comprising the steps of:
dividing a specific space into at least one region;
acquiring a frame of image in real time; the images comprise an environment image and a user image which are shot by a three-dimensional image sensor;
acquiring an area where a three-dimensional image sensor corresponding to the image is located;
intercepting a user image from the image;
inputting the user image to a gesture detection model; and
and judging whether a first switch instruction can be acquired, if so, sending the first switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located.
2. The control method of claim 1, wherein prior to said step of acquiring a frame of image in real time, comprising the steps of:
an electric device, a three-dimensional image sensor and a voice recognition device are arranged in each area.
3. The control method of claim 1, wherein the step of determining whether a first switching command can be obtained further comprises the steps of:
if not, intercepting an image of the arm of the user from the image;
inputting the user arm image to a gesture detection model; and
and judging whether a second switch instruction can be acquired, if so, sending the second switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located.
4. The control method of claim 3, wherein the step of determining whether a second switching command can be obtained further comprises the steps of:
if not, acquiring a voice instruction; the voice instruction is user voice collected by voice recognition equipment;
acquiring the area where the voice recognition equipment collecting voice is located; and
and judging whether the voice command is consistent with at least one preset voice command, if so, sending a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located.
5. The control method of claim 1, wherein the step of determining whether a first switching command can be obtained further comprises the steps of:
if not, acquiring a voice instruction; the voice instruction is user voice collected by voice recognition equipment;
acquiring the area where the voice recognition equipment collecting voice is located; and
and judging whether the voice command is consistent with at least one preset voice command, if so, sending a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located.
6. The control method of claim 1, wherein the step of determining whether a first switching command can be obtained further comprises the steps of:
judging whether a first switch instruction can be acquired or not, and if not, intercepting an image of the arm of the user from the image;
inputting the user arm image to a gesture detection model;
judging whether a second switching instruction can be acquired, if so, sending the second switching instruction to a voice recognition device;
collecting a voice instruction; the voice instruction is user voice collected by voice recognition equipment;
acquiring the area where the voice recognition equipment collecting voice is located; and
and judging whether the voice command is consistent with at least one preset voice command, if so, sending a third switch command corresponding to the voice command to the electric equipment corresponding to the area where the voice recognition equipment is located.
7. The control method of claim 1, wherein prior to said step of acquiring a frame of image in real time, comprising the steps of:
collecting more than two groups of training samples, wherein each group of training samples comprises a plurality of frames of human body posture images, the training samples of the same group are marked as the same group labels, and the group labels comprise a starting instruction, a stopping instruction and an invalid instruction;
inputting a plurality of groups of training samples into a machine learning model for training; and
and generating a gesture detection model, and judging the instruction type corresponding to the gesture according to the gesture of the user.
8. The control method of claim 1, wherein prior to said step of acquiring a frame of image in real time, comprising the steps of:
collecting more than two groups of training samples, wherein each group of training samples comprises multi-frame gesture images, the training samples in the same group are marked as the same group labels, and the group labels comprise a starting instruction, a shutdown instruction and an invalid instruction;
inputting the training sample into a machine learning model for training; and
and generating a gesture detection model, and judging the instruction type corresponding to the gesture according to the gesture of the user.
9. The control method of claim 1, wherein prior to said step of acquiring a frame of image in real time, comprising the steps of:
and storing the serial numbers and the areas of the electric equipment, the three-dimensional image sensor and the voice recognition equipment to a database.
10. The method according to claim 1, wherein the step of sending the first switch command to the electric device corresponding to the area where the three-dimensional image sensor is located includes the following steps:
acquiring the area of the three-dimensional image sensor according to the number of the three-dimensional image sensor of the acquired image;
acquiring the electric equipment of the area according to the area of the three-dimensional image sensor; and
and sending the first switch instruction to the electric equipment.
11. The control method according to claim 1, wherein the step of acquiring a frame of image in real time specifically comprises the steps of:
acquiring a frame of image;
processing the image with a data filtering algorithm; and
and judging whether the number of the difference pixel points of the two adjacent processed images is greater than or equal to a preset threshold value, if not, deleting the last acquired frame of image, and returning to the step of acquiring one frame of image.
12. A control device, characterized by comprising a data processing unit, the data processing unit comprising:
the area dividing unit is used for dividing a specific space into at least one area;
the image acquisition unit is used for acquiring a frame of image in real time; the images comprise an environment image and a user image which are shot by a three-dimensional image sensor;
the area acquisition unit is used for acquiring an area where the three-dimensional image sensor corresponding to the image is located;
an image intercepting unit configured to intercept a user image from the image;
the image input unit is used for inputting the user image to a gesture detection model and judging whether a first switch instruction can be acquired or not; and
and the instruction sending unit is used for sending the first switch instruction to the electric equipment corresponding to the area where the three-dimensional image sensor is located.
13. The control apparatus according to claim 12, further comprising:
a speech recognition device; connected to the data processing unit;
a communication unit connected to the data processing unit;
at least one powered device connected to the data processing unit; and
and the three-dimensional image sensor is connected to the data processing unit.
14. An electronic device, comprising:
a processor; and
memory storing a computer program which, when executed by the processor, performs at least one of the steps of any of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110001669.2A CN112750437A (en) | 2021-01-04 | 2021-01-04 | Control method, control device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110001669.2A CN112750437A (en) | 2021-01-04 | 2021-01-04 | Control method, control device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112750437A true CN112750437A (en) | 2021-05-04 |
Family
ID=75649655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110001669.2A Pending CN112750437A (en) | 2021-01-04 | 2021-01-04 | Control method, control device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112750437A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114258176A (en) * | 2021-12-31 | 2022-03-29 | 欧普照明股份有限公司 | Lamp and lamp control method |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932212A (en) * | 2012-10-12 | 2013-02-13 | 华南理工大学 | Intelligent household control system based on multichannel interaction manner |
| CN202801140U (en) * | 2012-08-31 | 2013-03-20 | 华南理工大学 | Smart gesture and voice control system for curtain |
| CN105700372A (en) * | 2016-03-11 | 2016-06-22 | 珠海格力电器股份有限公司 | Intelligent device and control method thereof |
| CN106934334A (en) * | 2015-12-31 | 2017-07-07 | 芋头科技（杭州）有限公司 | A kind of gesture motion recognition methods and smart machine |
| US20170270924A1 (en) * | 2016-03-21 | 2017-09-21 | Valeo Vision | Control device and method with voice and/or gestural recognition for the interior lighting of a vehicle |
| CN107390881A (en) * | 2017-09-14 | 2017-11-24 | 西安领讯卓越信息技术有限公司 | A kind of gestural control method |
| WO2018031758A1 (en) * | 2016-08-11 | 2018-02-15 | Alibaba Group Holding Limited Fourth Floor, One Capital Place | Control system and control processing method and apparatus |
| CN108108709A (en) * | 2017-12-29 | 2018-06-01 | 纳恩博（北京）科技有限公司 | A kind of recognition methods and device, computer storage media |
| CN108181992A (en) * | 2018-01-22 | 2018-06-19 | 北京百度网讯科技有限公司 | Voice awakening method, device, equipment and computer-readable medium based on gesture |
| CN108307116A (en) * | 2018-02-07 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Image capturing method, device, computer equipment and storage medium |
| CN108536302A (en) * | 2018-04-17 | 2018-09-14 | 中国矿业大学 | A kind of teaching method and system based on human body gesture and voice |
| CN109886070A (en) * | 2018-12-24 | 2019-06-14 | 珠海格力电器股份有限公司 | Equipment control method and device, storage medium and equipment |
| CN209693124U (en) * | 2018-12-30 | 2019-11-26 | 欧普照明股份有限公司 | Intelligent illumination control device and intelligent illuminating system |
| CN111245688A (en) * | 2019-12-26 | 2020-06-05 | 的卢技术有限公司 | Method and system for intelligently controlling electrical equipment based on indoor environment |
| CN111274460A (en) * | 2020-01-20 | 2020-06-12 | 重庆百事得大牛机器人有限公司 | Semantic information acquisition system and method based on posture control |
| CN111368800A (en) * | 2020-03-27 | 2020-07-03 | 中国工商银行股份有限公司 | Gesture recognition method and device |
| CN111522524A (en) * | 2020-03-19 | 2020-08-11 | 浙江省北大信息技术高等研究院 | Presentation control method and device based on conference robot, storage medium and terminal |
| CN112053683A (en) * | 2019-06-06 | 2020-12-08 | 阿里巴巴集团控股有限公司 | Voice instruction processing method, device and control system |
-
2021
- 2021-01-04 CN CN202110001669.2A patent/CN112750437A/en active Pending
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202801140U (en) * | 2012-08-31 | 2013-03-20 | 华南理工大学 | Smart gesture and voice control system for curtain |
| CN102932212A (en) * | 2012-10-12 | 2013-02-13 | 华南理工大学 | Intelligent household control system based on multichannel interaction manner |
| CN106934334A (en) * | 2015-12-31 | 2017-07-07 | 芋头科技（杭州）有限公司 | A kind of gesture motion recognition methods and smart machine |
| CN105700372A (en) * | 2016-03-11 | 2016-06-22 | 珠海格力电器股份有限公司 | Intelligent device and control method thereof |
| US20170270924A1 (en) * | 2016-03-21 | 2017-09-21 | Valeo Vision | Control device and method with voice and/or gestural recognition for the interior lighting of a vehicle |
| WO2018031758A1 (en) * | 2016-08-11 | 2018-02-15 | Alibaba Group Holding Limited Fourth Floor, One Capital Place | Control system and control processing method and apparatus |
| CN107390881A (en) * | 2017-09-14 | 2017-11-24 | 西安领讯卓越信息技术有限公司 | A kind of gestural control method |
| CN108108709A (en) * | 2017-12-29 | 2018-06-01 | 纳恩博（北京）科技有限公司 | A kind of recognition methods and device, computer storage media |
| CN108181992A (en) * | 2018-01-22 | 2018-06-19 | 北京百度网讯科技有限公司 | Voice awakening method, device, equipment and computer-readable medium based on gesture |
| CN108307116A (en) * | 2018-02-07 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Image capturing method, device, computer equipment and storage medium |
| CN108536302A (en) * | 2018-04-17 | 2018-09-14 | 中国矿业大学 | A kind of teaching method and system based on human body gesture and voice |
| CN109886070A (en) * | 2018-12-24 | 2019-06-14 | 珠海格力电器股份有限公司 | Equipment control method and device, storage medium and equipment |
| CN209693124U (en) * | 2018-12-30 | 2019-11-26 | 欧普照明股份有限公司 | Intelligent illumination control device and intelligent illuminating system |
| CN112053683A (en) * | 2019-06-06 | 2020-12-08 | 阿里巴巴集团控股有限公司 | Voice instruction processing method, device and control system |
| CN111245688A (en) * | 2019-12-26 | 2020-06-05 | 的卢技术有限公司 | Method and system for intelligently controlling electrical equipment based on indoor environment |
| CN111274460A (en) * | 2020-01-20 | 2020-06-12 | 重庆百事得大牛机器人有限公司 | Semantic information acquisition system and method based on posture control |
| CN111522524A (en) * | 2020-03-19 | 2020-08-11 | 浙江省北大信息技术高等研究院 | Presentation control method and device based on conference robot, storage medium and terminal |
| CN111368800A (en) * | 2020-03-27 | 2020-07-03 | 中国工商银行股份有限公司 | Gesture recognition method and device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114258176A (en) * | 2021-12-31 | 2022-03-29 | 欧普照明股份有限公司 | Lamp and lamp control method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11017217B2 (en) | System and method for controlling appliances using motion gestures | |
| WO2017215297A1 (en) | Cloud interactive system, multicognitive intelligent robot of same, and cognitive interaction method therefor | |
| CN107957120B (en) | Air supply method of air conditioner, air conditioner and computer readable storage medium | |
| CN111752165A (en) | Intelligent equipment control method and device of intelligent home system | |
| CN108681399B (en) | Equipment control method, device, control equipment and storage medium | |
| CN111128157B (en) | Wake-up-free voice recognition control method for intelligent household appliance, computer readable storage medium and air conditioner | |
| CN111492426A (en) | Voice control of gaze initiation | |
| CN110186161B (en) | Default parameter updating method, air conditioner self-learning control method and air conditioner | |
| CN111667798A (en) | Screen adjusting method and device | |
| CN107566227A (en) | Control method and device of household appliance, intelligent device and storage medium | |
| CN112747434B (en) | Indoor temperature control method, system, device and computer readable storage medium | |
| CN111681675B (en) | Data dynamic transmission method, device, equipment and storage medium | |
| CN110716444A (en) | Sound control method and device based on smart home and storage medium | |
| CN111801650A (en) | Electronic device and method of controlling external electronic device based on usage pattern information corresponding to user | |
| CN105912632A (en) | Device service recommending method and device | |
| CN109143882A (en) | A kind of control method and device | |
| CN114120996A (en) | Voice interaction method and device | |
| CN113867163A (en) | Intelligent household scene switching method and device, intelligent terminal and storage medium | |
| CN109986553B (en) | Active interaction robot, system, method and storage device | |
| WO2019180434A1 (en) | Processing a command | |
| CN112750437A (en) | Control method, control device and electronic equipment | |
| CN107274899A (en) | Air conditioner sound control method, device, air conditioner and readable storage medium storing program for executing | |
| CN106951071B (en) | Equipment control method and device based on motion capture | |
| CN115235067A (en) | Air conditioner control method, device, equipment and storage medium | |
| CN118151761B (en) | Data processing method and device based on virtual reality and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210504 |