CN108131808B - Air conditioner control device and method based on hierarchical gesture recognition - Google Patents
Air conditioner control device and method based on hierarchical gesture recognition Download PDFInfo
- Publication number
- CN108131808B CN108131808B CN201711293662.2A CN201711293662A CN108131808B CN 108131808 B CN108131808 B CN 108131808B CN 201711293662 A CN201711293662 A CN 201711293662A CN 108131808 B CN108131808 B CN 108131808B
- Authority
- CN
- China
- Prior art keywords
- gesture
- instruction
- image
- primary
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
- Image Analysis (AREA)
- Air Conditioning Control Device (AREA)
Abstract
The invention relates to an air conditioner control device and method based on hierarchical gesture recognition, which opens up two parallel threads through an image recognition module to perform primary gesture recognition and secondary gesture recognition on an indoor image, and stores a primary instruction or a secondary instruction matched with the primary gesture or the secondary gesture if the primary gesture or the secondary gesture is recognized; only when the primary instruction and the secondary instruction are simultaneously stored in the instruction storage module, the primary instruction and the secondary instruction are combined together to realize the control of the air conditioner. Compared with the prior art, the invention realizes the control of the temperature, the wind speed, the mode selection and the like of the air conditioner through different gestures of people, so that the interaction mode of a user and the air conditioner is more intelligent, and the air conditioner is more convenient to use. In addition, the invention realizes the control of the air conditioner by combining two-stage gestures, so that the control of the air conditioner is more stable.
Description
Technical Field
The invention relates to the field of household appliance control, in particular to an air conditioner control device and method based on hierarchical gesture recognition.
Background
The control of the existing air conditioner is controlled by a remote controller, and people can know the running state of the air conditioner by an indicator lamp or a display panel on the air conditioner after using the remote controller. In addition, some air conditioners are provided with keys which can be used for operating the air conditioner through the keys on the air conditioner body. That is, the current air conditioner control can be realized only by a medium (such as a remote controller and keys on the air conditioner body) independent of the human body, and the user needs to find the remote controller or run to the air conditioner to press the keys before operating the air conditioner, which is inconvenient to operate.
Disclosure of Invention
The invention aims to provide an air conditioner control device and method based on hierarchical gesture recognition, which can improve the use convenience of a user for an air conditioner.
In order to achieve the purpose, the invention adopts the technical scheme that:
an air conditioner control device based on hierarchical gesture recognition comprises an image acquisition module, an image processing module, an image recognition module, a gesture instruction matching module, an instruction storage module and an instruction interpretation module, wherein,
the image acquisition module is used for acquiring indoor images; the image processing module is used for processing the image to remove noise in the image;
the image recognition module is used for storing preset primary gestures and preset secondary gestures and performing gesture recognition on the image;
the gesture instruction matching module is used for storing a preset instruction and searching an instruction matched with the recognized gesture, wherein the instruction comprises a primary instruction and a secondary instruction, the primary instruction corresponds to the primary gesture and is used for selecting the temperature, the wind speed and the air conditioning mode of the air conditioner; the second-level command corresponds to the second-level gesture, comprises an addition item, a subtraction item, a previous item and a next item, and is used for controlling the temperature, the wind speed and the air-conditioning mode options;
the instruction storage module is used for storing the first-level instruction and the second-level instruction which are searched in the gesture instruction matching module, and comprises a first-level instruction storage unit and a second-level instruction storage unit;
and the instruction interpretation module is used for interpreting the stored instruction into an air conditioner control signal.
The image acquisition module comprises an infrared camera and an infrared light supplement lamp.
The air conditioner control method based on the classification gesture recognition adopts the air conditioner control device, and specifically comprises the following steps:
Acquiring indoor pictures and videos of various people, manually calibrating a gesture area, and sending calibrated data into a deep convolutional neural network for training to obtain a gesture recognition model;
the gesture is defined by combining the gesture of the arm through the orientation of the palm, the orientation of the fingers and the bending or not of the fingers;
step 4.1, primary gesture recognition
Step 4.1.1, performing gesture recognition processing on the image according to the gesture recognition model obtained in the step 1, and outputting the image as a gesture X or no gesture; if the output is no gesture or the gesture X is not a primary gesture, abandoning the picture and continuously identifying the next frame of image; if the gesture X is a primary gesture, sending the gesture X to a gesture instruction matching module;
step 4.1.2, the gesture instruction matching module searches a matched primary instruction command _1_ X according to the type of the gesture X, writes the primary instruction command _1_ X into a primary instruction storage unit of the instruction storage module, and starts a Timer; when the Timer expires, clearing a primary instruction memory stored in a primary instruction memory unit of the instruction memory module;
step 4.2, two-stage gesture recognition
Step 4.2.1, performing gesture recognition processing on the image according to the gesture recognition model obtained in the step 1, and outputting the image as a gesture Y or no gesture; if the output is no gesture or the gesture Y is not a secondary gesture, abandoning the picture and continuously identifying the next frame; if the gesture Y is a secondary gesture, the gesture Y is sent to a gesture instruction matching module;
step 4.2.2, the gesture instruction matching module searches an instruction command _2_ Y matched with the gesture Y according to the type of the gesture Y, and writes the command _2_ Y into a secondary instruction storage unit of the instruction storage module;
4.2.3, judging whether the instruction stored in the instruction storage unit is complete, namely whether a primary instruction and a secondary instruction exist at the same time; if the command is complete, sending the complete command _1_ X + command _2_ Y to the command interpretation module, clearing the secondary commands stored in the command storage module, and refreshing a Timer for the retention time of the primary commands; if the data is not complete, directly emptying a secondary instruction stored in a secondary instruction storage unit of the instruction storage module;
and 4.2.4, the instruction interpretation module calculates a control signal corresponding to the instruction according to the received instruction command _1_ X + command _2_ Y, and sends the control signal to the air conditioner.
The gesture is defined as a primary gesture and a secondary gesture, wherein the primary gesture is used for selecting items needing to be adjusted, namely the temperature, the wind speed and the mode of an air conditioner, and the secondary gesture is used for adjusting the size of specific items, such as the temperature, the wind speed and the mode options; two parallel threads are opened up through the image recognition module to perform primary gesture recognition and secondary gesture recognition on the indoor image, and if the primary gesture or the secondary gesture is recognized, a primary instruction or a secondary instruction matched with the primary gesture or the secondary gesture is stored; only when the primary instruction and the secondary instruction are simultaneously stored in the instruction storage module, the primary instruction and the secondary instruction are combined together to realize the control of the air conditioner. Compared with the prior art, the invention realizes the control of the temperature, the wind speed, the mode selection and the like of the air conditioner through different gestures of people, so that the interaction mode of a user and the air conditioner is more intelligent, and the air conditioner is more convenient to use. In addition, the invention realizes the control of the air conditioner by combining two-stage gestures, so that the control of the air conditioner is more stable.
Drawings
FIG. 1 is a block diagram of the present invention;
FIG. 2 is a flow chart of a first level gesture recognition of the present invention;
FIG. 3 is a schematic diagram of a primary gesture in accordance with the present invention;
FIG. 4 is a flow chart of a two-stage gesture recognition process of the present invention;
FIG. 5 is a schematic diagram of a two-level gesture according to the present invention.
Detailed Description
Referring to fig. 1 to 5, the present invention discloses an air conditioner control device based on hierarchical gesture recognition, which includes an image acquisition module 1, an image processing module 2, an image recognition module 3, a gesture command matching module 4, a command storage module 5 and a command interpretation module 6, wherein,
the image acquisition module 1 is used for acquiring indoor images and consists of an infrared camera and an infrared light supplement lamp;
the image processing module 2 is used for processing the image to remove noise in the image;
the image recognition module 3 is used for storing preset primary gestures and secondary gestures and performing gesture recognition on the image;
the gesture instruction matching module 4 is used for searching an instruction matched with the recognized gesture, wherein the instruction comprises a primary instruction and a secondary instruction, the primary instruction corresponds to the primary gesture and is used for selecting the temperature, the wind speed and the air conditioning mode of the air conditioner; the second-level command corresponds to the second-level gesture, comprises an addition item, a subtraction item, a previous item and a next item, and is used for controlling the temperature, the wind speed and the air-conditioning mode options;
the instruction storage module 5 is used for storing the first-level instruction and the second-level instruction which are searched in the gesture instruction matching module, and comprises a first-level instruction storage unit and a second-level instruction storage unit; (ii) a
And the instruction interpretation module 6 is used for interpreting the stored instruction into an air conditioner control signal.
A complete instruction comprises a primary instruction and a secondary instruction, wherein the primary instruction comprises temperature, wind speed, air-conditioning mode and the like; the secondary instructions include add, subtract, previous, next, etc.
Based on the device, the invention also discloses an air conditioner control method based on the classification gesture recognition, which comprises the following steps:
Acquiring indoor pictures and videos of various people, manually calibrating accurate gesture areas (including palms, fingers and arms), and sending calibrated data into a deep Convolutional Neural Network (CNN) for training to obtain a gesture recognition model;
gestures are defined by the orientation of the palm, the orientation of the fingers, and whether the fingers are bent or not, in combination with the posture of the arm. The angle and the orientation in the gesture definition are relative to the visual angle of the camera, and the orientation of the palm is the direction in which the palm center or the palm is oriented to the camera; the orientation of the fingers comprises four directions of up, down, left and right; the posture of the arm includes an angle between the large arm and the human body and an angle between the small arm and the large arm.
step 4.1, primary gesture recognition
Step 4.1.1, performing gesture recognition processing on the image according to the gesture recognition model obtained in the step 1, and outputting the image as a gesture X or no gesture; if the output is no gesture or the gesture X is not a primary gesture, abandoning the picture and continuously identifying the next frame of image; if the gesture X is a primary gesture, the gesture X is sent to the gesture instruction matching module 4;
step 4.1.2, the gesture instruction matching module 4 searches a matched primary instruction command _1_ X according to the type of the gesture X, writes the primary instruction command _1_ X into a primary instruction storage unit of the instruction storage module 5, and starts a Timer; when the Timer expires, clearing the primary instruction memory stored in the primary instruction memory unit of the instruction memory module 5;
step 4.2, two-stage gesture recognition
Step 4.2.1, performing gesture recognition processing on the image according to the gesture recognition model obtained in the step 1, and outputting the image as a gesture Y or no gesture; if the output is no gesture or the gesture Y is not a secondary gesture, abandoning the picture and continuously identifying the next frame; if the gesture Y is a secondary gesture, the gesture Y is sent to the gesture instruction matching module 4;
step 4.2.2, the gesture instruction matching module 4 searches an instruction command _2_ Y matched with the gesture Y according to the type of the gesture Y, and writes the command _2_ Y into a secondary instruction storage unit of the instruction storage module 5;
4.2.3, judging whether the instruction stored by the instruction storage module 5 is complete, namely whether a primary instruction and a secondary instruction exist at the same time; if the command is complete, sending the complete command _1_ X + command _2_ Y to the command interpretation module 6, clearing the secondary commands stored in the command storage module 5, and refreshing the Timer of the primary command reserve time; if the instruction is not complete, directly clearing the secondary instructions stored in the secondary instruction storage unit of the instruction storage module 5;
and 4.2.4, calculating a control signal corresponding to the command by the command interpretation module 6 according to the received command _1_ X + command _2_ Y, and sending the control signal to the air conditioner.
The method is characterized in that gestures are defined as primary gestures and secondary gestures, two parallel threads are opened up through an image recognition module to perform primary gesture recognition and secondary gesture recognition on indoor images, and primary commands or secondary commands matched with the primary gestures or the secondary gestures are stored if the primary gestures or the secondary gestures are recognized; only when the primary instruction and the secondary instruction are simultaneously stored in the instruction storage module, the primary instruction and the secondary instruction are combined together to realize the control of the air conditioner. Compared with the prior art, the invention realizes the control of the temperature, the wind speed, the mode selection and the like of the air conditioner through different gestures of people, so that the interaction mode of a user and the air conditioner is more intelligent, and the air conditioner is more convenient to use. In addition, the invention realizes the control of the air conditioner by combining two-stage gestures, so that the control of the air conditioner is more stable.
The above description is only exemplary of the present invention and is not intended to limit the technical scope of the present invention, so that any minor modifications, equivalent changes and modifications made to the above exemplary embodiments according to the technical spirit of the present invention are within the technical scope of the present invention.
Claims (1)
1. An air conditioner control method based on hierarchical gesture recognition is characterized in that: the air conditioner control device based on the classification gesture recognition is adopted and comprises an image acquisition module, an image processing module, an image recognition module, a gesture instruction matching module, an instruction storage module and an instruction interpretation module, wherein,
the image acquisition module is used for acquiring indoor images;
the image processing module is used for processing the image to remove noise in the image;
the image recognition module is used for storing preset primary gestures and preset secondary gestures and performing gesture recognition on the image;
the gesture instruction matching module is used for storing a preset instruction and searching an instruction matched with the recognized gesture; the command comprises a primary command and a secondary command, wherein the primary command corresponds to the primary gesture and is used for selecting the temperature, the wind speed and the air-conditioning mode of the air conditioner; the second-level command corresponds to the second-level gesture, comprises an addition item, a subtraction item, a previous item and a next item, and is used for controlling the temperature, the wind speed and the air-conditioning mode options;
the instruction storage module is used for storing the first-level instruction and the second-level instruction which are searched in the gesture instruction matching module, and comprises a first-level instruction storage unit and a second-level instruction storage unit;
the instruction interpretation module is used for interpreting the stored instruction into an air conditioner control signal;
the air conditioner control method specifically comprises the following steps:
step 1, training a gesture recognition model
Acquiring indoor pictures and videos of various people, manually calibrating a gesture area, and sending calibrated data into a deep convolutional neural network for training to obtain a gesture recognition model;
the gesture is defined by combining the gesture of the arm through the orientation of the palm, the orientation of the fingers and the bending or not of the fingers;
step 2, presetting a primary gesture and a secondary gesture in the image recognition module, and presetting a primary instruction and a secondary instruction matched with the primary gesture and the secondary gesture in the gesture instruction matching module;
step 3, acquiring an indoor image through an image acquisition module, and sending the image to an image processing module; wavelet transformation is carried out on the image through an image processing module, then the image is processed through a median filter and a Gaussian filter to remove noise in the image, and the processed image is sent to an image identification module;
step 4, two parallel threads are opened up through the image recognition module to respectively perform primary gesture recognition and secondary gesture recognition on the image;
step 4.1, primary gesture recognition
Step 4.1.1, performing gesture recognition processing on the image according to the gesture recognition model obtained in the step 1, and outputting the image as a gesture X or no gesture; if the output is no gesture or the gesture X is not a primary gesture, abandoning the picture and continuously identifying the next frame of image; if the gesture X is a primary gesture, sending the gesture X to a gesture instruction matching module;
step 4.1.2, the gesture instruction matching module searches a matched primary instruction command _1_ X according to the type of the gesture X, writes the primary instruction command _1_ X into a primary instruction storage unit of the instruction storage module, and starts a Timer; when the Timer expires, clearing a primary instruction stored in a primary instruction storage unit of the instruction storage module;
step 4.2, two-stage gesture recognition
Step 4.2.1, performing gesture recognition processing on the image according to the gesture recognition model obtained in the step 1, and outputting the image as a gesture Y or no gesture; if the output is no gesture or the gesture Y is not a secondary gesture, abandoning the picture and continuously identifying the next frame; if the gesture Y is a secondary gesture, the gesture Y is sent to a gesture instruction matching module;
step 4.2.2, the gesture instruction matching module searches a secondary instruction command _2_ Y matched with the gesture Y according to the type of the gesture Y, and writes the secondary instruction command _2_ Y into a secondary instruction storage unit of the instruction storage module;
4.2.3, judging whether the instruction stored in the instruction storage unit is complete, namely whether a primary instruction and a secondary instruction exist at the same time; if the command is complete, sending the complete command _1_ X + command _2_ Y to the command interpretation module, clearing the secondary commands stored in the command storage module, and refreshing a Timer for the retention time of the primary commands; if the data is not complete, directly emptying a secondary instruction stored in a secondary instruction storage unit of the instruction storage module;
and 4.2.4, the instruction interpretation module calculates a control signal corresponding to the instruction according to the received instruction command _1_ X + command _2_ Y, and sends the control signal to the air conditioner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711293662.2A CN108131808B (en) | 2017-12-08 | 2017-12-08 | Air conditioner control device and method based on hierarchical gesture recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711293662.2A CN108131808B (en) | 2017-12-08 | 2017-12-08 | Air conditioner control device and method based on hierarchical gesture recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108131808A CN108131808A (en) | 2018-06-08 |
| CN108131808B true CN108131808B (en) | 2020-03-27 |
Family
ID=62389281
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711293662.2A Active CN108131808B (en) | 2017-12-08 | 2017-12-08 | Air conditioner control device and method based on hierarchical gesture recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108131808B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110967976A (en) * | 2018-09-28 | 2020-04-07 | 珠海格力电器股份有限公司 | Control method and device of intelligent home system |
| CN109814717B (en) * | 2019-01-29 | 2020-12-25 | 珠海格力电器股份有限公司 | Household equipment control method and device, control equipment and readable storage medium |
| CN109948542B (en) * | 2019-03-19 | 2022-09-23 | 北京百度网讯科技有限公司 | Gesture recognition method and device, electronic equipment and storage medium |
| CN113190107B (en) * | 2021-03-16 | 2023-04-14 | 青岛小鸟看看科技有限公司 | Gesture recognition method and device and electronic equipment |
| CN113548061B (en) * | 2021-06-25 | 2023-04-18 | 北京百度网讯科技有限公司 | Man-machine interaction method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007078270A (en) * | 2005-09-14 | 2007-03-29 | Daikin Ind Ltd | Air conditioning system |
| CN105373785A (en) * | 2015-11-30 | 2016-03-02 | 北京地平线机器人技术研发有限公司 | Method and device of hand gesture recognition and detection on the basis of deep neural network |
| CN106054627A (en) * | 2016-06-12 | 2016-10-26 | 珠海格力电器股份有限公司 | Control method and device based on gesture recognition and air conditioner |
| CN106247561A (en) * | 2016-08-30 | 2016-12-21 | 广东美的制冷设备有限公司 | A kind of air-conditioning and long-range control method thereof and device |
| CN106406524A (en) * | 2016-09-05 | 2017-02-15 | 珠海格力电器股份有限公司 | Equipment control method and device |
| CN106886275A (en) * | 2015-12-15 | 2017-06-23 | 比亚迪股份有限公司 | The control method of car-mounted terminal, device and vehicle |
-
2017
- 2017-12-08 CN CN201711293662.2A patent/CN108131808B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007078270A (en) * | 2005-09-14 | 2007-03-29 | Daikin Ind Ltd | Air conditioning system |
| CN105373785A (en) * | 2015-11-30 | 2016-03-02 | 北京地平线机器人技术研发有限公司 | Method and device of hand gesture recognition and detection on the basis of deep neural network |
| CN106886275A (en) * | 2015-12-15 | 2017-06-23 | 比亚迪股份有限公司 | The control method of car-mounted terminal, device and vehicle |
| CN106054627A (en) * | 2016-06-12 | 2016-10-26 | 珠海格力电器股份有限公司 | Control method and device based on gesture recognition and air conditioner |
| CN106247561A (en) * | 2016-08-30 | 2016-12-21 | 广东美的制冷设备有限公司 | A kind of air-conditioning and long-range control method thereof and device |
| CN106406524A (en) * | 2016-09-05 | 2017-02-15 | 珠海格力电器股份有限公司 | Equipment control method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108131808A (en) | 2018-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108131808B (en) | Air conditioner control device and method based on hierarchical gesture recognition | |
| CN107576022B (en) | Control method of air conditioner, air conditioner and storage medium | |
| CN108431733B (en) | Electronic device for gesture detection | |
| CN111752165A (en) | Intelligent equipment control method and device of intelligent home system | |
| KR102121785B1 (en) | Air-conditioner controlling direction of the wind using artificial intelligence by instructed position and method of controlling thereof | |
| CN103295028B (en) | gesture operation control method, device and intelligent display terminal | |
| US20210201661A1 (en) | System and Method of Hand Gesture Detection | |
| CN108489034B (en) | Air conditioner control method, terminal, air conditioner and computer readable storage medium | |
| JP2019532543A (en) | Control system and control processing method and apparatus | |
| CN105042789B (en) | The control method and system of a kind of intelligent air condition | |
| Kim et al. | Watch & Do: A smart iot interaction system with object detection and gaze estimation | |
| CN111128157B (en) | Wake-up-free voice recognition control method for intelligent household appliance, computer readable storage medium and air conditioner | |
| US10970128B2 (en) | Server, air conditioner and method for controlling thereof | |
| CN110426962A (en) | A kind of control method and system of smart home device | |
| CN205053686U (en) | Curtain control system with gesture recognition | |
| CN111240217B (en) | State detection method and device, electronic equipment and storage medium | |
| CN107229262A (en) | A kind of intelligent domestic system | |
| CN113918019A (en) | Gesture recognition control method and device for terminal equipment, terminal equipment and medium | |
| CN113009897A (en) | Control method and device of intelligent household appliance, intelligent household appliance and storage medium | |
| CN110555383A (en) | Gesture recognition method based on convolutional neural network and 3D estimation | |
| CN111461059A (en) | Multi-zone multi-classification extensible gesture recognition control device and control method | |
| CN106354129A (en) | Kinect based gesture recognition control system and method for smart car | |
| CN109934155B (en) | Depth vision-based collaborative robot gesture recognition method and device | |
| CN108415572B (en) | Module control method and device applied to mobile terminal and storage medium | |
| CN204787062U (en) | Air conditioner control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |