CN110687816A - Intelligent household control system and method based on millimeter wave radar - Google Patents
Intelligent household control system and method based on millimeter wave radar Download PDFInfo
- Publication number
- CN110687816A CN110687816A CN201911052118.8A CN201911052118A CN110687816A CN 110687816 A CN110687816 A CN 110687816A CN 201911052118 A CN201911052118 A CN 201911052118A CN 110687816 A CN110687816 A CN 110687816A
- Authority
- CN
- China
- Prior art keywords
- subsystem
- wave radar
- millimeter wave
- data
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 56
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 14
- 230000009471 action Effects 0.000 claims description 38
- 238000004891 communication Methods 0.000 claims description 26
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 12
- 230000003993 interaction Effects 0.000 claims description 12
- 238000013461 design Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 238000013145 classification model Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000002592 echocardiography Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 230000002093 peripheral effect Effects 0.000 claims description 4
- 230000003213 activating effect Effects 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000013500 data storage Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 claims description 2
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 claims description 2
- 230000004044 response Effects 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims description 2
- 230000026676 system process Effects 0.000 abstract 1
- 230000000875 corresponding effect Effects 0.000 description 18
- 230000001276 controlling effect Effects 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B15/00—Systems controlled by a computer
- G05B15/02—Systems controlled by a computer electric
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/26—Pc applications
- G05B2219/2642—Domotique, domestic, home control, automation, smart house
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention belongs to the technical field of intelligent identification and control, and particularly relates to an intelligent home control system and method based on a millimeter wave radar. The system comprises a millimeter wave radar system, a signal processing system, an artificial intelligence classification system and a central control system; the millimeter wave radar system transmits linear frequency modulation continuous wave signals to a radar irradiation scene, receives echo signals reflected by the scene, and obtains intermediate frequency original data after processing; the signal processing system processes the original data to obtain characteristic data and transmits the characteristic data to the artificial intelligent classification system; the artificial intelligence classification system performs off-line training and on-line classification on the millimeter wave radar characteristic data and transmits the millimeter wave radar characteristic data to the central control system; the central control system controls, monitors and communicates the system in real time. The invention realizes non-contact remote control in the field of smart home by using the millimeter wave radar, improves convenience, comfort and intelligence of daily life, and is suitable for places such as ordinary families, offices, meeting rooms and the like.
Description
Technical Field
The invention belongs to the technical field of intelligent identification and control, and particularly relates to an intelligent home control system and method based on a millimeter wave radar.
Background
With the energizing of artificial intelligence technology, the intelligent household industry develops rapidly, and the ecology gradually becomes mature. The explosion of the industrial revolution 4.0 makes the smart home industry have great interest, and as a new industry development direction, the smart home has been just like a trend of future home trend development.
Motion recognition is playing an increasingly important role as a way of human-computer interaction control. With the development of human-computer interaction, the interaction control mode is being changed from traditional contact control to non-contact control.
At present, some products controlled in a non-contact manner exist in the market, such as microsoft Kinect for recognizing human body actions based on visible light images and depth images, a mobile phone based on voice recognition control, a smart sound box and the like. However, the existing non-contact control technology and product have the defects of application scenes and performance:
1. the method based on image information such as visible light, depth and the like has high power consumption, high calculation cost, low efficiency of extracting features, limited capability of an image processing model, and easy influence of factors such as illumination, shielding and the like;
2. the speech recognition based method is limited by the acoustic wave propagation distance and speed, user pronunciation and speech recognition accuracy, and is not suitable for noisy and silent scenes;
3. both of the above methods present a risk of revealing user privacy.
The intelligent home control system based on the millimeter wave radar adopts a multichannel Linear Frequency Modulation Continuous Wave (LFMCW) scheme, can realize high distance and speed resolution and acquire more useful information. The target echo is separated from the radar echo reflected by the human body, the characteristic information of the target, such as the reflection energy, the scattering cross section, the distance, the speed, the azimuth angle, the elevation angle and the like, is further calculated, and the artificial intelligence method combining the characteristic matching with the deep learning is further adopted for recognition and classification, so that the three-dimensional real-time man-machine interaction action control in a large field angle range can be realized.
The intelligent home control scheme based on the multichannel LFMCW millimeter wave radar has the advantages that the advantages are prominent, and the advantages are mainly reflected in the following aspects:
1. better speed resolution and distance resolution can be obtained by utilizing the millimeter waves with high frequency and large bandwidth;
2. background interference in different distance ranges can be easily and accurately filtered by utilizing an FMCW technology;
3. the scheme of forming a two-dimensional virtual antenna array (multichannel) by adopting the transceiving antennas is matched with hardware to accurately measure the phase information, so that not only can the azimuth angle and elevation angle information of a target be obtained, but also the phase information sensitive to micro-motion can be obtained.
Disclosure of Invention
The invention aims to provide a system and a method capable of realizing intelligent home control through non-contact human-computer interaction. The man-machine interaction action can be customized and designed according to the requirements of users.
In order to improve the control convenience and accuracy in intelligent home control and overcome the defects caused by the influence factors of illumination, a visual field angle, a sensing distance, calculation expense, obstacles and the like of the traditional identification technology and products based on visual images, voice and the like.
The invention provides an intelligent home control system and method based on a millimeter wave radar, which are innovatively designed by combining the advantages of the millimeter wave radar and artificial intelligence so as to be suitable for intelligent man-machine interaction control, in particular to control application in an intelligent home.
The invention provides an intelligent home control system based on a millimeter wave radar, which comprises a millimeter wave radar subsystem, a signal processing subsystem, an artificial intelligence classification subsystem and a central control subsystem, wherein the millimeter wave radar subsystem is used for generating a signal; the millimeter wave radar subsystem is used for transmitting Linear Frequency Modulation Continuous Wave (LFMCW) signals to a radar irradiation scene, receiving radar echo signals reflected by the scene, mixing the echoes with the transmitted signals, and then performing band-pass filtering and analog-to-digital conversion (ADC) sampling to obtain intermediate-frequency original data; the signal processing subsystem is used for carrying out a series of signal processing on the original data to obtain characteristic data and transmitting the characteristic data to the artificial intelligent classification subsystem in a wired or wireless communication mode; the artificial intelligence classification subsystem is used for performing off-line training and classification on the millimeter wave radar characteristic data and transmitting a classification result to the central control subsystem in a wired or wireless communication mode; and the central control subsystem is used for controlling, monitoring and communicating the other systems in real time, activating corresponding control instructions according to the classification results and controlling corresponding intelligent household hardware facilities to react.
In the invention, the millimeter wave radar subsystem comprises a millimeter wave radar chip, a receiving and transmitting antenna component, an MCU, a communication component, a power supply module and a peripheral circuit; the millimeter wave radar chip assembly is provided with main functional devices such as signal generation, signal receiving, frequency multiplication, frequency mixing, filtering, analog-to-digital conversion (ADC), data caching, a communication interface and the like; the receiving and transmitting antenna assembly adopts a multi-input multi-output antenna array, which is called an MIMO antenna array for short; the MCU is used for controlling the radar, configuring parameters and preprocessing data; the communication component is used for communication and data transmission with other subsystems; the power supply module is used for supplying power to the whole millimeter wave radar subsystem; the peripheral circuits are used to connect the various modules, components and external subsystems.
In the invention, the working process of the millimeter wave radar subsystem comprises the following steps:
the central control subsystem sends radar configuration parameters and a starting command to the millimeter wave radar subsystem MCU, the MCU controls the millimeter wave radar to periodically transmit signals at the transmitting and receiving front end and receive echoes reflected by a radar irradiation scene, and the time delay between the echo signals and the transmitted signals is delta tdCenter of carrier waveFrequency fc. The transmitting signal adopts a linear continuous frequency modulation wave, and the expression isWhere B is the signal bandwidth and T is the sweep period. The expression of the transmitting signal and the receiving signal is as follows:
sT(t)＝ATcos2π[fct+∫0 tfT(τ)dτ]
sR(t)＝ARcos2π[fc(t-Δtd)+∫0 tfR(τ)dτ]
wherein A isTAnd ARRepresenting the amplitude of the transmitted and received signals, respectively, fR(τ) is the received signal frequency, expressed as
Wherein Δ fdIs the doppler shift.
The echo signal and the transmitting signal are subjected to frequency mixing and filtering in a millimeter wave radar chip to obtain an intermediate frequency signal, and the expression is as follows:
and the intermediate frequency signal is sampled by an ADC (analog to digital converter) to obtain original digital intermediate frequency signal data, which are called original data for short.
In the invention, the signal processing subsystem comprises a DSP module, an MCU module and a communication module. The DSP module and the MCU module are used for processing the signals of the original data, and specifically comprise: the millimeter wave radar chip transmits the acquired original data to the MCU module, the MCU module and the DSP module perform data processing on the original data and transmit processing process data to each other, the data processing mainly comprises data arrangement, basic mathematical operation, Fast Fourier Transform (FFT), Constant False Alarm Rate (CFAR) detection, arrival angle calculation and the like, and the characteristic data obtained by data processing are sent to the artificial intelligence classification subsystem.
In the invention, the central control subsystem has high-speed read-write capability, data storage capability, complex computing capability, neural network processing and computing capability, real-time classification and identification capability and real-time communication control capability. And when the central control subsystem identifies an effective human-computer interaction control action, the central control subsystem controls corresponding terminal hardware to make a corresponding response. The central control subsystem has a real-time GUI operational display interface. The central control subsystem is communicated with the MCU in the millimeter wave radar subsystem in a wired or wireless mode and is used for controlling the starting, the closing, the parameter configuration and the signal receiving and sending of the radar sensor, and is used for further processing the characteristic data obtained in the signal processing subsystem, calling an artificial neural network classification model obtained by the artificial intelligent classification subsystem to classify and identify, and controlling related hardware terminal equipment in a wired or wireless communication mode according to an instruction function corresponding to a classification result.
In the invention, the artificial intelligence classification subsystem comprises main functions of control action design, artificial neural network design, characteristic design and optimization, sample data elimination and optimization and the like.
In the invention, the control action is designed by combining the principles of real life scenes, correlation with control functions, high degree of feature discrimination, smoothness, naturalness and the like.
In the invention, the artificial neural network design comprises a network structure, a network layer type, an input/output size, various parameters and the like, and is designed according to the performance, the real-time requirement, the operation complexity requirement and the like of a hardware terminal where the artificial intelligent classification subsystem is located by comprehensively considering indexes such as training time consumption, model size, classification time consumption, CPU (Central processing Unit) and memory occupation.
In the invention, the data flow of the intelligent home control system based on the millimeter wave radar is as follows: the millimeter wave radar subsystem collects original data containing human body action information and transmits the original data to a signal processing subsystem formed by an MCU (microprogrammed control unit) and a DSP (digital signal processor) module for data processing, characteristic data obtained by data processing are transmitted to an artificial intelligent classification subsystem through a wired or wireless communication interface, the artificial intelligent classification subsystem calls a pre-trained artificial neural network classification model to classify the characteristic data and transmits classification results to a central control subsystem, the central control subsystem is communicated with all controlled terminal facilities, corresponding control commands are sent to the terminal facilities according to the classification results, and the terminal facilities react according to the control commands.
In the invention, in order to meet the requirement of better real-time performance and avoid using a high-speed data acquisition card with larger volume and higher cost, the millimeter wave radar subsystem and the signal processing subsystem are integrated in the same hardware terminal so as to realize high-speed data transmission of original data.
In the invention, the artificial intelligence classification subsystem and the central control subsystem can be built on an embedded processor, a mobile phone, a computer or a cloud server.
The invention also provides an intelligent home control method based on the intelligent home control system, which comprises the following specific steps:
designing a man-machine interaction control action, millimeter wave radar parameters and an artificial neural network structure and parameters according to an application scene;
step two, the human body performs predefined actions in an application scene, the millimeter wave radar subsystem transmits LFMCW signals to a radar irradiation scene, receives radar echo signals reflected by the scene, performs band-pass filtering and ADC sampling after mixing the echo and the transmitting signals to obtain intermediate-frequency original data, and transmits the original data to the signal processing subsystem;
thirdly, the signal processing subsystem carries out a series of data processing on the original data, extracts characteristic data related to human body actions and transmits the characteristic data to the artificial intelligence classification subsystem;
step four, repeating the process of the step one and the process of the step two to obtain a plurality of characteristic data samples of predefined actions, generating an off-line training sample data set in an artificial intelligence classification subsystem, screening and eliminating the training sample data set, then carrying out deep learning training and generating a classification model;
installing terminal nodes integrated by the millimeter wave radar subsystem and the signal processing subsystem at proper positions of each room, and establishing communication connection between the terminal nodes and the artificial intelligence classification subsystem and the central control system as well as between the central control subsystem and hardware equipment facilities in a wired or wireless communication mode;
step six, repeating the step one and the step two, calling the classification model generated in the step four in the artificial intelligence classification subsystem to perform predefined action recognition, and transmitting the recognition result to the central control subsystem;
and step seven, the central control subsystem sends a corresponding control instruction to the controlled hardware terminal facility according to the predefined action category corresponding to the identification result, and if the identification result is not the predefined action category, the step six is returned.
The invention focuses on the non-contact remote control of the millimeter wave radar in the field of smart home, improves the convenience, comfort and intelligence of daily life, can be suitable for common families, offices, meeting rooms and other indoor places, and has the following advantages:
the non-contact type non-inductive control is realized, the operation distance is long, and the use is convenient;
the control command action can be customized according to the user requirement, and the use is flexible;
the device has certain capability of penetrating or diffracting non-metallic obstacles, resists interference and does not depend on illumination conditions;
the integration degree is high, and the millimeter wave radar subsystem and the signal processing subsystem are integrated in a chip;
the system has the advantages that the calculation cost is low, the real-time performance is high, and the artificial intelligence classification subsystem and the central control subsystem can be built on a platform with limited computing capability, such as a smart phone, embedded hardware equipment and the like;
the system can be networked with the existing Internet of things equipment in a wired or wireless mode, and is matched with operation terminals such as mobile phones, computers and embedded hardware equipment for use;
the method combines the multichannel millimeter wave radar and the artificial intelligence technology, and has the advantages of abundant information acquisition quantity, high identification accuracy and good use experience.
Drawings
FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present invention.
FIG. 2 is a schematic diagram of a system architecture of one embodiment of the present invention.
Fig. 3 is a schematic diagram of an integrated structure of a millimeter wave radar subsystem and a signal processing subsystem according to an embodiment of the present invention.
FIG. 4 is a data flow diagram of one embodiment of the present invention.
FIG. 5 is a flowchart of the operation of one embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiments. All other embodiments obtained without inventive step should be included within the scope of protection of the present invention.
An application scenario of one embodiment of the present invention is shown in fig. 1:
(1) in a room similar to the room shown in fig. 1, data acquisition and processing terminals integrating the millimeter wave radar subsystem and the data processing subsystem, namely radar terminals for short, are respectively installed in the center of the ceiling, at four corners of the ceiling and on doors of the room, and an identity number is set for each terminal, and the maximum effective detection distance of the radar is set to be 5 meters by the terminals installed on the ceiling; the terminal of installation sets up the biggest effective detection distance of radar and is 2 meters on the door, guarantees effective detection and prevents the mistake and survey.
(2) Designing radar configuration parameters, controlling a radar terminal in a wired or wireless mode, and acquiring human body action characteristic data, wherein the characteristic data acquisition process comprises the following steps: modeling characteristic parameters of the motion multi-scattering points, including but not limited to radial distance, radial speed, azimuth angle, elevation angle and the like of the motion multi-scattering points relative to the radar; performing distance-wise windowing FFT and filtering on the time domain signals of each channel and each Chirp period, inhibiting side lobes and reducing background noise interference; extracting the distance from the target area, and filtering out the influence of background and other irrelevant factors; coherent accumulation and subtraction, subtracting the antenna coupling signal and the static background; performing Doppler windowing FFT and filtering, suppressing side lobes of velocity distribution to obtain a range Doppler characteristic matrix, and calculating inter-channel average amplitude of the range Doppler characteristic matrix, which is called range Doppler amplitude; acquiring the maximum value of the range-Doppler characteristic amplitude value and estimating the target distance and speed according to the index value; aligning the data so that the characteristic peak value is positioned in the middle of the frame window; filling a frame window, wherein the frame window slides one frame each time, and new effective data are filled; and normalizing the data size to 0-255 to obtain a feature matrix.
(3) The method comprises the steps of presetting 15 human body actions, respectively corresponding to 15 control instructions, enabling a person to do various actions at different positions of a room, obtaining corresponding samples by a data acquisition terminal 300 times every time to form a preset human body action characteristic data set, removing samples with poor characteristics, ensuring similarity and difference among sample data types, and training by using a self-defined lightweight artificial neural network to obtain a classification network model. The preset human-computer interaction action and the corresponding function design comprise the following steps:
firstly, advancing the body: opening a door;
II, retreating the body: closing the door;
thirdly, swinging the two arms to the top of the head in parallel: turning on a water heater;
fourthly, the two arms fall down from the head top in parallel: turning off the water heater;
fifthly, rotating the single arm at the top of the head: lamp on/off;
sixthly, pushing forwards slowly with one hand: turning on a television;
seventhly, slowly pulling back with one hand: a television gate;
eighthly, pushing forward with two hands rapidly: turning on an air conditioner;
ninthly, rapidly pulling back by two hands: turning off an air conditioner;
ten, left-waving with one hand: the television channel is decreased;
eleven, right-handed swing: adding a television channel;
twelve, single-hand volatilization: adding the volume of the television;
thirteen, one-hand downward volatilization: the volume of the television is reduced;
fourteen, the two arms are opened: adding the temperature of an air conditioner;
fifteen, folding the two arms: the air conditioner volume is decreased.
(4) The radar terminal is installed at a corresponding position of a user room and is connected with a user mobile phone or a computer and intelligent household facilities in the room in a wireless local area network mode, the user mobile phone or the computer is provided with a software client (software for short), and the software has the functions of adding networking equipment, controlling the radar terminal, carrying out artificial intelligence classification, controlling other equipment facilities and the like.
(5) Software controls a plurality of radar terminals to work simultaneously without mutual interference, data acquisition and processing processes are respectively carried out in each radar terminal, and the acquired characteristic data are classified on a user mobile phone or a computer.
(6) If the classification recognition result is a predefined action, controlling a television, an air conditioner, an electric lamp, a water heater, a door and the like to execute a corresponding instruction; otherwise, no reaction is carried out; for example, when a user moves towards a door within a range of 2 meters, the user is detected by a radar terminal and transmits action characteristic data to a user mobile phone or a computer through wireless local area network communication, a pre-training classification model built in software is called for classification and identification, and if the identification result is 'forward' action, a door opening instruction is triggered; the user moves away from the door within the range of 2 meters, and if the recognition result is 'backward' action, a door closing instruction is triggered; similarly, when the user performs other control actions within the range of 5 meters, the software recognizes the other control actions as corresponding actions, and triggers corresponding instructions to control related equipment and facilities.
(7) The software can execute a plurality of commands in parallel, and controls doors, lamps, televisions, air conditioners and the like of different rooms to execute respective commands according to the identity numbers of the millimeter wave radar terminals, so that the commands are not interfered with each other, a plurality of control personnel can simultaneously make control actions at different positions to trigger corresponding control commands, and intelligent home control is realized.
The system structure of this embodiment is as shown in fig. 2, and is composed of a millimeter wave radar sensor (component 1), signal processing modules (component 2) such as an MCU and a DSP, a central processing control device (component 3), and an intelligent home device facility (component 4). In the structure, the millimeter wave radar sensor is used for transmitting Linear Frequency Modulation Continuous Wave (LFMCW) signals to a radar irradiation scene, receiving radar echo signals reflected by the scene, mixing the echo and the transmitting signals, and then performing band-pass filtering and analog-to-digital conversion (ADC) sampling to obtain intermediate-frequency original data; the signal processing module is used for carrying out a series of signal processing on the original data to obtain characteristic data and transmitting the characteristic data to the central processing control equipment in a wired or wireless communication mode; the central processing control equipment is used for providing a graphical user interface, networking, controlling, monitoring and communicating other components, performing off-line training and classification on the millimeter wave radar characteristic data in the central processing control equipment, activating a corresponding control instruction according to a classification result, and controlling corresponding intelligent household equipment facilities to react.
The above-described embodiment is only one example of the present invention, and the present invention is not limited to this example. Without making any substantial innovation, the scope of the invention is intended to be covered by the following claims.
Claims (10)
1. An intelligent home control system based on a millimeter wave radar is characterized by comprising a millimeter wave radar subsystem, a signal processing subsystem, an artificial intelligence classification subsystem and a central control subsystem; the millimeter wave radar subsystem is used for transmitting linear frequency modulation continuous wave signals to a radar irradiation scene, receiving radar echo data reflected by the scene, mixing the echo with the transmitting signals, and then performing band-pass filtering and analog-digital conversion sampling to obtain intermediate frequency original data; the signal processing subsystem is used for carrying out a series of signal processing on the original data to obtain characteristic data and transmitting the characteristic data to the artificial intelligent classification subsystem in a wired or wireless communication mode; the artificial intelligence classification subsystem is used for performing off-line training and classification on the millimeter wave radar characteristic data and transmitting a classification result to the central control subsystem in a wired or wireless communication mode; the central control subsystem is used for controlling, monitoring and communicating other systems in real time, activating corresponding control instructions according to classification results and controlling corresponding intelligent household hardware facilities to react;
the data flow of the system is as follows: the millimeter wave radar subsystem collects original data containing human body action information, transmits the original data to a signal processing subsystem formed by an MCU (microprogrammed control unit) and a DSP (digital signal processor) module for data processing, transmits obtained characteristic data to an artificial intelligent classification subsystem through a wired or wireless communication interface, the artificial intelligent classification subsystem calls a pre-trained artificial neural network classification model to classify the characteristic data and transmits a classification result to a central control subsystem, the central control subsystem is communicated with all controlled terminal facilities, corresponding control commands are sent to the terminal facilities according to the classification result, and the terminal facilities react according to the control commands.
2. The smart home control system based on millimeter wave radar of claim 1, wherein the millimeter wave radar subsystem comprises a millimeter wave radar chip, a transceiver antenna assembly, an MCU, a communication assembly, a power module and a peripheral circuit; the millimeter wave radar chip assembly has the functions of signal generation, signal reception, frequency multiplication, frequency mixing, filtering, analog-digital conversion, data caching and a communication interface; the receiving and transmitting antenna assembly adopts a multi-input multi-output antenna array, which is called an MIMO antenna array for short; the MCU is used for controlling the radar, configuring parameters and preprocessing data; the communication component is used for communication and data transmission with other subsystems; the power supply module is used for supplying power to the whole millimeter wave radar subsystem; the peripheral circuits are used to connect the various modules, components and external subsystems.
3. The smart home control system based on millimeter wave radar according to claim 2, wherein the millimeter wave radar subsystem has a work flow of:
the central control subsystem sends radar configuration parameters and a starting command to the millimeter wave radar subsystem MCU, the MCU controls the millimeter wave radar to periodically transmit signals at the transmitting and receiving front end and receive echoes reflected by a radar irradiation scene, and the time delay between the echo signals and the transmitted signals is delta tdCarrier center frequency of fc(ii) a The transmitting signal adopts a linear continuous frequency modulation wave, and the expression is
Wherein B is the signal bandwidth, and T is the sweep frequency period; the expression of the transmitting signal and the receiving signal is as follows:
sT(t)＝ATcos2π[fct+∫0 tfT(τ)dτ]
sR(t)＝ARcos2π[fc(t-Δtd)+∫0 tfR(τ)dτ]
wherein A isTAnd ARRepresenting the amplitude of the transmitted and received signals, respectively, fR(τ) is the received signal frequency, expressed asWherein Δ fdIs a Doppler shift;
the echo signal and the transmitting signal are subjected to frequency mixing and filtering in a millimeter wave radar chip to obtain an intermediate frequency signal, and the expression is as follows:
and the intermediate frequency signal is sampled by an ADC (analog to digital converter) to obtain original digital intermediate frequency signal data, which are called original data for short.
4. The millimeter-wave radar-based smart home control system of claim 1, wherein the signal processing subsystem comprises a DSP module, an MCU module and a communication module; the DSP module and the MCU module are used for processing the signals of the original data, and specifically comprise: the millimeter wave radar chip transmits the acquired original data to the MCU module, the MCU module and the DSP module perform data processing on the original data and transmit processing process data to each other, the data processing mainly comprises data arrangement, basic mathematical operation, fast Fourier transform, constant false alarm rate detection and arrival angle calculation, and the characteristic data obtained by the data processing are sent to the artificial intelligence classification subsystem.
5. The smart home control system based on the millimeter wave radar of claim 1, wherein the central control subsystem has high-speed read-write capability, data storage capability, complex computation capability, neural network processing and computation capability, real-time classification and identification capability, and real-time communication control capability; when the central control subsystem identifies an effective human-computer interaction control action, corresponding terminal hardware is controlled to make a corresponding response; the central control subsystem is provided with a real-time GUI operation display interface; the central control subsystem is communicated with the MCU in the millimeter wave radar subsystem in a wired or wireless mode and is used for controlling the starting, the closing, the parameter configuration and the signal receiving and sending of the radar sensor, and is used for further processing the characteristic data obtained in the signal processing subsystem, calling an artificial neural network classification model obtained by the artificial intelligent classification subsystem to classify and identify, and controlling related hardware terminal equipment in a wired or wireless communication mode according to an instruction function corresponding to a classification result.
6. The millimeter-wave radar-based smart home control system of claim 1, wherein the artificial intelligence classification subsystem includes functions of control action design, artificial neural network design, feature design and optimization, and sample data rejection and optimization.
7. The millimeter wave radar-based smart home control system and method according to claim 6, wherein the control action is designed based on a principle of combining a real life scene, correlation with a control function, high feature discrimination, smoothness and naturalness;
the artificial neural network design comprises a network structure, a network layer type, an input/output size and various parameter designs, and is designed according to the performance, the real-time requirement and the operation complexity requirement of a hardware terminal where the artificial intelligent classification subsystem is located, and the indexes of training time consumption, model size, classification time consumption and CPU and memory occupation are comprehensively considered.
8. The millimeter-wave radar-based smart home control system of claim 1, wherein the millimeter-wave radar subsystem and the signal processing subsystem are integrated in the same hardware terminal to achieve high-speed data transmission of raw data.
9. The millimeter-wave radar-based smart home control system of claim 1, wherein the artificial intelligence classification subsystem and the central control subsystem are built on an embedded processor, a mobile phone, a computer or a cloud server.
10. An intelligent home control method based on the intelligent home control system according to claim 1, comprising the following specific steps:
designing a man-machine interaction control action, millimeter wave radar parameters and an artificial neural network structure and parameters according to an application scene;
step two, the human body performs predefined actions in an application scene, the millimeter wave radar subsystem transmits linear frequency modulation continuous wave signal signals to a radar irradiation scene, receives radar echo signals reflected by the scene, performs band-pass filtering and ADC (analog to digital converter) sampling after mixing echoes and the transmitting signals to obtain intermediate-frequency original data, and transmits the original data to the signal processing subsystem;
thirdly, the signal processing subsystem carries out a series of data processing on the original data, extracts characteristic data related to human body actions and transmits the characteristic data to the artificial intelligence classification subsystem;
step four, repeating the process of the step one and the process of the step two to obtain a plurality of characteristic data samples of predefined actions, generating an off-line training sample data set in an artificial intelligence classification subsystem, screening and eliminating the training sample data set, then carrying out deep learning training and generating a classification model;
installing terminal nodes integrated by the millimeter wave radar subsystem and the signal processing subsystem at proper positions of each room, and establishing communication connection between the terminal nodes and the artificial intelligence classification subsystem and the central control system as well as between the central control subsystem and hardware equipment facilities in a wired or wireless communication mode;
step six, repeating the step one and the step two, calling the classification model generated in the step four in the artificial intelligence classification subsystem to perform predefined action recognition, and transmitting the recognition result to the central control subsystem;
and step seven, the central control subsystem sends a corresponding control instruction to the controlled hardware terminal facility according to the predefined action category corresponding to the identification result, and if the identification result is not the predefined action category, the step six is returned.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911052118.8A CN110687816A (en) | 2019-10-31 | 2019-10-31 | Intelligent household control system and method based on millimeter wave radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911052118.8A CN110687816A (en) | 2019-10-31 | 2019-10-31 | Intelligent household control system and method based on millimeter wave radar |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110687816A true CN110687816A (en) | 2020-01-14 |
Family
ID=69115082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911052118.8A Pending CN110687816A (en) | 2019-10-31 | 2019-10-31 | Intelligent household control system and method based on millimeter wave radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110687816A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111409584A (en) * | 2020-02-27 | 2020-07-14 | 广汽蔚来新能源汽车科技有限公司 | Pedestrian protection method, device, computer equipment and storage medium |
| CN111738060A (en) * | 2020-05-07 | 2020-10-02 | 复旦大学 | Human gait recognition system based on millimeter wave radar |
| CN112130133A (en) * | 2020-08-26 | 2020-12-25 | 南京熊猫电子制造有限公司 | Man-machine interaction method based on millimeter wave radar |
| CN112158215A (en) * | 2020-09-21 | 2021-01-01 | 五邑大学 | Intelligent air conditioner control system based on accurate sensing of millimeter wave radar and control method thereof |
| CN112444805A (en) * | 2020-11-01 | 2021-03-05 | 复旦大学 | Distributed multi-target detection, positioning tracking and identity recognition system based on radar |
| CN113138557A (en) * | 2020-01-17 | 2021-07-20 | 北京小米移动软件有限公司 | Household equipment control method and device and storage medium |
| US20220113394A1 (en) * | 2020-10-13 | 2022-04-14 | Google Llc | Smart-Device-Based Radar System Performing Location Tagging |
| CN115529204A (en) * | 2022-10-08 | 2022-12-27 | 东莞理工学院 | Indoor control system and method |
| WO2022266931A1 (en) * | 2021-06-24 | 2022-12-29 | 京东方科技集团股份有限公司 | Interaction control apparatus and interaction system |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105786185A (en) * | 2016-03-12 | 2016-07-20 | 浙江大学 | Non-contact type gesture recognition system and method based on continuous-wave micro-Doppler radar |
| CN106408915A (en) * | 2016-10-28 | 2017-02-15 | 四川长虹电器股份有限公司 | Radar-wave-based gesture remote-control system and working method thereof |
| US20170168586A1 (en) * | 2015-12-15 | 2017-06-15 | Purdue Research Foundation | Method and System for Hand Pose Detection |
| CN108344995A (en) * | 2018-01-25 | 2018-07-31 | 宁波隔空智能科技有限公司 | A kind of gesture identifying device and gesture identification method based on microwave radar technology |
| US20180322351A1 (en) * | 2016-11-21 | 2018-11-08 | George Shaker | System and method for sensing with millimeter waves |
| CN108805112A (en) * | 2018-09-18 | 2018-11-13 | 深圳大学 | A kind of motion recognition system combined based on machine learning and radar |
| CN109032349A (en) * | 2018-07-10 | 2018-12-18 | 哈尔滨工业大学 | A kind of gesture identification method and system based on millimetre-wave radar |
| WO2019006473A1 (en) * | 2017-06-30 | 2019-01-03 | The Johns Hopkins University | Systems and method for action recognition using micro-doppler signatures and recurrent neural networks |
| CN109188414A (en) * | 2018-09-12 | 2019-01-11 | 北京工业大学 | A kind of gesture motion detection method based on millimetre-wave radar |
| CN109583436A (en) * | 2019-01-29 | 2019-04-05 | 杭州朗阳科技有限公司 | A kind of gesture recognition system based on millimetre-wave radar |
| CN109857251A (en) * | 2019-01-16 | 2019-06-07 | 珠海格力电器股份有限公司 | Gesture identification control method, device, storage medium and the equipment of intelligent appliance |
| CN110045370A (en) * | 2019-05-10 | 2019-07-23 | 成都宋元科技有限公司 | Human perception method and its system based on millimetre-wave radar |
| CN110348288A (en) * | 2019-05-27 | 2019-10-18 | 哈尔滨工业大学（威海） | A kind of gesture identification method based on 77GHz MMW RADAR SIGNAL USING |
-
2019
- 2019-10-31 CN CN201911052118.8A patent/CN110687816A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170168586A1 (en) * | 2015-12-15 | 2017-06-15 | Purdue Research Foundation | Method and System for Hand Pose Detection |
| CN105786185A (en) * | 2016-03-12 | 2016-07-20 | 浙江大学 | Non-contact type gesture recognition system and method based on continuous-wave micro-Doppler radar |
| CN106408915A (en) * | 2016-10-28 | 2017-02-15 | 四川长虹电器股份有限公司 | Radar-wave-based gesture remote-control system and working method thereof |
| US20180322351A1 (en) * | 2016-11-21 | 2018-11-08 | George Shaker | System and method for sensing with millimeter waves |
| WO2019006473A1 (en) * | 2017-06-30 | 2019-01-03 | The Johns Hopkins University | Systems and method for action recognition using micro-doppler signatures and recurrent neural networks |
| CN108344995A (en) * | 2018-01-25 | 2018-07-31 | 宁波隔空智能科技有限公司 | A kind of gesture identifying device and gesture identification method based on microwave radar technology |
| CN109032349A (en) * | 2018-07-10 | 2018-12-18 | 哈尔滨工业大学 | A kind of gesture identification method and system based on millimetre-wave radar |
| CN109188414A (en) * | 2018-09-12 | 2019-01-11 | 北京工业大学 | A kind of gesture motion detection method based on millimetre-wave radar |
| CN108805112A (en) * | 2018-09-18 | 2018-11-13 | 深圳大学 | A kind of motion recognition system combined based on machine learning and radar |
| CN109857251A (en) * | 2019-01-16 | 2019-06-07 | 珠海格力电器股份有限公司 | Gesture identification control method, device, storage medium and the equipment of intelligent appliance |
| CN109583436A (en) * | 2019-01-29 | 2019-04-05 | 杭州朗阳科技有限公司 | A kind of gesture recognition system based on millimetre-wave radar |
| CN110045370A (en) * | 2019-05-10 | 2019-07-23 | 成都宋元科技有限公司 | Human perception method and its system based on millimetre-wave radar |
| CN110348288A (en) * | 2019-05-27 | 2019-10-18 | 哈尔滨工业大学（威海） | A kind of gesture identification method based on 77GHz MMW RADAR SIGNAL USING |
Non-Patent Citations (3)
| Title |
|---|
| WANG GUOCHAO,ET AL.: "A hybrid FMCW-Interferometry Radar for Indoor Precise Positioning and Versatile Life Activity Monitoring", 《IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES》 * |
| 王俊等: "基于卷积神经网络的手势动作雷达识别方法", 《北京航空航天大学学报》 * |
| 王勇 等: "基于FMCW雷达的多维参数手势识别算法", 《电子与信息学报》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113138557A (en) * | 2020-01-17 | 2021-07-20 | 北京小米移动软件有限公司 | Household equipment control method and device and storage medium |
| US11831459B2 (en) | 2020-01-17 | 2023-11-28 | Beijing Xiaomi Mobile Software Co., Ltd. | Control method and device for home device and storage medium |
| CN113138557B (en) * | 2020-01-17 | 2023-11-28 | 北京小米移动软件有限公司 | Household equipment control method and device and storage medium |
| CN111409584A (en) * | 2020-02-27 | 2020-07-14 | 广汽蔚来新能源汽车科技有限公司 | Pedestrian protection method, device, computer equipment and storage medium |
| CN111738060A (en) * | 2020-05-07 | 2020-10-02 | 复旦大学 | Human gait recognition system based on millimeter wave radar |
| CN112130133A (en) * | 2020-08-26 | 2020-12-25 | 南京熊猫电子制造有限公司 | Man-machine interaction method based on millimeter wave radar |
| CN112158215A (en) * | 2020-09-21 | 2021-01-01 | 五邑大学 | Intelligent air conditioner control system based on accurate sensing of millimeter wave radar and control method thereof |
| US20220113394A1 (en) * | 2020-10-13 | 2022-04-14 | Google Llc | Smart-Device-Based Radar System Performing Location Tagging |
| CN112444805A (en) * | 2020-11-01 | 2021-03-05 | 复旦大学 | Distributed multi-target detection, positioning tracking and identity recognition system based on radar |
| WO2022266931A1 (en) * | 2021-06-24 | 2022-12-29 | 京东方科技集团股份有限公司 | Interaction control apparatus and interaction system |
| CN115529204A (en) * | 2022-10-08 | 2022-12-27 | 东莞理工学院 | Indoor control system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110687816A (en) | Intelligent household control system and method based on millimeter wave radar | |
| CN110765974B (en) | Micro gesture recognition method based on millimeter wave radar and convolutional neural network | |
| US11061115B2 (en) | Method for gesture recognition, terminal, and storage medium | |
| WO2021218753A1 (en) | Gesture recognition method and related apparatus | |
| CN106446801B (en) | Micro-gesture recognition method and system based on ultrasonic active detection | |
| CN112034446A (en) | Gesture recognition system based on millimeter wave radar | |
| Wu et al. | Dynamic hand gesture recognition using FMCW radar sensor for driving assistance | |
| Xia et al. | Time-space dimension reduction of millimeter-wave radar point-clouds for smart-home hand-gesture recognition | |
| CN108344996B (en) | Panel switch based on microwave radar gesture recognition technology and control method thereof | |
| CN114390463B (en) | Indoor multi-target passive positioning method and system and electronic equipment | |
| CN112444805A (en) | Distributed multi-target detection, positioning tracking and identity recognition system based on radar | |
| CN115343704A (en) | Gesture recognition method of FMCW millimeter wave radar based on multi-task learning | |
| Gan et al. | Gesture recognition system using 24 GHz FMCW radar sensor realized on real-time edge computing platform | |
| CN113064483A (en) | Gesture recognition method and related device | |
| Jin et al. | Interference-robust millimeter-wave radar-based dynamic hand gesture recognition using 2D CNN-transformer networks | |
| Li et al. | A trajectory-based gesture recognition in smart homes based on the ultrawideband communication system | |
| CN116125458A (en) | Personnel positioning method based on millimeter wave radar | |
| CN114708663A (en) | Millimeter wave radar sensing gesture recognition method based on few-sample learning | |
| Chen et al. | A hand gesture recognition method for Mmwave radar based on angle-range joint temporal feature | |
| Khawaja et al. | UWB radar based beyond wall sensing and tracking for ambient assisted living | |
| Chmurski et al. | Novel Radar-based Gesture Recognition System using Optimized CNN-LSTM Deep Neural Network for Low-power Microcomputer Platform. | |
| CN112731332A (en) | Millimeter wave-based static target existence identification method and system | |
| Wu et al. | A mini radar system for low altitude targets detection | |
| Wu et al. | Deep learning-based uav detection in the low altitude clutter background | |
| Trinh et al. | Hand Gesture Recognition using MIMO Radar and Lightweight Convolutional Neural Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200114 |
|
| WD01 | Invention patent application deemed withdrawn after publication |