CN106331602A - Home monitoring system based on infrared thermal imaging technology - Google Patents

Abstract

The present invention relates to the field of smart home. It is a home monitoring system based on infrared thermal imaging technology, mainly including cloud server, central server and terminal equipment. Motion commands, upload real-time data to the cloud server and receive the information issued by the cloud server; the cloud server forwards the received infrared video information to the user terminal, and accepts the query information and command information of the user terminal; the invention solves the problem of traditional video surveillance The technical problems of low system security and disrespect for user privacy; the present invention solves technical problems such as poor monitoring effect and small monitoring range of most video surveillance systems at night; the present invention uses the latest algorithm to solve the technical problem of real-time tracking of targets; The invention adopts the latest data analysis and data mining technology, and solves the technical problem of automatic alarm for suspicious behaviors.

Description

A Home Monitoring System Based on Infrared Thermal Imaging Technology

technical field

The invention belongs to the technical field of home monitoring, in particular to a home monitoring system based on infrared thermal imaging technology.

Background technique

The traditional video surveillance system can achieve real-time monitoring, and users can use various mobile terminals to see real-time video information anytime and anywhere. The traditional video surveillance system collects video signals by installing cameras in the user's home, transmits the data to the central server through wireless communication or wired communication, and then forwards the data to the cloud server by the central server, or directly uploads the data to the cloud server by using a network camera. Each terminal delivered by the cloud server. Such cameras are also usually able to change poses, but require human manipulation.

This kind of monitoring system collects clear video images, and sometimes even clear face images, and users actually live in an environment without privacy. Traditional video surveillance systems generally use network cameras, which use open output formats and transmission protocols, and video information is easily intercepted by those with ulterior motives, so this kind of surveillance system will become a useful tool for criminals. The traditional video surveillance system is greatly affected by the external environment, and the image quality will be greatly reduced in the case of insufficient light.

Contents of the invention

In order to overcome the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a home monitoring system based on infrared thermal imaging technology, which solves the problems of traditional video monitoring systems not protecting user privacy and being greatly affected by light.

In order to achieve the above object, the technical scheme adopted in the present invention is:

A home monitoring system based on infrared thermal imaging technology, including:

The infrared thermal imaging unit is used to collect indoor infrared video signals and transmit the collected signals to the central server;

The central server receives the infrared video signal collected by the infrared thermal imaging unit and sends it out through the cloud server;

The terminal equipment receives the data sent by the central server for transfer, storage and analysis.

The infrared thermal imaging unit includes a circuit module and a mechanical module, the circuit module is used for collecting infrared video signals and sending them to the outside, and the mechanical module is used for motion control of the circuit module.

The circuit module includes a communication module and an infrared video signal acquisition module. The infrared video signal acquisition module takes an infrared thermal imaging sensor as the core, and the electrical signal generated by the sensor forms digital information, which is uploaded to the central server through the communication module; the mechanical module has three degrees of freedom, according to the command to fix the infrared thermal imaging sensor to any desired posture in three-dimensional space.

The mechanical module has two rotating shafts, and the rotating shafts are perpendicular to each other. The rotation angle range of the main shaft is [-179,179], and the rotation angle range of the secondary shaft is [-90,90]. In the case of no obstacle occlusion, it can monitor the three-dimensional space with approximately zero dead angle.

The central server includes a central processing module, a communication module and a built-in following control algorithm module;

The central processing module is used to process the data information of each module and control other modules;

The communication module is used to receive infrared video data and perform corresponding preprocessing, and is also used to receive position status information of the infrared thermal imaging unit and issue control commands;

The built-in following control algorithm module is used to control the motor to change the posture of the infrared thermal imaging unit after the target is locked, so as to track and monitor the target.

The identity mark is set in the central processing module for registering an account in the cloud server and binding with the terminal device, and the built-in following control algorithm module adopts the mean shift algorithm combined with Kalman filter to realize target tracking.

On the basis of target tracking, network communication is used to realize the information interaction of each sensor, so as to realize the overall linkage of the sensor group:

Firstly, camera 1 keeps track of the target object in the image frame captured by area 1, and transmits the feature information of the object to the server; after that, the server notifies other cameras to perform cooperative tracking, and other cameras run background subtraction at the same time The algorithm detects objects, and feeds back the detection result, that is, the difference between the current frame and the background frame obtained by subtracting the background, to the server. The server judges which camera to continue tracking based on the data fed back from each camera.

The terminal equipment is a mobile phone, a tablet or a PC, including a UI interface module, a security alarm response module and a communication module, wherein:

The UI interface module is used to display real-time infrared video information and respond to user commands to the user. The UI interface includes a user account login sub-interface, and the main interface includes video controls, alarm display, and several user input buttons;

The security alarm response module is used to send an alarm reminder to the user after receiving the alarm command issued by the cloud server, including calling the terminal speaker to sound, calling the vibration or page pop-up function;

The communication module undertakes the data exchange task between the terminal device and the cloud server.

Described cloud server comprises hardware configuration and software design;

The hardware configuration is used to configure and set the server hardware. It is the carrier of software and algorithm operation and the storage medium. The server deployment adopts a cluster strategy to improve reliability and computing performance. The data storage adopts a virtualized storage strategy, and each server shares data. ;

The software is designed to receive uploaded data from the home center server, issue user terminal commands to it, manage data under user accounts, receive user terminal requests and forward real-time data, and its built-in algorithm includes identifying suspicious targets, Identify suspicious high temperatures in households, analyze and mine historical data, and extract user behavior characteristics.

The identification of the target identity adopts a strategy of combining face recognition and behavior recognition.

Compared with the prior art, the present invention has the advantages of:

1. The present invention solves the problem that the traditional video surveillance system does not protect user privacy. The image acquisition device used in the system is a thermal imaging camera. The video image cannot clearly see the face, or even whether it is wearing clothes, and effectively protects user privacy.

2. The present invention solves the problem that the traditional video monitoring system is greatly affected by light. Objects above absolute zero will radiate infrared rays. The infrared thermal imaging camera can capture the infrared rays radiated by objects to form useful image information.

3. The present invention solves the problem that the traditional video surveillance system needs a special person to control the camera posture. This system uses a target tracking algorithm. Once the entrant is determined to be suspicious, the camera will lock on the target and turn on the conventional camera to take a clear face image.

4. The present invention adopts big data analysis technology to analyze the user's daily behavior data, dig out the user's behavioral characteristics, compare the behavioral characteristics of the intruder, and realize functions such as automatic alarm and camera locking.

5. The present invention can also alarm the abnormal high temperature in the home. For example, after the water in the pot is boiled dry, the temperature will be much higher than normal. After the thermal imaging camera captures the abnormal high temperature, it can alarm. The general fire alarm system can also capture and send out alarm.

Description of drawings

Fig. 1 is a system structure diagram of the present invention.

Fig. 2 is a schematic diagram of the mechanical module control method of the present invention.

Fig. 3 is a schematic diagram of the target tracking method of the present invention.

Fig. 4 is a schematic diagram of the working control method of the sensor group in the present invention.

Fig. 5 is a schematic diagram of the face recognition process of the DeepID algorithm of the present invention.

Fig. 6 is a schematic diagram of the process of behavior recognition of the DBNs algorithm of the present invention.

Fig. 7 is a schematic diagram of the RBM of the present invention.

Fig. 8 is a schematic diagram of the RBM of the present invention forming a DBNs.

detailed description

The implementation of the present invention will be described in detail below in conjunction with the drawings and examples.

As shown in Figure 1, a home monitoring system based on infrared thermal imaging technology includes:

The infrared thermal imaging unit is used to collect indoor infrared video signals and transmit the collected signals to the central server;

The central server receives the infrared video signal collected by the infrared thermal imaging unit and sends it out through the cloud server;

The terminal equipment receives the data sent by the central server for transfer, storage and analysis.

Among them, the infrared thermal imaging unit includes a circuit module and a mechanical module. The circuit module is used to collect infrared video signals and send them to the outside, including a communication module and an infrared video signal collection module. The infrared video signal collection module takes the infrared thermal imaging sensor as the core. The generated electrical signal forms digital information, which is uploaded to the central server through the communication module.

The mechanical module is used for the motion control of the circuit module, which has three degrees of freedom, and fixes the infrared thermal imaging sensor to any required posture in three-dimensional space according to the command. Specifically, it has two rotation axes, the rotation axes are perpendicular to each other, the rotation angle range of the main rotation axis is [-179,179], and the rotation angle range of the secondary rotation axis is [-90,90], supplemented by the field of view of the infrared thermal imaging sensor, In the case of no obstacle occlusion, it can realize the monitoring of nearly zero dead angle in three-dimensional space. For a given target coordinate, the infrared thermal imaging sensor can quickly reach the required posture through the action of the primary and secondary rotation axes; for continuous target coordinates, the infrared thermal imaging sensor can always face the target through the continuous movement of the primary and secondary rotation axes. When the target is tracking, the center The data sent by the server to the infrared thermal imaging sensor contains a set of continuous coordinate values. The linkage of primary and secondary shafts is realized by digital integral interpolation method, as shown in Figure 2.

When the main/secondary axis pulse is output, the main/secondary servo motor moves at a certain angle, and finally reaches the given target coordinates. The action of the servo motor is measured by the angle, and the movement of the object in the three-dimensional space is measured by the distance. Here is a mapping from the three-dimensional coordinates to the two-dimensional angular coordinates. Full mapping:

$\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}\\ \mathrm{<span\; class="notranslate">\; \&theta;</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\left(\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\end{array}\right)<span\; class="notranslate">\; \&Center\; Dot;</span>\left(\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\\ {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}\end{array}\right)<span\; class="notranslate">\; \&Center\; Dot;</span>\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; z</span>}\end{array}\right)$

in Indicates the main axis angle and the secondary axis angle, K is the adjustment parameter, is the external parameter matrix, is the internal parameter matrix, is the three-dimensional coordinates of the space.

The central server of the present invention includes a central processing module, a communication module and a built-in following control algorithm module, wherein:

The central processing module is used to process the data information of each module and control other modules; the central processing module is also equipped with an identity mark, which is used to register an account in the cloud server and bind with the terminal device.

The communication module is used to receive infrared video data and perform corresponding preprocessing, and is also used to receive position status information of the infrared thermal imaging unit and issue control commands;

The built-in following control algorithm module is used to control the motor to change the attitude of the infrared thermal imaging unit after locking the target, so as to track and monitor the target. In an example of the present invention, a mean-shift algorithm (hereinafter referred to as the Mean-shift algorithm) combined with a Kalman filter is used to realize target tracking. The mean-shift algorithm is a non-parametric method based on density gradient ascent, which relies entirely on the sample points in the feature space for analysis, does not require prior knowledge, and has a fast convergence speed. However, in actual monitoring, the spatial scale of a moving target will change with its own displacement, and the Mean-shift algorithm is based on a fixed window for target search and tracking, so tracking will fail due to inaccurate center positioning. Kalman filtering can predict the starting center of the target in the Mean-shift algorithm, which greatly improves the adaptability and robustness of the Mean-shift algorithm. The flow chart shown in Figure 3 clarifies the target tracking strategy in this example.

On the basis of target tracking, network communication is used to realize the information interaction of each sensor, so as to realize the overall linkage of the sensor group:

The data transmission between the server and the imaging sensor is carried out through the socket. As shown in Figure 4, firstly, the target object in the image captured by area 1, camera 1 keeps track of the object, and transmits the feature information of the object to the server; after that, the server notifies other cameras to carry out cooperative tracking, and other The camera runs the background subtraction algorithm to detect objects at the same time, and feeds back the detection result, that is, the difference between the current frame and the background frame obtained after background subtraction, to the server. The server judges which camera to continue with based on the data fed back from each camera. to track.

The terminal device of the present invention can be a mobile phone, a tablet or a PC, and includes a UI interface module, a security alarm response module and a communication module, wherein:

The UI interface module is used to display real-time infrared video information to the user and respond to user commands. The UI interface includes a user account login sub-interface, and the main interface includes video controls, alarm display, and several user input buttons;

The security alarm response module is used to send an alarm reminder to the user after receiving the alarm command issued by the cloud server, including calling the terminal speaker to sound, calling the vibration or page pop-up function;

The communication module is responsible for the data exchange task between the terminal device and the cloud server.

The cloud server of the present invention includes hardware configuration and software design;

Hardware configuration is used to configure and set server hardware. It is the carrier of software and algorithm operation and the storage medium. Server deployment adopts cluster strategy to improve reliability and computing performance. Data storage adopts virtualization storage strategy, and each server shares data;

Software design, used to receive uploaded data from the home center server, issue user terminal commands to it, manage data under user accounts, receive requests from user terminals and forward real-time data. Its built-in algorithms include identifying suspicious targets, Identify suspicious high temperatures, analyze and mine historical data, and extract user behavior characteristics.

Among them, the identification of the target identity adopts the strategy of combining face recognition and behavior recognition. Face recognition uses the DeepID face recognition algorithm, which is a deep learning algorithm based on convolutional neural networks. The basic work of face recognition is to judge whether two pictures are the same person. The role of convolutional neural network in DeepID is to learn features, input pictures, learn a 160-dimensional vector, and then use various existing classifier to get the result. As shown in Figure 5, the complete DeepID algorithm face recognition process is illustrated.

Behavior recognition uses the deep belief network algorithm (hereinafter referred to as the DBNs algorithm). The DBNs algorithm is a probabilistic generation model, as shown in Figure 6. Compared with the neural network of the traditional discriminant model, the generation model is to establish a relationship between observation data and labels. joint distribution of .

Restricted Boltzmann Machine (hereinafter referred to as RBM) is an important component of the DBNs algorithm. RBM is composed of an explicit layer and a hidden layer. There is no connection. V is the visible layer and H is the hidden layer. A typical RBM is an undirected cyclic graph whose energy function is defined as follows:

E(x,h)=-b'x-c'x-x'Wh

Figure 7 shows a typical RBM structure. DBNs is a probabilistic model containing multiple hidden layers, which can be seen as the accumulation of multiple RBMs. The output of each underlying RBM is used as input data for training the next RBM. Greedy learning obtains a set of RBMs, which can form a DBNs, as shown in Figure 8.

Behavioral learning is performed using a greedy layer-by-layer training algorithm. The main idea of the greedy unsupervised learning algorithm is to conduct unsupervised learning for each layer in DBNs, and finally supervise and fine-tune the entire network. The learning process includes three processes: pre-training, encoding and decoding, and fine-tuning. In the pre-training stage, the next layer and the upper layer constitute a typical RBM, using unsupervised learning to adjust the parameters of the network, so that the output of the RBM can accurately or approximately describe the input, so that it can reach a balanced state. Then the output of the next layer is used as the input of the previous layer, and a new RBM is formed with the upper layer, and the parameters are adjusted to make the RBM reach a balance. Repeat this until the last layer. The process of using trained DBNs to recognize objects is called encoding and decoding. After the unsupervised training and learning are completed, the entire network is supervised through the original input and the final output, and the weight of each layer is adjusted. This process is called fine-tuning.

Based on the foregoing methods and principles, in one or more embodiments, thermal imaging sensors can be deployed in every corner of the user's home, a central server can be deployed in the user's home, and a cloud server can be configured remotely.

Thermal imaging sensors collect infrared video signals through sensitive components to form thermal image data. In one example, these data can be transmitted to the central server through wireless communication, such as ZigBee or wireless WiFi; in another example, these data can be transmitted to the central server through wired communication.

The central server receives these data and performs preliminary processing. If there is an abnormally high temperature, it will directly control the action device to alarm. In one example, it is a bell, and in another example, it is a flashing light. And send an abnormal high temperature alarm to the cloud server.

In one example, the central server encrypts the data so that even if the data is intercepted, it cannot be parsed.

In another example, the central server will mark the data with the user's unique identifier to ensure that the data is sent to the corresponding user account on the cloud server and can only be accessed by the user's registered device.

In one instance, the central server compresses the data, improving transmission efficiency and reducing required bandwidth.

The cloud server receives and stores the thermal image data from the central server.

In one instance, the cloud server analyzes historical data to mine user behavior characteristics. Comparing the behavior characteristics of the entrant, confirming the suspicious person, notifying the central server to lock the target, and notifying the user terminal, the user terminal generates an alarm after receiving the message, calls the terminal to ring in one instance, and pops up an alarm page in another instance.

In one example, the user terminal has an account login system, a video display window, a query button, and a human-controlled camera action button, and in another example, a 3D joystick.

Complete work process of the present invention is:

After the sensor group captures someone entering, it will be tracked by the nearest sensor, and the other sensors will be on standby, and receive deployment commands from the central server at any time for overall linkage to complete collaborative tracking. At the same time, the thermal imaging sensor in the working state uploads thermal image data to the center Server, the server conducts behavior recognition. The result of the recognition is that the visitor is suspicious, and the sensor that is performing the tracking task is notified to turn on the conventional camera to capture the face, upload the image data to the central server for face recognition, and confirm that it is an intruder. Lock the target, send an alarm message to the cloud server and upload the face image, the server notifies the client that someone has broken in and sends the face image, and then the user decides the next action, contact the alarm (relatives who do not come often) or call the police (is a thief).

The sensor sends data to the central server, which includes the orientation and moving speed of the target object. The central server sends the tracking command to the sensor concerned through the target tracking algorithm. The tracking command contains a set of coordinate values, the primary and secondary rotation axes of the sensor According to the coordinate value action, so as to realize the tracking. The cooperative tracking of sensors is realized by using the linkage strategy mentioned above.

The cloud server analyzes the video data on the cloud, extracts user behavior characteristics, and sends the characteristic value when the central server performs behavior recognition, as the judgment basis of the central server.

Claims (10)

1. a home furnishing monitoring system based on infrared thermal imaging technique, it is characterised in that including:

Infrared thermal imaging unit, is used for gathering indoor infrared video signal, and is sent to central server by signal collected；

Central server, the Infrared Video Signal that reception infrared thermal imaging unit is gathered is sent out by Cloud Server；

Terminal unit, receives the data that central server sends, carries out transfer, stores and analyze.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 1, it is characterised in that described infra-red heat Image-generating unit includes circuit module and mechanical module, and circuit module is used for gathering Infrared Video Signal and being sent out, machinery mould Block is for the motor control of circuit module.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 2, it is characterised in that described circuit mould Block includes communication module and Infrared Video Signal acquisition module, and Infrared Video Signal acquisition module with infrared thermal imaging sensor is Core, the signal of telecommunication that sensor produces is formed digital information, is uploaded on central server by communication module；Described machinery mould Block has three degree of freedom, arbitrarily needs attitude according to what infrared thermal imaging sensor was fixed as under three dimensions by order.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 2, it is characterised in that described machinery mould Block has two rotating shafts, and rotating shaft is mutually perpendicular to, and the rotation angle range of main shaft is [-179,179], the anglec of rotation model of secondary shaft Enclose for [-90,90], be aided with the ken of infrared thermal imaging sensor, in the case of clear is blocked, it is achieved to three-dimensional space Between approximation 0 dead angle monitoring.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 1, it is characterised in that genuinely convinced in described Business device includes central processing module, communication module and built-in model-following control algoritic module therein；

Described central processing module is for processing the data message of modules, and is controlled other modules；

Described communication module is used for receiving IR video stream and carrying out corresponding pretreatment, is additionally operable to receive infrared thermal imaging unit Location status information and issue control command；

Described built-in model-following control algoritic module, after lock onto target, controls motor and changes infrared thermal imaging unit attitude, right Target is tracked monitoring.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 5, it is characterised in that described centre Reason module arranges identity, for login account in Cloud Server, binds with terminal unit, described built-in model-following control Algoritic module uses mean shift algorithm to combine Kalman filtering and realizes target following.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 5, it is characterised in that in target following On the basis of to utilize network service to realize the information of each sensor mutual, thus realize the Coupled motion of sensor group:

First target object in the image frame that region 1 photographs, photographic head 1 keeps the tracking to object, and by this object Characteristic information send server to；Hereafter, server notifies that other each photographic head carry out collaborative tracking, and other photographic head are simultaneously Run background subtracting algorithm and carry out object detection, and by the present frame obtained after the result of detection i.e. background subtracting and background frames Difference feeds back to server, and server is according to the data come on each cam feedback, it is judged that by which platform photographic head proceeded Follow the tracks of.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 1, it is characterised in that described terminal sets Standby for mobile phone, flat board or PC, including UI interface module, safety alarm respond module and communication module, wherein:

Described UI interface module, for showing real-time Infrared video information, response user command to user, UI interface includes user The sub-interface of Account Logon, main interface includes video control, alarm displaying, some user's load buttons；

Described safety alarm respond module, carries for user being sent alarm after receiving the alarm command that Cloud Server issues Wake up, including calling terminal speaker sounding, calling vibrations or page ejection function；

Described communication module, undertakes the data switching task of terminal unit and Cloud Server.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 1, it is characterised in that described cloud service Device includes hardware configuration and software design；

Described hardware configuration, arranges server hardware for configuration, is software, the carrier of algorithm operation, is the medium of storage, clothes Business device is disposed and is used cluster strategy, improves reliability and calculated performance, and data storage uses virtualization storage strategy, each server Share data；

Described software design, for receive home centers server upload data, it is issued user terminal order, management use Data under the account of family, receiving the request of user terminal and forward real time data, its built-in algorithms includes the knowledge to suspicious object Not, the identification of high temperature suspicious to family, analysis mining to historical data, extract user behavior feature.

Home furnishing monitoring system based on infrared thermal imaging technique the most according to claim 9, it is characterised in that described to mesh The identification of mark identity uses the strategy that recognition of face and Activity recognition combine.