CN106372576A - Deep learning-based intelligent indoor intrusion detection method and system - Google Patents

Abstract

The invention discloses an intelligent indoor intrusion detection method and system based on deep learning, including: establishing a BP neural network model; using an inter-frame difference algorithm to obtain a difference image between adjacent frames in a monitoring video screen; Carry out binarization processing, and extract the changing foreground area image under the static background in the processed image; Detect and identify whether there is a human figure in the extracted changing foreground image; When identifying the presence of a human figure, from the changing foreground area Detecting and extracting to obtain a face area image; detecting and identifying whether the extracted face area image is a user image; and when identifying and judging that it is a non-user image, determining that it is a non-user intrusion and sending an alarm signal to the user. The invention has strong anti-interference ability of other moving objects, low misjudgment rate, can analyze a large amount of video data, and can accurately detect and identify intrusions.

Description

A method and system for intelligent indoor intrusion detection based on deep learning

technical field

The invention relates to an intelligent indoor intrusion detection method and system based on deep learning, belonging to the technical field of video surveillance.

Background technique

With the development of economy and science and technology, people pay more and more attention to indoor security. Intelligent indoor intrusion detection system has become a trend, and then research on the intelligence, accuracy and real-time performance of intrusion target detection and identification become very meaningful.

In the traditional intelligent monitoring method, a camera and an infrared detector are used to sense whether there is a moving object intruding into its sensing range, and if an intruder is detected, the video information is sent to the mobile terminal.

However, there are defects in the traditional monitoring method. Specifically, the method cannot determine whether the intruder is a user, has low anti-interference ability from other moving objects, high false positive rate, low intelligence, and does not have high-precision detection and identification functions; Due to the limitation of the device, it is impossible to analyze a large amount of video data, and the real-time performance is low. Therefore, there is a problem that users and intruders, moving objects and intruders cannot be accurately detected and identified.

Contents of the invention

The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art, provide an intelligent indoor intrusion detection method and system based on deep learning, and solve the problem that users and intruders, moving objects and intruders cannot be accurately detected and identified.

The present invention specifically adopts the following technical solutions to solve the above technical problems:

A method for intelligent indoor intrusion detection based on deep learning, comprising the following steps:

Establish a BP neural network model, and train the BP neural network model according to the input training data including user images;

Using the inter-frame difference algorithm to obtain the difference image between adjacent frames from the surveillance video screen;

Performing binarization processing on the difference images between the acquired adjacent frames, and extracting the changing foreground area under the static background in the processed image;

Utilize the established BP neural network model to detect and identify whether there is a human figure in the extracted changing foreground image; when identifying that there is a human figure, detect and extract from the changing foreground image to obtain a human face area image;

Utilize the established BP neural network model to detect and recognize whether the extracted face area image is a user image; and when it is recognized and judged as a non-user image, determine that it is a non-user intrusion and send an alarm signal to the user.

Further, as a preferred technical solution of the present invention, the face area image in the method is obtained by the following steps:

Establish a color-based skin color model and normalize skin color similarity;

Use the established skin color model to detect the changing foreground image, and obtain a normalized skin color similarity image;

Setting the skin color similarity threshold, performing binarization on the obtained normalized skin color similarity image to obtain the skin color area;

Select the skin color area according to the set range to obtain the image of the face area.

Further, as a preferred technical solution of the present invention: in the method, the PRP conjugate gradient algorithm is used to train the BP neural network model.

Further, as a preferred technical solution of the present invention: the training data in the method at least includes user's whole body images and user's face images with different poses.

Further, as a preferred technical solution of the present invention: in the method, the training data also includes non-humanoid images. Furthermore, as a preferred technical solution of the present invention: in the method, when it is recognized and judged that there is no human figure, the monitoring video picture is reacquired.

Furthermore, as a preferred technical solution of the present invention: in the method, when the image is identified as a user, it is determined as the user and the monitoring video picture is reacquired according to the set time.

The present invention also proposes an intelligent indoor intrusion detection system based on deep learning, including:

The video monitoring module is used to dynamically monitor the home environment and obtain monitoring video images;

The image acquisition module is used to obtain the differential image between adjacent frames from the monitoring video screen by using the inter-frame difference algorithm;

The image extraction module is used to perform binarization processing on the difference images between the acquired adjacent frames, and extract the image of the changing foreground area under the static background in the processed image;

The recognition analysis module includes a modeling unit, a recognition unit, an area extraction unit and an output unit; wherein, the modeling unit is used to establish a BP neural network model, and to train the BP neural network model according to input training data that includes user images The recognition unit is used to input the extracted change foreground image into the BP neural network model to detect and identify whether there is a human figure; the region extraction unit is used to extract from the change foreground image when the recognition determines that there is a human figure Detecting and extracting to obtain the face area image; the recognition unit is also used to input the extracted face area image into the BP neural network model to detect and identify whether it is a user image; When the user image is determined to be non-user intrusion, and generate and output control signals;

The sending unit is configured to send the intrusion prompt data to the user according to the control signal.

Further, as a preferred technical solution of the present invention: the intrusion prompt data sent by the unit includes an intruder image.

Further, as a preferred technical solution of the present invention: an alarm module is also included, and the alarm module is used to generate an audible and visual alarm according to the control signal.

The present invention adopts above-mentioned technical scheme, can produce following technical effect:

The intelligent indoor intrusion detection method and system based on deep learning provided by the present invention firstly detects the moving target in the monitoring video picture, so no follow-up processing is performed on the video image with the same scene, saving server overhead; and in the system The detection of human figures is added to avoid the interference of non-human factors such as pets and sweeping robots; because the recognition of users is finally converted to the recognition of users' faces, changes in users' clothing and hairstyles do not affect their correct recognition. With good universality, the neural network requires less training data. In addition, the use of infrared cameras to collect images is not limited by light; therefore, the design is more humane, with strong anti-interference ability from other moving objects and low misjudgment rate. It can use high-performance processing methods to analyze a large amount of video data, which is highly efficient. And real-time, can accurately detect and identify users and intruders, moving objects and intruders.

The method and system of the present invention have the following advantages:

(1) The intelligent indoor intrusion detection method and system based on deep learning provided by the present invention adopts the BP neural network based on the PRP conjugate gradient algorithm to improve the network training speed and ensure the correctness of convergence.

(2) The present invention combines the inter-frame difference algorithm and the skin color extraction model to extract the target area hierarchically, with simple calculation and real-time performance.

(3) Extract the detected target area and input it into the neural network, without identifying complex background and other interference factors, to ensure the correctness of classification.

(4) The present invention is based on image processing technology, can complete recognition efficiently, and can save power consumption. It can effectively solve the problem that users and intruders, moving objects and intruders cannot be accurately detected and identified.

Description of drawings

FIG. 1 is a schematic flowchart of the deep learning-based intelligent indoor intrusion detection method of the present invention.

Fig. 2 is a schematic structural diagram of the BP neural network model of the present invention.

Fig. 3 is a schematic diagram of the principle of the intelligent indoor intrusion detection system based on deep learning of the present invention.

detailed description

Embodiments of the present invention will be described below in conjunction with the accompanying drawings.

As shown in Figure 1, the present invention has designed a kind of intelligent indoor intrusion detection method based on deep learning, comprises the following steps:

Step 1. Establish a BP neural network model, and train the BP neural network model according to input training data including user images. In the method, the first time the model is used to input training data, the BP neural network model is trained, and the steps include initializing weights and thresholds, adopting the PRP conjugate gradient algorithm to adjust the weights and thresholds layer by layer, iterating to the maximum iteration frequency. The process is as follows:

Input training data to the BP neural network model using the PRP conjugate gradient algorithm, the training data at least includes user body images and user face images with different postures, preferably, non-human images such as pets, smart vacuum cleaners, etc. .

The iterative process of the BP neural network model is shown in Figure 2. F is a class of training data, and the output of the neural network is y=ψ(W _i X _i -θ), where ψ represents the activation function, and the sigma function can be used; X _i , X _j represents the neuron input vectors of the i-th layer and the j-th layer respectively; W _i represents the weight vector connecting the i-th layer and the next layer, and W _j represents the weight vector connecting the j-th layer and the next layer; θ is threshold. In Figure 2, two hidden layers i and j are used to refer to all hidden layers in the neural network. The number of hidden layers is determined according to the actual situation. Because the iteration method of each layer is the same, the following only refers to the input vector X _l and The weight vector W _l has

In the present invention, the PRP conjugate gradient algorithm is used to converge the weight and threshold. Then the objective function is:

L(W _l , θ)=W _l X _l -θ

Moreover, assuming that the error gradient of the objective function obtained in the kth iteration is E _k , the coefficients can be obtained based on the PRP conjugate gradient algorithm The search direction can thus be determined: p _k =-E _k +β _k p _k-1 . The search step size α _k is obtained by linear search, and the iterative formulas of weight vector and input vector obtained by conjugate gradient algorithm are as follows:

W _l (k+1)=W _l (k)+α _k p _k , k=0, 1,...;

θ(k+1)=θ(k)+α _k p _k , k=0, 1, . . .

In the above formula, W _l (k) represents the weight vector iteration value obtained by the k-th iteration W _l , and θ(k) represents the threshold value iteration value obtained by the k-th iteration θ.

And, the BP neural network learning process of the PRP conjugate gradient algorithm adopted in the present invention is as follows:

First, input: a class of training images F={f ₁ , f ₂ , . . . , f _n }, where F may include full-body images of human figures or user face images with different poses.

Then enter the iterative process, as follows:

1. Initialization: Initialize the weight vector W _l (0) of neurons in each layer, and the threshold θ (0).

2. Adjust the weight value; fix the threshold θ, and use the weight vector iterative method to adjust the weight vector W _l of each layer layer by layer until convergence.

3. Adjust the threshold; fix the weight W _l , and use the threshold iterative method to adjust the threshold θ until convergence.

4. Iteration, repeating processes 2 and 3 until the maximum number of iterations is reached.

Input human-shaped full-body images with different postures to the BP neural network model using the PRP conjugate gradient algorithm, which can be trained and learned so that the model can obtain the function of a human-shaped recognizer, which can identify human-shaped and non-human-shaped areas; and, input the user's face image to After training and learning, the model obtains the face recognition function, which can identify users and non-user intruders.

The resulting BP neural network model can identify users and intruders, human beings with human figures and moving objects without human figures.

Step 2, using the inter-frame difference algorithm to obtain the difference image between adjacent frames from the surveillance video picture. It is to capture multiple frames of images from a surveillance video screen, specifically including the following steps:

Let the input video images of frame p and frame p+1 be f _p (i, j) and f _p+1 (i, j) respectively, where (i, j) represents a pixel point, and the pixels corresponding to the two frames of images Point difference:

D _p (i, j)=|f _p+1 (i, j)-f _p (i, j)|

Then the differential image can be obtained:

Step 3. Binarize the acquired difference images between adjacent frames to obtain a binary image in which the static background is black and the changing foreground area is white. Therefore, the changing foreground area can be extracted to provide for the recognition and detection process . Moreover, for video images with a static background, there is no need to perform subsequent processing on the background area, which can save server overhead.

The foreground image extraction of the changes in the foreground in the image is specifically:

Set the threshold T to obtain the binary image of the changing foreground image:

${\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; T</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; T</span>}\end{array}\right.$

Step 4, using the established BP neural network model to detect and recognize the extracted changing foreground image to determine whether there is a human figure. Specifically include the following steps:

After extracting the image area where the moving target is located in the static background, the extracted image is processed and cropped to match the input image characteristics of the neural network model. Then input it into the BP neural network model, and use the BP neural network model to identify the human form and non-human form of the image. This step uses the detection of the human form to avoid the interference of non-human factors such as pets and sweeping robots. If the BP neural network model outputs y=1, there is a human-shaped area in the changing foreground image, continue to perform face detection and recognition operations on the target area, and enter step 5; otherwise, the BP neural network model outputs y=0, indicating that there is no non-existent For the human-shaped area, do not perform the following step 5, and continue to monitor. That is, when the recognition determines that there is no human figure, it indicates that the non-human body is in an active state. This method can return to step 2 to reacquire the monitoring video screen and repeat steps 3 to 4 to continue video monitoring and recognition until the recognition determines that there is a human figure.

When the BP neural network model outputs y=1, and it is recognized and judged that there is a human figure, the human face area image is obtained by detecting and extracting from the changing foreground image. The image acquisition process of the human face area is as follows:

First, using the skin color model based on the Y Cb Cr color system, the blue chromaticity component value C _b and the red chromaticity component value C _r at the pixel point (i, j) can be obtained; set the chromaticity value x _ij of the pixel point =( C _b , C _r ) ^T , then the skin color similarity of pixel point (i, j) is:

P(C _b , C _r )=exp[-0.5(x _ij -M) ^T C ^-1 (x _ij -M)]

Among them, M is the mean value of x _ij , and C is the variance of x _ij .

Then normalize P(C _b , C _r ) by the maximum skin color similarity MAX P(C _b , C _r ) in the region:

Then the normalized skin color similarity image F(i, j)=P'(C _b , C _r ) can be obtained. Set the adaptive threshold Q of skin color similarity, and obtain a binary image by the following formula:

${\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; Q</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; Q</span>}\end{array}\right.$

Then the maximum height C _max and the maximum width L _max of the skin color area can be obtained, and the approximate area of the face can be obtained based on the geometric features of the face. The steps are as follows:

(1) The skin color area in the vertical direction is expanded by 0.5*C _max up and down, and the point F _T (x, y) is set to 1 as the face area.

(2) Expand the skin color area in the horizontal direction by 0.5*L _max up and down, and set the point F _T (x, y) to 1 as the face area.

In this way, the image of the face area can be obtained, and then input into the following step 5 for recognition.

Step 5, using the established BP neural network model to detect and identify the extracted face area image to determine whether it is a user image. Specifically include the following steps:

After the face area image in step 4 is extracted, the extracted image is processed and cropped to match the input image characteristics of the neural network model. Then input it into the BP neural network model for face recognition.

If the BP neural network model outputs y=0, the face is a non-user image, indicating that there is an intruder; when it is identified as a non-user image, it is determined to be a non-user intrusion and an alarm signal is sent to the user.

Otherwise, the BP neural network model outputs y=1, which is recognized as a user image. When it is judged as the user's face image, it is determined to be the user and the monitoring video picture can be reacquired according to the set time, that is, return to steps 2 to 5 to complete the continuation of the video monitoring and identification process.

On this basis, the present invention also proposes an intelligent indoor intrusion detection system based on deep learning, which can use the above-mentioned detection method for image acquisition, extraction and identification detection. The system can be divided into normal working mode and sleep mode according to user settings. When there are visitors, the user can turn on the sleep mode, which can save power consumption and make the design more humanized. When the user leaves, he can choose to start the system and turn on the normal working mode, so that it can monitor the home.

Specifically, the system includes:

The video monitoring module is used to dynamically monitor the home environment and obtain monitoring video images;

The image acquisition module is used to obtain the differential image between adjacent frames from the monitoring video screen by using the inter-frame difference algorithm;

The image extraction module is used to perform binarization processing on the difference images between the acquired adjacent frames, and extract the processed image to obtain the changing foreground image under the static background;

The recognition analysis module includes a modeling unit, a recognition unit, an area extraction unit and an output unit; wherein, the modeling unit is used to establish a BP neural network model, and to train the BP neural network model according to input training data that includes user images The recognition unit is used to input the extracted change foreground image into the BP neural network model to detect and identify whether there is a human figure; the region extraction unit is used to extract from the change foreground image when the recognition determines that there is a human figure Detecting and extracting to obtain the face area image; the recognition unit is also used to input the extracted face area image into the BP neural network model to detect and identify whether it is a user image; When the user image is determined to be non-user intrusion, and generate and output control signals;

The sending unit is configured to send the intrusion prompt data to the user according to the control signal.

In the system, when the normal working mode is started, the video monitoring module can monitor the home in different situations during the day and night, preferably multiple frames of input images can be collected by the infrared monitoring and camera electronic equipment, and it is not limited by the light, so as to obtain Monitor the video screen.

And, its principle of described system is as shown in Figure 3, and the modeling unit in the identification analysis module, after building a model, uses the model for the first time to input training data, and BP neural network model is trained, and its steps include initialization weight and Threshold, the PRP conjugate gradient algorithm is used to adjust the weight and threshold layer by layer, and iterate to the maximum number of iterations. And the engineering process of the recognition unit, the area extraction unit and the output unit are all as described in the above method.

In order to better monitor the home, the system may also include an alarm module, which is used to generate sound and light alarms according to control signals. If the system is in the normal working mode, when it is determined that it is a non-user intrusion, it will send the intrusion prompt data to the user and generate an audible and visual alarm through the alarm module; wherein, the intrusion prompt data can include the extracted humanoid image of the intruder or the humanoid image of the intruder. face image, or both.

If the system is in sleep mode, both the sending unit and the alarm module receive control signals, but do not send intrusion prompt data and generate an alarm.

To sum up, the intelligent indoor intrusion detection method and system based on deep learning provided by the present invention adopts the BP neural network based on the PRP conjugate gradient algorithm, improves the network training speed, and ensures the correctness of convergence and real-time performance. Extract the detected target area and input it into the neural network, without identifying complex background and other interference factors, to ensure the correctness of classification. Therefore, identification can be performed efficiently and power consumption can be saved. It can effectively solve the problem that users and intruders, moving objects and intruders cannot be accurately detected and identified.

The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments, and can also be made without departing from the gist of the present invention within the scope of knowledge possessed by those of ordinary skill in the art. Variations.

Claims (10)

1. a kind of Intelligent indoor intrusion detection method based on deep learning is it is characterised in that comprise the following steps:

Set up bp neural network model, and according to the training data that input comprises user images, bp neural network model is trained；

The difference image of adjacent interframe is obtained using inter-frame difference algorithm from monitor video picture；

Binary conversion treatment is carried out to above-mentioned difference image, the Prospects For Changes region in the image after extraction process；

To the Prospects For Changes image detection extracted and identify it with the presence or absence of humanoid using the bp neural network model set up； When identification has humanoid, detection and extraction from described Prospects For Changes image obtains human face region image；

Using the bp neural network model set up, user is determined whether to the human face region image detection extracted and identification Image；And when identification is judged as non-user image, is defined as non-user invasion and sends alarm signal to user.

2. according to claim 1 the Intelligent indoor intrusion detection method based on deep learning it is characterised in that methods described Middle human face region image is obtained by following steps:

Set up based on colored complexion model, and normalization colour of skin similarity；

Using the complexion model set up to Prospects For Changes image detection, obtain normalized colour of skin similarity graph picture；

Set colour of skin similarity threshold, acquired normalized colour of skin similarity graph picture is carried out with binary conversion treatment, obtain the colour of skin Region；

According to the scope setting, area of skin color is chosen, obtain human face region image.

3. according to claim 1 the Intelligent indoor intrusion detection method based on deep learning it is characterised in that: methods described Middle utilization prp conjugate gradient algorithms are trained to bp neural network model.

4. according to claim 1 the Intelligent indoor intrusion detection method based on deep learning it is characterised in that: methods described Middle training data at least includes user's whole body images and the user's facial image possessing different attitudes.

5. according to claim 1 the Intelligent indoor intrusion detection method based on deep learning it is characterised in that: methods described Middle training data also includes non-humanoid image.

6. according to claim 1 the Intelligent indoor intrusion detection method based on deep learning it is characterised in that: methods described In when identification judge do not have humanoid when, reacquire monitor video picture.

7. according to claim 1 the Intelligent indoor intrusion detection method based on deep learning it is characterised in that: methods described In when identification is judged as user images, be defined as user and according to setting time reacquire monitor video picture.

8. a kind of Intelligent indoor intruding detection system based on deep learning is it is characterised in that include:

Video monitoring module, for domestic environment is entered with Mobile state monitoring, obtains monitor video picture；

Image capture module, for obtaining the difference image of adjacent interframe from monitor video picture using inter-frame difference algorithm；

Image zooming-out module, for carrying out binary conversion treatment to the difference image of acquired adjacent interframe, and extracts after treatment Image in Prospects For Changes region；

Discriminatory analysiss module, including modeling unit, recognition unit, area extracting unit and output unit；Wherein, described modeling is single Unit, is used for setting up bp neural network model, and according to the training data that input comprises user images, bp neural network model is instructed Practice；Described recognition unit, for inputting the Prospects For Changes being extracted image, bp neural network model is detected and identification is sentenced Break with the presence or absence of humanoid；Described area extracting unit is used for when identifying that judgement has humanoid, from described Prospects For Changes image Detection and extraction obtain human face region image；Described recognition unit is additionally operable to the human face region image extracting input bp nerve net Network model is detected and identification determines whether user images；Described output unit, for being judged as non-user figure in identification As when be defined as non-user invasion, and generate and output control signal；

Transmitting element, for sending invasion prompting data according to control signal to user.

9. according to claim 8 the Intelligent indoor intruding detection system based on deep learning it is characterised in that: described transmission The invasion prompting data that unit sends includes invader's image.

10. according to claim 8 the Intelligent indoor intruding detection system based on deep learning it is characterised in that: also include Alarm module, described alarm module is used for producing sound and light alarm according to control signal.