CN111310689B - Method for recognizing human body behaviors in potential information fusion home security system - Google Patents

Abstract

A method of human behavior recognition in a family security system based on potential information fusion. Taking the time series of human motion obtained from tracking as the research object, the relationship between gesture features and behaviors, between interactive object features and behaviors, and between behaviors and behaviors The correlation of human body behavior recognition in the home security system is fully exploited by introducing constraints in the extraction of gesture spatio-temporal features and the feature extraction of interactive objects to fully mine the impact of potential information on human behavior recognition in home security systems, thereby increasing the differences between behavior categories and reducing Small behavior intra-class differences improve the accuracy and generalization of human behavior recognition methods. Use the mutual information of each joint point about the behavior category as a constraint condition, sort all the mutual information obtained, retain the joint point group with the largest mutual information that can represent a specific behavior, and use the filtered joint point group and interactive object feature fusion Perform behavior recognition to improve the real-time and accuracy of recognition.

Description

Human behavior recognition method in home security system based on latent information fusion

Technical Field

The present invention relates to the field of computer vision technology, and in particular to a method for human behavior recognition in a home security system with potential information fusion.

Background Art

At present, many people have begun to install video surveillance systems at home to protect their property and life safety. However, these surveillance systems are installed inside the home and cannot prevent accidents before they happen. In addition, traditional digital surveillance systems mainly rely on monitoring personnel to monitor and analyze the surveillance images, which is not only inefficient, but also cannot meet the increasingly high security requirements in terms of real-time and effectiveness.

Summary of the invention

In order to overcome the deficiencies of the above-mentioned prior art, the purpose of the present invention is to provide a method for human behavior recognition in a home security system with potential information fusion, in which a computer is used to automatically analyze human behavior in the monitoring screen on behalf of family members. When an abnormal phenomenon is found, the family members can be immediately reminded to pay attention to the situation at the door of the house. Not only can 24×7 all-weather reliable monitoring be carried out, but the real-time performance and effectiveness are also greatly improved.

In order to achieve the above object, the technical solution adopted by the present invention is:

A method for human behavior recognition in a home security system providing potential information fusion includes the following steps;

Step 1: Use a camera to capture images;

Step 2: Use the illumination adaptive method based on background difference method to detect human targets in the collected images, and then use the Staple method to track the detected human targets to obtain the human motion time series;

Step 3: Identify the detected face to determine whether it is a family member. If it is a family member, do not perform any operation on the motion time sequence obtained in step 2. Otherwise, perform human behavior recognition.

Step 4, extracting the temporal and spatial characteristics of the human body posture from the motion time series obtained in step 2;

Step 5: Use the cue-enhanced deep convolutional neural network to extract the features of the interactive objects;

Step 6: Fusing the global spatiotemporal features of the posture and the local interactive object features extracted in steps 4 and 5;

Step 7: Input the fused feature vector into the SVM classifier for behavior recognition.

In the step 2, the human body entering the detection range is detected using a lighting adaptive method based on the background difference method, and the background modeling is performed using the Vibe algorithm. The number of pixels of the human target detected in the previous frame is recorded, represented by Y, and the number of pixels of the foreground target detected in the current frame is represented by L. Since a large area of white will appear at the moment of sudden change in illumination, the system will mistakenly detect the background as the foreground, and L>Y at this time. Therefore, a threshold value (the number of pixels of the human target range detected in the previous frame) is set during foreground detection to determine the detection range of the foreground. If the range exceeds the threshold value, it means that a sudden illumination change has occurred. Otherwise, no sudden illumination change has occurred. If a sudden illumination change has occurred, the brightness change value of the pixel point in two adjacent frames of the image is used to perform illumination compensation on the background model. The compensation formula is as follows:

Δ _t (x,y)＝|V _t (x,y)-V _t-1 (x,y)|

in:

V _t represents the global average brightness value of the image _It (x, y), where n is the total number of pixels in the image, n = 1280 × 480 = 614400 pixels, _It (x, y) _{max (R, G, B)} and _It (x, y) _{min (R, G, B)} represent the maximum and minimum values of the R, G, and B components at the pixel point (x, y), respectively;

After detecting the human target, the Staple method is used to track the detected human target. During the tracking process, the translation filter and color filter are used to find the position of the target, and then the scale filter is used to get the size of the target, and finally the human motion time series is obtained.

In the step 4, the temporal and spatial features of postures are extracted from the obtained human body motion time series, and the specific process includes:

1) Calculate the mutual information of each joint point, and use the mutual information to determine the response degree of each joint point to a specific behavior. Finally, retain the joint point group with the largest mutual information that can represent the specific behavior. The formula for calculating the mutual information of each joint point is as follows:

I(f ^j ,Y)＝H(f ^j )-H(f ^j |Y)

表示第j个关节点随时间变化的动态过程，N 表示人体运动时序序列的帧数，Y为人体行为的类别，在家庭安防场景下，主要识别送水、送快递、送外卖、朋友、保洁人员、其他人等，因此Y＝1，2，3，4，5，6，其中熵的计算公式如下：Where H(f ^j ) represents the information entropy of the jth joint point, j = 1, 2, ..., 20,

represents the dynamic process of the jth joint point changing over time, N represents the number of frames of the human motion time series, and Y is the category of human behavior. In the home security scenario, it mainly identifies water delivery, express delivery, takeaway delivery, friends, cleaning staff, and other people, so Y = 1, 2, 3, 4, 5, 6, where the entropy calculation formula is as follows:

Where p(f ^j ) is the probability density function, i represents the number of frames in the time series, i=1,2...,N;

2) Extract the posture spatiotemporal features of the above-screened joint points, where the features in the spatial dimension are:

Where K represents the joint point of the human body posture, K = 1, 2, ..., 20, N represents the number of frames of the human body motion time series, the human hip joint point is selected as the human body center of mass, T represents the joint coordinate trajectory feature matrix, θ represents the direction matrix of each joint point after each selection relative to the human body center of mass, D represents the spatial distance matrix of any two joint points, ψ represents the direction matrix of the vector formed by any two joints relative to the upward vector of the center of mass, and A represents the 3 inner angle size matrices formed by any 3 joint points;

The characteristics in the time dimension are:

Among them, ΔT is the trajectory displacement matrix of the joint point, Δθ is the direction of the same joint point with displacement, ΔD is the matrix of the distance change between any two joint points over time, Δψ is the change in direction of the upward vector of any two joint points relative to the center of mass, and ΔA is the change matrix of the size of the internal angle formed by any three joint points.

The extracted spatiotemporal features of the posture are expressed as:

F _pose =F _spatial +F _temporal .

In the step 5, the detected human body is used as a clue, and the effective object interacting with the human body is used as a high-level clue, and the features of the object interacting with the human body are extracted using a convolutional neural network, and the positional relationship between the object in the detected human body and the human body is implicitly integrated into the convolutional neural network to extract the features of the effective object interacting with the human body;

The loss function is used during training, and the parameters are adjusted during loss back propagation. The formula for calculating the mixed loss function is:

L(M,D)＝L _main (M,D)+αL _hint (M,D)

作为 N个样本图片的训练集合，

表示N张图像，

表示相关的类别标签，α取值在0～1之间。Where L _main (M, D) represents the loss function of interactive object feature extraction, L _hint (M, D) represents the loss function of the distance hint task, M represents the network model,

As a training set of N sample images,

represents N images,

Represents the relevant category label, and α takes a value between 0 and 1.

In step 6, since the spatiotemporal features of the posture and the interactive object features have different response levels to the recognition of different human behaviors, the two features are weightedly fused, and the formula is as follows:

F＝w ₁ F _pose +w ₂ F _object

Wherein, _w1 is the weighted coefficient of the posture spatiotemporal feature, _w2 is the weighted coefficient of the human body interactive object feature, and _w1 + _w2 =1, _Fpose represents the posture spatiotemporal feature, and _Fobject represents the interactive object feature.

In the step seven, the fused feature vector is input into the SVM classifier for classification to obtain the final recognition result.

Beneficial effects of the present invention:

The present invention uses machine vision technology to detect human behavior in a home security environment. Before behavior detection, face recognition is first performed to determine whether the person is a family member. If the person is a family member, behavior detection is not performed. Otherwise, behavior detection is performed. Interactive object detection is integrated into behavior detection to improve recognition accuracy. Behavior is detected before the doorbell is pressed, which can prevent problems before they occur and improve real-time performance and effectiveness. In the present invention, the body characteristics of human interactive objects and human motion are combined for behavior detection, which has important research value for processing human behavior recognition through interactive objects in different scenarios.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a behavior recognition method provided by an embodiment of the present invention.

FIG. 2 is a flow chart of human target detection provided by an embodiment of the present invention.

FIG3 is a flow chart of posture spatiotemporal feature extraction provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention will be further described in detail below in conjunction with the accompanying drawings.

In view of the problem of outsider intrusion in existing home security systems and the problem that the real-time and effectiveness of traditional digital monitoring cannot meet security needs, the present invention uses machine vision technology in security scenarios. It does not require monitoring personnel to analyze monitoring images. It uses face recognition technology for family members and human behavior recognition technology for people other than family members. It detects human behavior before knocking on the door and notifies family members in time to prevent problems before they happen, thereby improving real-time and accuracy.

The application principle of the present invention is further described below in conjunction with the accompanying drawings:

FIG1 is a schematic diagram of the overall process of the method of the present invention. The method for identifying human behavior in a home security system with potential information fusion according to the present invention is performed in the following steps:

Step 1: Use the camera installed at the door to collect images within the detection range.

Step 2, as shown in Figure 2: First, perform real-time human target detection on the collected image. When this system uses the background difference method for human target detection, the establishment and update of the background model is the key to the background difference method. Since the home security system has high real-time requirements and there is a sudden change in light, the Vibe background modeling method is used to achieve model establishment. The algorithm is very fast, has a relatively small amount of calculation, and has a certain degree of robustness to noise. Record the number of pixels in the human target range detected in the previous frame, represented by Y, and the number of pixels detected as foreground in the current frame, represented by L. Since a large area of white will appear at the moment of sudden change in light, the system will mistakenly detect the background as foreground, and L>Y at this time. Therefore, when detecting the foreground, a threshold is set (the number of pixels in the human target range detected in the previous frame) to determine the detection range of the foreground. If the range exceeds the threshold, it means that a sudden change in light has occurred. Otherwise, no sudden change in light has occurred. If a sudden change in light has occurred, the background model is compensated for light by using the brightness change value of the pixel point in two adjacent frames of the image. The compensation formula is as follows:

Δ _t (x,y)＝|V _t (x,y)-V _t-1 (x,y)|

in:

It represents the global average brightness value of the image _It (x, y), where n is the total number of pixels in the image, n = 1280 × 480 = 614400, _It (x, y) _{max (R, G, B)} and _It (x, y) _{min (R, G, B)} represent the maximum and minimum values of the R, G, and B components at the pixel point (x, y), respectively.

In actual situations, since there are boundary parts or some background objects with strong reflection coefficients in the captured pictures, they cannot be completely offset after background difference processing. They are reflected as some point-shaped, small block-shaped and line-shaped noises. These need to make reasonable judgments in the process of target detection to distinguish the actual moving targets. Therefore, morphological filtering is used to solve this problem for the binary image obtained. For multiple targets, the binary image after morphological filtering generally contains multiple regions. Since the multi-target region is generally composed of several sub-regions that are not connected to each other, it is necessary to detect the connectivity of each region, and then distinguish them by marking, and frame each target in the original image according to these marks, so as to further calculate the position of each target in each frame image.

Furthermore, the Staple method is used to establish a correlation filter model and a color filter template for the first frame image. For a new frame image, the translation filter and the color filter are first used to find the position of the target, and then the scale filter is used to extract different candidate target frames with the target position as the center point. The scale corresponding to the value with the largest response value is used as the final target scale to obtain the position and size of the target, and then the correlation filter model and the color filter model are updated.

After obtaining the score of the translation filter and the score of the color histogram, a weighted sum is performed, and the calculation formula is:

f(x)＝γ _tmpl f _tmpl (x)+γ _hist f _hist (x)

Where x=T( _xt , p); θt _-1 , T is the feature extraction function, _xt represents the t-th frame image, p represents the rectangular box in any frame image, θ represents the model parameters, θt _-1 is the target model parameter established according to the previous t-1 frame, in order to combine the gradient features with the color features on the basis of meeting the real-time performance, a linear method is used to derive the scoring function, where _γtmpl is the score coefficient of the filter template, _γhist is the histogram score coefficient, and _γtmpl + _γhist =1.

Step three, perform facial recognition on the detected human body to determine whether it is a family member. If it is a family member, the recognition result will be displayed on the human-computer interaction interface; otherwise, behavioral recognition will be performed.

Step 4, as shown in Figure 3: Taking the human motion time series as the research object, extracting the spatiotemporal features of human posture. The specific process includes:

1) In the home security scenario, people move forward, so the human body captured by the camera changes in distance, so there may be large differences in the coordinates of the human joints. In order to eliminate this difference, the coordinates of the human joints are first normalized.

Assume that the original coordinates of a joint point of the human body are (x ₀ ,y ₀ ), and the normalized coordinates are (x,y). The normalization formula is as follows:

Where d = max{w,h}, w and h are the width and height of the image respectively, and after normalization x,y∈(-1,1).

2) After obtaining the normalized human body posture coordinates, the temporal and spatial features of the posture time series are extracted. The spatial features describe the positions of the joint points and their relative positions on the same frame image, and the temporal features describe the changes in the joint positions caused by posture changes.

Since each joint of the human body responds to a specific behavior to a different degree, if all joints are taken into account without distinguishing them, those joints with low response levels will inevitably introduce noise and reduce the recognition effect. Therefore, some noise points must be selectively discarded. Therefore, by calculating the mutual information of each joint about the behavior category, the group of joints with the largest mutual information that can represent the specific behavior is retained.

Assuming that the behavior time series has a total of N frames, the dynamic process of the jth (j=1, 2, ..., 20) joint point changing over time can be expressed as:

The mutual information of each joint point for the human behavior category is:

I(f ^j ,Y)＝H(f ^j )-H(f ^j |Y)

Where H(f ^j ) represents the information entropy of the jth joint point, and Y is the category of human behavior. In the home security scenario, it mainly identifies water delivery, express delivery, takeaway delivery, friends, cleaning staff, and other people, so Y = 1, 2, 3, 4, 5, 6. The above formula is used to measure the response degree of each joint point to a specific behavior category. The entropy calculation formula is:

Where p(f ^j ) is the probability density function, p(Y, _fi ^j ) is the joint probability density function, p(fi _j |Y) is the conditional probability density function, and ⁱ represents the number of frames of the time series i = 1, 2..., N.

After calculating the mutual information of each joint point to the human behavior category, the mutual information is sorted from large to small, and the joint point group with the largest mutual information that can represent a specific behavior is selected.

The selection rule of the joint point group with the largest mutual information is: for human behavior recognition in the home security scenario, for a normal human body, the main focus is on the information of the joint points of the human body's arms, hands, and legs, while for a person with intrusion behavior, the focus will be on the information of the entire joint point. Therefore, the constraints for selecting the sorted mutual information matrix are:

The matrix composed of the mutual information of the joint points obtained for each behavior is:

Where N represents the Nth (N=1, 2, 3, 4, 5, 6) type of behavior, K represents the Kth (K=1, 2, ..., 20) joint point, and the mutual information obtained for each joint point is sorted. Since the arms, hands, and legs are the main joint points of concern, the joint point group composed of the above three parts of joint points that can represent specific behaviors is selected from the sorted mutual information group.

The posture matrix after screening by the response degree of the joint points to the behavior is:

Wherein, r _ij =(x _ij ,y _ij ), i∈{1,2,...,N}, j∈{1,2,...,K}, N=6, the value range of K is 4 to 14, and K here is the maximum subscript of the obtained maximum joint point group.

After obtaining the filtered posture matrix, the features in the spatial dimension and the features in the temporal dimension are extracted. The features in the spatial dimension are:

表示筛选后的每个关节点相对于人体质心的方向矩阵，

表示任意两个关节点的空间距离矩阵，

表示任意2个关节构成的向量相对于质心向上的向量的方向矩阵，

表示任意3个关节点构成的3个内角大小矩阵。The human hip joint point ( _xi0 , _yi0 ) is selected as the human body center of mass, T = ( _tij ) _{N × K} represents the joint coordinate trajectory feature matrix, _tij = ( _xi0 - _xi0 , _yi0 - _yi0 ),

Represents the direction matrix of each joint point after screening relative to the center of mass of the human body,

Represents the spatial distance matrix of any two joint points,

Represents the direction matrix of the vector formed by any two joints relative to the upward vector of the center of mass,

Represents the 3 inner angle matrices formed by any 3 joint points.

The characteristics in the time dimension are:

为同一关节点随位移的方向，

为任意两个关节点的距离随时间变化的矩阵，

为任意两个关节点相对质心向上的向量的方向变化，

为任意3个关节点构成的内角大小变化矩阵。Among them, ΔT＝(xi _+s,j - _xij ,yi _+s,j - _yij ) _(Ns)×2K is the trajectory displacement matrix of the joint point,

is the direction of displacement of the same joint point,

is the matrix of the distance between any two joint points changing over time,

is the change in the direction of the upward vector of any two joint points relative to the center of mass,

It is the inner angle size change matrix formed by any three joint points.

The posture spatiotemporal features obtained by time features and space features are expressed as:

F _pose =F _spatial +F _temporal

Furthermore, by taking the detected human body as a clue and the effective objects interacting with the human body as high-level clues, a convolutional neural network is used to extract the features of the objects interacting with the human body. The positional relationship between the detected objects in the human body and the human body is implicitly integrated into the convolutional neural network to extract the features of the effective objects interacting with the human body.

作为N个样本图片的训练集合，

表示N张图像，

表示相关的类别标签，α取值在0～1之间，混合损失函数的公式为：The present invention jointly performs two tasks, including the main task of interactive object recognition and the auxiliary task of distance hint enhancement. The auxiliary task plays a role in regularizing the network and enhancing the network's expressiveness. The influence of the hint task on the main task is reflected in all the convolutional layers before the shared full connection. In order to learn the weights of these layers in a mixed way, a hybrid loss function is used, which combines the loss functions of the two tasks. Specifically, the network model is expressed by M,

As a training set of N sample images,

represents N images,

Represents the relevant category label, α takes a value between 0 and 1, and the formula of the mixed loss function is:

L(M,D)＝L _main (M,D)+αL _hint (M,D)

M _main (·) and M _hint (·) represent the output of the main task and the output of the hint task, respectively. The model parameters are trained and fine-tuned by stochastic gradient descent. Stochastic gradient descent is used to optimize L(M,D). After calculating the gradient, the weight ω is updated using the rule expressed in the following formula.

Furthermore, since the posture features and interactive object features have different response levels to different human behavior recognition, the two features are weighted fused, and the formula is as follows:

F＝w ₁ F _pose +w ₂ F _object

Wherein, w ₁ is the weighted coefficient of the posture spatiotemporal feature, w ₂ is the weighted coefficient of the human body interactive object feature, and w ₁ +w ₂ = 1. F _pose is the extracted posture spatiotemporal feature, and F _object is the extracted interactive object feature.

Furthermore, the fused features are classified. This system mainly recognizes behaviors such as delivering water, express delivery, takeout delivery, friends, and other people. Therefore, a multi-classification support vector machine is needed. It is implemented by designing a binary classification model between any two categories, and finally combining multiple binary classifiers to realize the construction of multiple classifiers. The binary classification here still uses the above method. There are 6 categories in this system. Each classification takes 1 category as a positive sample and the other 1 category as a negative sample, and so on. In this way, there are 15 classifiers in total. During classification, these 15 classifiers answer in turn which category they belong to in the two categories. Finally, the category with the highest number of votes is the category to which they belong.

Step 5: Since the image processing is performed on the cloud server, the recognition result cannot be seen by every user, so the human-computer interaction module is required to receive the recognition result and display it. If there is a situation where the family is not at home but someone is active at the door, the recognition result will be sent to the family in the form of a text message.

Claims (2)

1. A method for human behavior recognition in a home security system with potential information fusion is provided, characterized in that it comprises the following steps; Step 1: Use a camera to capture images; Step 2: Use the illumination adaptive method based on background difference method to detect human targets in the collected images, and then use the Staple method to track the detected human targets to obtain the human motion time series; Step 3: Identify the detected face to determine whether it is a family member. If it is a family member, do not perform any operation on the motion time sequence obtained in step 2. Otherwise, perform human behavior recognition. Step 4, extracting the temporal and spatial characteristics of the human body posture from the motion time series obtained in step 2; Step 5: Use the cue-enhanced deep convolutional neural network to extract the features of the interactive objects; Step 6: Fusing the global spatiotemporal features of the posture and the local interactive object features extracted in steps 4 and 5; Step 7: Input the fused feature vector into the SVM classifier for behavior recognition; In the step 2, the human body entering the detection range is detected using a lighting adaptive method based on the background difference method, and the Vibe algorithm is used to perform background modeling. The number of pixels of the human target detected in the previous frame is recorded, represented by Y, and the number of pixels of the foreground target detected in the current frame is represented by L. Since a large area of white will appear at the moment of sudden change in illumination, the system will mistakenly detect the background as the foreground. At this time, L>Y, so a threshold (the number of pixels of the human target range detected in the previous frame) is set during foreground detection to determine the detection range of the foreground. If the range exceeds the threshold, it means that a sudden illumination change has occurred. Otherwise, no sudden illumination change has occurred. If a sudden illumination change has occurred, the background model is compensated for illumination using the brightness change value of the pixel point in two adjacent frames of the image. The compensation formula is as follows: Δ _t (x,y)＝|V _t (x,y)-V _t-1 (x,y)| in:

V _t represents the global average brightness value of the image _It (x, y), where n is the total number of pixels in the image, n = 1280 × 480 = 614400 pixels, _It (x, y) _{max (R, G, B)} and _It (x, y) _{min (R, G, B)} represent the maximum and minimum values of the R, G, and B components at the pixel point (x, y), respectively; After detecting the human target, the Staple method is used to track the detected human target. During the tracking process, the translation filter and color filter are used to find the position of the target, and then the scale filter is used to obtain the size of the target, and finally the human motion time series is obtained; In the step 4, the temporal and spatial features of postures are extracted from the obtained human body motion time series, and the specific process includes: 1) Calculate the mutual information of each joint point, and use the mutual information to determine the response degree of each joint point to a specific behavior. Finally, retain the joint point group with the largest mutual information that can represent the specific behavior. The formula for calculating the mutual information of each joint point is as follows: I(f ^j ,Y)＝H(f ^j )-H(f ^j |Y) Where H(f ^j ) represents the information entropy of the jth joint point, j = 1, 2, ..., 20,

represents the dynamic process of the jth joint point changing over time, N represents the number of frames of the human motion time series, and Y is the category of human behavior. In the home security scenario, it mainly identifies water delivery, express delivery, takeaway delivery, friends, cleaning staff, and other people, so Y = 1, 2, 3, 4, 5, 6, where the entropy calculation formula is as follows:

Where p(f ^j ) is the probability density function, i represents the number of frames in the time series, i=1,2...,N; 2) Extract the posture spatiotemporal features of the above-screened joint points, where the features in the spatial dimension are:

Where K represents the joint point of the human body posture, K = 1, 2, ..., 20, N represents the number of frames of the human body motion time series, the human hip joint point is selected as the human body center of mass, T represents the joint coordinate trajectory feature matrix, θ represents the direction matrix of each joint point after each selection relative to the human body center of mass, D represents the spatial distance matrix of any two joint points, ψ represents the direction matrix of the vector formed by any two joints relative to the upward vector of the center of mass, and A represents the 3 inner angle size matrices formed by any 3 joint points; The characteristics in the time dimension are:

Among them, ΔT is the trajectory displacement matrix of the joint point, Δθ is the direction of the same joint point with displacement, ΔD is the matrix of the distance change between any two joint points over time, Δψ is the direction change of the upward vector of any two joint points relative to the center of mass, and ΔA is the internal angle size change matrix formed by any three joint points; The extracted spatiotemporal features of the posture are expressed as: F _pose =F _spatial +F _temporal ; In the step 5, the detected human body is used as a clue, and the effective object interacting with the human body is used as a high-level clue, and the features of the object interacting with the human body are extracted using a convolutional neural network, and the positional relationship between the object in the detected human body and the human body is implicitly integrated into the convolutional neural network to extract the features of the effective object interacting with the human body; The loss function is used during training, and the parameters are adjusted during loss back propagation. The formula for calculating the mixed loss function is: L(M,D)＝L _main (M,D)+αL _hint (M,D) Where L _main (M, D) represents the loss function of interactive object feature extraction, L _hint (M, D) represents the loss function of the distance hint task, M represents the network model,

As a training set of N sample images,

represents N images,

Represents the relevant category label, and α takes a value between 0 and 1; In step 6, since the spatiotemporal features of the posture and the interactive object features have different response levels to the recognition of different human behaviors, the two features are weightedly fused, and the formula is as follows: F＝w ₁ F _pose +w ₂ F _object Wherein, _w1 is the weighted coefficient of the posture spatiotemporal feature, _w2 is the weighted coefficient of the human body interactive object feature, and _w1 + _w2 =1, _Fpose represents the posture spatiotemporal feature, and _Fobject represents the interactive object feature. 2. According to the method for human behavior recognition in a home security system providing potential information fusion as described in claim 1, it is characterized in that in the step seven, the fused feature vector is input into the SVM classifier for classification to obtain the final recognition result.

Images