CN113963371A

CN113963371A - Human body abnormal posture detection method

Info

Publication number: CN113963371A
Application number: CN202110818490.6A
Authority: CN
Inventors: 张琴
Original assignee: Chongqing Communication Industry Services Co ltd
Current assignee: Chongqing Communication Industry Services Co ltd
Priority date: 2021-07-20
Filing date: 2021-07-20
Publication date: 2022-01-21

Abstract

The invention discloses a method for detecting abnormal postures of a human body, which comprises the following steps: s11, acquiring a target image; s12, detecting the region position and the human body posture of the human body in the target image; s13, judging whether the human body posture is in an abnormal posture or not; s15, if the abnormal posture exists, judging whether the abnormal posture is the abnormal posture which appears in the same human body for the first time; if the abnormal posture is the abnormal posture which occurs for the first time in the same human body, the step S16 is carried out, otherwise, the step S17 is carried out; s16, storing abnormal posture categories corresponding to the abnormal postures of the human body, abnormal times, the area position of the human body in the target image and storage time as abnormal target data; s17, finding out the area with the maximum intersection ratio of the area positions corresponding to the abnormal postures, and comparing the similarity of the areas; s18, judging whether the similarity exceeds a first preset threshold value; and S19, if the first preset threshold value is exceeded, generating an abnormal posture early warning.

Description

Human body abnormal posture detection method

Technical Field

The invention relates to a method for detecting abnormal postures of a human body.

Background

At present, in the field of abnormal posture detection, such as fall detection, three research directions are mainly provided: 1. detection based on wearable equipment sensor 2. detection based on thing networking environmental information 3. detection based on intelligent monitoring technique. Based on the wearable device approach, it may not be possible to detect an abnormal state due to the device being worn or damaged or forgotten to wear; the intelligent monitoring technology collects images through a camera and then identifies the images, so that the real-time performance is high, the accuracy is high, and the cost is relatively low. Therefore, it is a good choice to realize abnormal posture detection through intelligent monitoring technology.

However, most of the current abnormal posture detection based on the intelligent monitoring technology has the following two problems: 1) the method is suitable for scene limitation, mainly takes a single indoor scene, and cannot use the existing monitoring equipment; 2) the algorithm accuracy and speed do not reach a balance. Part of the invention is based on the traditional algorithm, the characteristics are manually constructed, although the operation speed is high, the robustness is not enough, and the influence of scene complexity, illumination and camera installation position is large; some of the invention are based on human body key point detection, and have high accuracy, but too large calculation amount, too high data marking cost and weak practicability.

Disclosure of Invention

Aiming at the defects of the prior art, the technical problems to be solved by the invention are as follows: the method for detecting the abnormal posture of the human body solves the defects of limited applicable scenes, low recognition rate, high possibility of being influenced by the environment, large calculated amount and the like, and provides a method and a system for detecting and tracking the abnormal posture of the human body based on deep learning and light weight.

In order to solve the technical problems, the invention adopts a technical scheme that: the method for detecting the abnormal posture of the human body comprises the following steps:

s11, acquiring a target image;

s12, detecting the region position and the human body posture of the human body in the target image;

s13, judging whether the human body posture is in an abnormal posture or not;

s15, if the abnormal posture exists, judging whether the abnormal posture is the abnormal posture which appears in the same human body for the first time; if the abnormal posture is the abnormal posture which occurs for the first time in the same human body, the step S16 is carried out, otherwise, the step S17 is carried out;

s16, storing abnormal posture categories corresponding to the abnormal postures of the human body, abnormal times, the area position of the human body in the target image and storage time as abnormal target data;

s17, finding out the area with the largest intersection ratio of the area positions corresponding to the abnormal posture from the abnormal target data, and comparing the similarity of the areas;

s18, judging whether the similarity exceeds a first preset threshold value;

and S19, if the first preset threshold value is exceeded, generating an abnormal posture early warning.

Further, the step of S10 includes the following substeps:

s111, acquiring video streams from a plurality of image pick-up devices in a target area;

and S112, performing frame extraction processing on the image video stream to obtain a target image of a single frame.

Further, in the step S12, detecting the region position and the body posture of the human body in the target image through an abnormal posture detection network, where the abnormal posture detection network includes a feature extraction network and a detection head network, and the feature extraction network includes several layers of convolutional layers; the detection head network comprises a positioner and a classifier; the step of S12 includes the following substeps:

s121, inputting the target image as a detection image of the feature extraction network, and extracting features of the detection image;

s122, selecting reciprocal N characteristic response graphs in the characteristic extraction network;

s123, acquiring the characteristics of the anchor frame area corresponding to each characteristic response image;

s124, the positioner generates a detection frame offset according to the input features, and coordinates of the detection frame offset are corrected according to the preset anchor frame position and size, so that coordinates of a central point, the length and the width are obtained, and the classifier generates a human body posture label according to the input features; wherein the human posture label comprises standing, squatting, lying and cutting;

s125, removing the detection frame with the label as the background so as to obtain a human body area detection frame; wherein, a maximum value is adopted to inhibit and eliminate repeated detection frames;

s126, outputting the human body region detection frame and the human body posture label;

in the step S13, it is determined whether there is an abnormal posture of the human body in the target image through the output human body region detection frame and the human body posture label.

Further, after the step of S13, the method further includes:

s14, sending the human body region detection frame to a pedestrian re-identification module to extract Reid features, wherein the pedestrian re-identification module comprises a global feature extraction module and a local feature extraction module, and the step comprises the following substeps;

s141, calculating the length-width ratio of the human body region detection frame;

s142, judging whether the calculated length-width ratio is in a preset threshold interval or not; if the current time is within the preset threshold interval, the step S143 is carried out, and if the current time is not within the preset threshold interval, the step S144 is carried out;

s143, rotating the human body area detection frame clockwise by 90 degrees;

s144, the global feature extraction module performs global feature extraction on the human body region detection frame to obtain global features, and the local feature extraction module performs local feature extraction on the human body region detection frame to obtain local features;

s445, splicing the global feature and the local feature vector to obtain a Reid feature corresponding to the current abnormal posture;

in the step S16, a Reid feature, an abnormal posture category, an abnormal frequency, a region position of the human body in the target image, and a storage time corresponding to the abnormal posture of the human body are stored as abnormal target data;

in step S17, a region having the largest intersection ratio of the region positions corresponding to the abnormal posture is found from the abnormal target data, and the Reid feature similarity between the regions is compared.

Further, in the step S144, the following sub-steps are included:

s1441, processing the human body region detection frame to obtain a corresponding characteristic diagram;

s1442, horizontally dividing the feature map into a plurality of blocks to obtain a plurality of feature vectors f ═ f₁,f₂,…,f_n]^T；

S1443, performing generalized average pooling operation on the plurality of feature vectors to obtain feature vectors subjected to generalized pooling operation

S1445, vertically splicing the plurality of feature vectors into a single feature vector serving as the local feature f _ local.

Further, in the step S1442, the generalized average pooling operation is obtained by the following formula:

wherein p isⁱThe pooling for the generalized average contains a learnable parameter with pi set to 1.0 at its initial value and x representing the input feature, i.e., the feature vector f.

Further, in the step S144, the global feature extraction includes the following sub-steps:

s1440', processing the human body area detection frame to obtain a corresponding characteristic diagram;

s1441', performing generalized average pooling operation on the feature map to obtain the global feature f _ global after the generalized pooling operation.

Further, in the step S18, if the first preset threshold is not exceeded, the process proceeds to a step S16.

Further, after the step of S18, the method further includes:

s18a, if the abnormal times of the abnormal posture exceed a first preset threshold, judging whether the abnormal times of the abnormal posture exceed a second preset threshold; if the result exceeds the predetermined value, the process proceeds to step S19, otherwise, the process proceeds to step S11.

And S19, if the second preset threshold value is exceeded, generating an abnormal posture early warning.

Further, before the step of S11, the method further includes:

s10, initializing an abnormal target list;

in step S16, storing the abnormal posture category and the abnormal frequency corresponding to the abnormal posture of the human body and the area position of the human body in the target image to the abnormal target list as abnormal target data, recording the position, posture category, abnormal frequency and last update time of the abnormal posture stored in the abnormal target list, and going to step S11;

in step S17, a region having the largest intersection ratio of the region positions corresponding to the abnormal postures is found from the abnormal target data in the abnormal target list, and the similarity of the abnormal postures is compared.

Further, after the step of S13, the method further includes:

s13a, traversing an abnormal target list if the human body posture is not in an abnormal posture, and judging whether the time for updating and storing corresponding abnormal target data last time in the abnormal target list exceeds a third preset threshold value;

s13b, if the third preset threshold is exceeded, removing the corresponding abnormal target data and returning to the step S11.

Further, in the step S15, it is determined whether the abnormal posture is an abnormal posture in which the same human body first appears by determining whether the abnormal target list is empty.

The method for detecting the abnormal posture of the human body has the following beneficial effects: 1) aiming at the defects of large calculated amount, high cost, high requirements on network bandwidth and quality and the like in the prior art, the invention designs a lightweight algorithm model, deploys the model to edge equipment, reduces network delay, reduces the bandwidth pressure of a server, and is particularly suitable for scenes with fewer computing nodes. 2) The invention expands the application scenes of the existing algorithm. At present, most algorithms are limited in single indoor scenes, and the method can utilize cameras deployed in old public areas to analyze multiple people simultaneously. 3) The invention introduces a pedestrian re-identification technology, continuously tracks the target state and reduces the false alarm rate.

Drawings

Fig. 1 is a flowchart of the entire human body abnormal posture detection method of the present application.

Fig. 1a is a flowchart of a first embodiment of a human body abnormal posture detection method according to the present application.

Fig. 2 is a flowchart of a second embodiment of the method for detecting an abnormal posture of a human body according to the present application.

Fig. 3 is a flowchart of a human body abnormal posture detection method according to a third embodiment of the present application.

Fig. 4 is a flowchart of a fourth embodiment of the method for detecting an abnormal posture of a human body according to the present application.

FIG. 5 is a schematic diagram of an abnormal posture detection network.

Fig. 6 is an internal structure diagram of the RFB receptor field module.

Fig. 7 is a schematic diagram of four poses.

FIG. 8 is a sub-flowchart for detecting body regions and body poses in a target image.

Fig. 9 is a schematic diagram of a human body region detection block in a normal posture.

Fig. 10 is a schematic diagram of a human body region detection block in an abnormal posture.

FIG. 11 is a pedestrian re-identification module schematic.

Fig. 12 is a diagram showing the effect of feature extraction and search performed by the pedestrian re-identification module.

Fig. 13 is a Reid feature extraction sub-flowchart.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

Referring to fig. 1, fig. 1 is a flowchart illustrating a method for detecting an abnormal posture of a human body according to a first embodiment of the present invention. The method for detecting the abnormal posture of the human body comprises the following steps:

s11, acquiring a target image;

the method comprises the following steps of:

The target area may be a mall, a cell, a specific place or may be divided according to the needs of the actual implementation.

including but not limited to standing, squatting, lying down, cutting, walking, falling, smoking, running, and the like. In this step, the region position and the human body posture in the target image may be detected through a detection network such as an SSD Single-stage target Detector (Single Shot multi box Detector).

S13, judging whether the human body posture is in an abnormal posture or not;

in the present embodiment, lying and squatting are defined as abnormal postures.

s16, storing abnormal posture categories and abnormal times corresponding to the abnormal postures of the human body and the area positions of the human body in the target image as abnormal target data, and turning to the step S11; the posture category refers to a posture category label corresponding to the abnormal posture: lying down and squatting down, and showing the posture category corresponding to the abnormal posture in a label mode. If the abnormal posture is the first appearance of the same human body, the abnormal times are recorded as 1.

S17, finding out the area with the largest intersection ratio of the area positions corresponding to the abnormal posture from the abnormal target data, and comparing the abnormal posture similarity of the areas, thereby judging whether the people corresponding to the abnormal posture are the same person or not;

s18, judging whether the similarity exceeds a first preset threshold value; if the first preset threshold value is exceeded, the step S19 is entered; otherwise, the process goes directly to step S16 to update the abnormal target data.

And S19, generating an abnormal posture early warning and turning to the step S16 to update the abnormal target data.

Specifically, with reference to the example, assuming that a certain camera takes a target image of the nail on the ground for a certain trip, the x, y, width, height of the certain area of the nail in the target image and the posture of the certain nail lying down, that is, the x, y coordinates of the certain nail in the target image, the width and the height of the target image of the certain nail are detected from the acquired target image. If the lying posture is judged to belong to an abnormal posture, if the abnormal posture is the first time that the nail lies down, the abnormal posture of the nail and the category and the abnormal frequency (1) of the nail and the corresponding area position (x, y, width, height) of the nail are stored, and therefore certain abnormal target data of the nail are obtained. If the abnormal posture of the first nail does not appear for the first time (from a plurality of given images, the currently detected image is not the image of the abnormal posture of the first nail appearing for the first time), finding out a history area with the largest intersection ratio of the area (x, y, width, height) of the first nail corresponding to the current abnormal posture from the stored data of the abnormal target of the first nail, and judging the similarity of the areas, so that the first nail can be judged to be not in the abnormal posture continuously in the duration, the judgment accuracy of the abnormal posture is improved, and the judgment of whether the first nail is a person needing help, such as a real fall, is further improved. And if the similarity is judged to be within the preset range, the nail is regarded as being in the abnormal posture continuously, the nail belongs to the abnormal posture, and early warning is further generated for prompting.

According to the embodiment, the existing monitoring camera can be used for acquiring the target image, the images acquired by the plurality of cameras in the target area and the multi-angle camera can be acquired, and the images acquired by the plurality of cameras in the target area are processed in different areas. The human body region, the abnormal posture feature and the abnormal times are taken as judgment keys, and the abnormal state is only considered to be an abnormal state in the same human body, the abnormal posture appears for many times and the region with the set range value, so that the judgment accuracy is improved, and the misjudgment probability is greatly reduced.

Referring to fig. 2, fig. 2 is a flowchart illustrating a method for detecting an abnormal posture of a human body according to a second embodiment of the present invention. The method for detecting the abnormal posture of the human body comprises the following steps:

s21, acquiring a target image;

s22, detecting the region position and the human body posture of the human body in the target image;

s23, judging whether the human body posture is in an abnormal posture or not;

s25, if the abnormal posture exists, judging whether the abnormal posture is the abnormal posture which appears in the same human body for the first time; if the abnormal posture is the abnormal posture which occurs for the first time in the same human body, the step S26 is carried out, otherwise, the step S27 is carried out;

s26, storing abnormal posture categories and abnormal times corresponding to the abnormal postures of the human body and the area positions of the human body in the target image as abnormal target data, and turning to the step S21;

s27, finding out the area with the largest intersection ratio of the area positions corresponding to the abnormal posture from the abnormal target data, and comparing the similarity of the areas;

s28, judging whether the similarity exceeds a first preset threshold value; if the first preset threshold value is not exceeded, directly switching to the step S26, and using the current abnormal target data as new abnormal target data (i.e. another new abnormal target data packet);

s28a, if the similarity exceeds a first preset threshold, turning to the step S26 to update the abnormal target data (namely, the abnormal target data in the original abnormal target data packet) and judging whether the abnormal frequency of the abnormal posture exceeds a second preset threshold; if the result exceeds the preset value, the step S29 is carried out, otherwise, the step S21 is carried out;

in this step, when the similarity exceeds a first preset threshold, it indicates that there is abnormal target data associated with the current abnormal posture of the same human body in the stored historical abnormal target data, that is, the same human body continues the abnormal posture, so that it is determined whether the number of times of occurrence of the current abnormal posture of the same human body is the second time and the number of times of occurrence of the current abnormal posture of the same human body exceeds a second preset threshold.

And S29, if the second preset threshold value is exceeded, generating an abnormal posture early warning.

In the embodiment, when the current abnormal posture does not appear for the first time, whether the historical abnormal target data has the similar abnormal posture of the same human body is judged, and if the historical abnormal target data has the similar abnormal posture of the same human body in the corresponding area and the corresponding abnormal posture appears, the second preset threshold value judgment is added, so that the accuracy of abnormal posture judgment is further improved, and the early warning is not directly generated only according to the current abnormal posture but also according to the non-first appearance of the current abnormal posture.

Referring to fig. 3, fig. 3 is a flowchart illustrating a method for detecting an abnormal posture of a human body according to a third embodiment of the present invention. The method for detecting the abnormal posture of the human body comprises the following steps:

s30, initializing an abnormal target list;

s31, acquiring a target image;

s32, detecting the region position and the human body posture of the human body in the target image;

s33, judging whether the human body posture is in an abnormal posture or not;

s33a, traversing an abnormal target list if the human body posture is not in an abnormal posture;

33 a' and judging whether the time for updating and storing the corresponding abnormal target data last time in the abnormal target list exceeds a third preset threshold value; if the third preset threshold value is not exceeded, the step S31 is carried out;

s33b, if the third preset threshold is exceeded, removing the corresponding abnormal target data and returning to the step S31.

S35, if an abnormal posture exists, judging whether the abnormal posture is the abnormal posture which appears in the same human body for the first time by judging whether the abnormal target list is empty; if the abnormal target list is empty, the step S26 is carried out, otherwise, the step S27 is carried out;

s36, storing abnormal posture types and abnormal times corresponding to the abnormal postures of the human body and the area positions of the human body in the target image to the abnormal target list as abnormal target data, recording the positions, posture types and abnormal times of the abnormal postures stored in the abnormal target list and the last updating time, and turning to the step S31;

s37, finding out the area with the maximum intersection ratio of the area positions corresponding to the abnormal postures from the abnormal target data of the abnormal target list, and comparing the similarity of the areas;

s38, judging whether the similarity exceeds a first preset threshold value; if the first preset threshold value is not exceeded, directly switching to the step S36, and using the current abnormal target data as new abnormal target data (i.e. another new abnormal target data packet);

s38a, if the similarity exceeds a first preset threshold, turning to the step S36 to update abnormal target data (namely, to update the abnormal target data in the original abnormal target data packet), and judging whether the abnormal frequency of the abnormal posture exceeds a second preset threshold; if the result exceeds the preset value, the step S39 is carried out, otherwise, the step S31 is carried out;

And S39, if the second preset threshold value is exceeded, generating an abnormal posture early warning.

Specifically, in combination with the embodiment, the body abnormal posture detection method firstly initializes the abnormal target list and determines that the abnormal target list is empty. Then, the obtained video is subjected to frame extraction to obtain a single image (namely a target image), and whether the human body in the target image has an abnormal posture or not is detected in a mode of a target detection network and the like.

Assuming that a certain nail in the target image a is in a falling state, determining that the certain nail has an abnormal posture, determining whether the abnormal target list is empty, if so, indicating that the certain nail in the target image has the abnormal posture for the first time, and storing the abnormal posture, the abnormal frequency (the first appearance and the abnormal frequency are 1) and the regional position of the certain nail in the target image (namely, the X and Y positions of the certain nail in the target image, the width and height occupied by the certain nail in the target image and other position characteristics) into the abnormal target list to be used as abnormal target data; if not, indicating that historical abnormal target data exist in the abnormal target list, searching whether abnormal target data corresponding to the abnormal posture of the certain nail exist in the abnormal target list, matching the current abnormal target data of the certain nail with the human body area and the abnormal posture in the historical abnormal target data existing in the abnormal target list, and comparing the similarity of the abnormal target data and the abnormal posture by comparing the intersection and the comparison of the two; judging whether the similarity of the first and second preset thresholds exceeds the first preset threshold through a preset first preset threshold related to the similarity, if the similarity exceeds the first preset threshold (indicating that the similarity is higher, the first is regarded as that the first is still in the abnormal posture and the current area position still corresponds to the area position of the historical abnormal target data of the first, indicating that the first is not moved from the area position of the abnormal posture, namely the first does not change the posture rapidly and moves to other positions, further indicating that the first continues the abnormal posture all the time, and the probability of belonging to an object needing important attention or help is higher), judging whether the current abnormal frequency exceeds the second preset threshold, if the similarity exceeds the set frequency, generating a pre-warning message, wherein the pre-warning message can be broadcasted through surrounding monitoring cameras or other devices in a voice mode, the early warning prompt can be carried out by other means, so that the condition that a certain nail in the corresponding area is in an abnormal state is prompted, and the help is provided for the certain nail in real time. And when the abnormal times do not exceed a second preset threshold value, the fact that the first person possibly lasts for a period of time but is not judged to be in a real abnormal state is shown, video frame extraction is continued to analyze the image.

Referring to fig. 4, fig. 4 is a flowchart illustrating a method for detecting an abnormal posture of a human body according to a fourth embodiment of the present invention. The method for detecting the abnormal posture of the human body comprises the following steps:

s40, initializing an abnormal target list;

s41, acquiring a target image;

s42, detecting the region position and the human body posture of the human body in the target image;

in this step, the area position and the body posture of the body in the target image are detected through an abnormal posture detection network, please refer to fig. 5, the abnormal posture detection network includes a feature extraction network and a detection head network, the feature extraction network includes a backbone network composed of a plurality of layers of convolutional layers; the detector head network includes a locator and classifier 100.

The feature extraction network is based on an SSD Single-stage target Detector (Single Shot multi Box Detector) network, and the following improvements are made to the feature extraction network:

the backbone network reduces the number of network layers and channels on the basis of the VGG16 deep convolutional neural network, the maximum number of channels does not exceed 256, and the parameter number is greatly reduced. With a batch normalization layer behind each convolutional layer (each convolutional layer has a batch normalization layer), training is accelerated. A receiving Field module (received Field Block) is added to the backbone network, so that multi-scale features can be better learned, and please refer to fig. 6 for an internal structure diagram of the RFB receiving Field module (please refer to conv4_3 in fig. 6). The finally designed network parameter number is 12.23M, and compared with an original SSD detector, the accuracy rate is only reduced by 2%, and the speed is improved by 4 times. With the acceleration frame provided by great, speeds of 25fps (frame per second) can be achieved on Jeston N ano equipment.

In the present embodiment, referring to fig. 7, four gestures are designed in the output layer of the abnormal gesture detection network: standing (stand), crouching (bend), lying (fall), cut (occluded, upper or lower body being blocked), wherein when the posture is judged as crouching and lying, it is considered as an abnormal posture and further analysis is required. The feature extraction network output layer uses 6 feature maps in total to detect targets with different scales, namely feature response maps output by the last 6 convolutional layers in the feature extraction network. And (3) outputting the position coordinates (x, y, width, height) of the human body and the corresponding posture categories of each layer respectively, finally combining the outputs of the layers, and performing maximum value inhibition to obtain a final result. Wherein, the number and the size of the anchor boxes (default boxes) of each layer are obtained by clustering through a k-means algorithm.

The structure of the complete abnormal posture detection network is shown in the following table 1: conv is a convolutional layer; stride is stride; padding; batch normalization; relu is a modified linear unit; maxpool, maximum pooling layer; dilation is the coefficient of expansion; RFB, receptive field module.

TABLE 1

Preferably, the abnormal posture detection network mainly trains as follows:

d1, searchCollecting monitoring pictures containing abnormal postures and normal postures to obtain a picture set, and recording the picture set as P, wherein the number of P is recorded as N; the pictures in the image set P are marked with manual position and type, the marking result of each picture is marked as,

wherein

Noted as the top left corner coordinate of the jth object in the kth image,

noted as the width and height of the jth object in the kth image,

and recording the category label of the jth target in the kth image, wherein T is the number of the targets in the kth image. Randomly dividing the picture set P into a training set and a testing set according to a predetermined ratio r1: r2, and respectively marking as P_r1And P_r2。

D2 training set P_r1And test set P_r2Converted to a standard coco dataset format and the picture size is transformed to 300x300 pixels.

D3, in the case that the image size is determined to be 300 × 300 pixels, the abnormal posture detection network generates 6 feature response maps (6 in this embodiment, but the number is not used to limit the scope of the present application), with the sizes of 40x40,20x20,10x10,5x5,3x3, and 1x1, respectively. And respectively traversing the 6 characteristic response graphs, taking each pixel on the characteristic response graphs as a center, and respectively forming a plurality of anchor frame sets with different small and wide widths for the 6 characteristic response graphs. Each anchor frame is represented by a center point coordinate cx, cy and a length and width w, h. Wherein:

the central point coordinate calculation formula is as follows:

wherein | f_kI is the size of the characteristic diagram, and k belongs to [1,2,3,4,5,6 ]]。

The length and width calculation formula of the anchor frame is as follows:

according to the set size and length-width ratio, the length and width of the anchor frame can be obtained

Wherein the content of the first and second substances,

the minimum size set for the kth feature response map,

is the k characteristic response diagram as a scale

The length-width ratio of the setting is,

the maximum size set for the kth characteristic response map,

is the k characteristic response diagram as a scale

The length-width ratio of the setting is,

the 6 signature response maps total 9698 anchor boxes.

D413, according to P_r1 P_r2The manual marking frame information in the method is used for respectively distributing the type and calculating the coordinate offset for the anchor frame of each characteristic response graph to be used as a real value ground route of network model training. Network lets through continuous learningAnd the output value approaches to a real value, so that the training of the network is completed.

The matching principle of the anchor frame and the marking frame mainly comprises two points:

1. and for each marking frame in the picture, finding the anchor frame with the largest intersection ratio with the marking frame, and matching the anchor frame with the marking frame, so that each marking frame can be ensured to be matched with a certain anchor frame.

2. For the remaining unmatched anchor boxes, if the intersection ratio of a certain labeled box is greater than a certain threshold (typically 0.5), then the anchor box matches this labeled box. An anchor frame matched with a certain marking frame is called a positive sample, and on the contrary, if a certain anchor frame is not matched with any marking frame, the anchor frame is a negative sample.

Let the anchor frame be d_iThe label box matched with it is g_j(ii) a When d is_iWhen being a positive sample, d_iIs of class g_jClass of (2), corresponding coordinate offset amount

The calculation formula is as follows:

when d is_iIn the case of a negative sample, d_iThe category of (2) is background and the coordinate offset is not calculated. Thus, the target value to be learned for training the network model is calculated.

D5, constructing and initializing an abnormal posture network model, and recording as W_abWherein W is_abThe system consists of a backbone network, a locator and a classifier. Backbone network denoted W_featureThe locator is marked as W_locThe classifier is denoted as W_cls。

D6, network forward computation. Randomly selecting 64 from P at a time_r1In the sample into the network W_abPerforming forward calculationsTo obtain W_locAnd W_clsThe output of the branch.

D7, loss function calculation and network parameter update. The loss function L (x, c, L, g) is lost by classification L_confAnd positioning loss L_locTwo parts are formed.

And D, sending the training true value g obtained in the step D4 and the network predicted value l obtained in the step D5 to a loss function for loss calculation. The number of the generated negative samples is far larger than that of the positive samples, so that the network training convergence is slow and unstable, online hard sample mining needs to be carried out according to the classification loss of each negative sample prediction frame in the network prediction value l, the proportion of the positive samples to the negative samples is kept not to exceed 1:3, and the screened positive samples and the screened negative samples participate in loss calculation of network back propagation.

Where N is the number of positive samples and α is 1. If N is 0, then the penalty is 0.

L_confFor multi-class cross entropy loss, the calculation formula is as follows:

L_locfor a smooth L1 regression loss, the calculation formula is as follows:

for indicating the function, when the anchor frame i is matched with the label frame j with the category k, the value is 1, otherwise, the anchor frame i is matched with the label frame j with the category kIs 0.

And (4) performing back propagation on the loss function to obtain an updated value of the network parameter, updating the network parameter by using the updated value, repeating the steps D6 and D7, and stopping training until the iteration number or the network loss is reduced to a preset value.

Specifically, referring to fig. 8, the step S42 includes the following sub-steps:

s421, inputting the target image as a detection image of the feature extraction network, and extracting features of the detection image; the input target image generates corresponding features through one convolution layer, one pooling layer and one full-connection layer, each time the input target image passes through one convolution layer and the like, the feature extraction process is performed, and the features can be a multi-dimensional array or a one-dimensional array.

S422, selecting reciprocal 6 feature response graphs in the feature extraction network; it is understood that in other embodiments, the

inverse

7, 5, 4, more or less feature response maps may be selected;

s423, obtaining the characteristics of the anchor frame area corresponding to each characteristic response graph;

s424, the positioner generates a detection frame offset according to the input features, and performs coordinate correction on the detection frame offset according to the preset anchor frame position and size, so as to obtain a center point coordinate, a length and a width, namely the region position of the human body in the target image, namely a human body position coordinate (x, y, width, height), and the classifier generates a human body posture label according to the input features; wherein the human posture label comprises standing, squatting, lying and cutting; in this step, the feature of the anchor frame region is a multi-dimensional feature, and the dimensions are loc _0 to loc _5 dimensions in the network structure table 1.

S425, removing the detection frame with the label as the background so as to obtain a human body area detection frame; wherein, a maximum value is adopted to inhibit and eliminate repeated detection frames;

and S426, outputting the human body region detection frame and the human body posture label.

Referring to fig. 9 and 10, the human body region detection frame in fig. 9 represents the human body region detection frame in the normal posture, and the human body region detection frame in fig. 10 represents the human body region detection frame in the abnormal posture.

S43, judging whether the human body posture is in an abnormal posture or not;

in this step, whether the human body in the target image has an abnormal posture is judged through the output human body region detection frame and the human body posture label.

S43a, traversing an abnormal target list if the human body posture is not in an abnormal posture;

s43 a', and judging whether the time for updating and storing the corresponding abnormal target data last time in the abnormal target list exceeds a third preset threshold value;

s43b, if the third preset threshold is exceeded, removing the corresponding abnormal target data and returning to the step S41;

and S44, sending the human body region detection frame to a pedestrian re-identification module to extract Reid features, please refer to FIG. 11, wherein the pedestrian re-identification module comprises a global feature extraction module and a local feature extraction module.

The pedestrian re-identification module tracks people in abnormal postures by combining human body position characteristics on the basis of an OSNet pedestrian re-identification network, and improves OSNet to obtain richer characteristic information. In the training stage, global branches and local branches are trained independently, and in the testing stage, f _ global and f _ local feature vectors are respectively spliced to form a (2048+512,1,1) dimensional vector serving as a reid feature of an input picture and used for subsequent feature similarity calculation. The complete pedestrian re-identification network module is shown in the following table 2: the OS block comprises an Omni-Scale module and a full-Scale network; GeM generalized average pooling; num _ class is the number of output classes; the number of pedestrians training the pedestrian re-recognition model is related, and no specific value is set here; average pore: average pooling; fc is the full connection layer.

TABLE 2

The pedestrian re-identification training process is as follows:

and B1, constructing a training data set. Collecting M pedestrian pictures with different IDs, wherein each pedestrian can correspond to a plurality of pictures to obtain a training picture set P, and the number of P is recorded as N (N)>M). And marking the labeling result corresponding to the training set as X { (X)_i,y_i),i＝1,2,...,N,y_i∈[0,M-1]}; the M IDs are pressed as 8: 2 (of course, other preset ratios may be used) into the training ID set P_trainAnd a set of test IDs P_testThe training set and the test set are not repeated for the person ID. All pictures under the training ID set are training sets and all pictures under the test ID set are test sets. The size of the pictures in the set is transformed to 256x192 pixels (this pixel is for illustration only).

B2, constructing and initializing a pedestrian re-identification network model, which is marked as W_reid，W_reidThe system is composed of a global branch and a local branch.

And B3, selecting a training batch sample. And randomly selecting 8 ID pedestrians each time, and randomly selecting 4 different pictures for each pedestrian. For each picture a of the 32 pictures, a triplet is formed by choosing one most difficult positive sample p and one most difficult negative sample n and a. A total of 32 triplet sets were obtained as training samples for the current batch.

B4, network forward calculation. Sending the training samples selected in the step 3) into a network W_reidForward calculations are performed. Note that the local features and the classification features obtained by the local branch are f _ local and f _ lcls, respectively, and the global features and the classification features obtained by the global branch are f _ global and f _ gcls, respectively.

B5, calculation of loss function and updating of network parameters. Total loss of network L_sumFor local branch loss L_localAnd global branch penalty L_globalAnd (3) the sum:

L_sum＝L_local+L_global

wherein the local branch and the global branch each define two loss functions, namely a classification loss and a triplet loss.

L_local＝L_{id_local}+L_{tri_local}

L_global＝L_{id_global}+L_{tri_global}

L_{id_local}And L_{id_global}The calculation formula is as follows:

wherein N is_sThe number of samples for the current training batch,

L_{tri_local}and L_{tri_global}The calculation formula is as follows:

for loss function L_sumAnd performing back propagation to obtain an updated value of the network parameter, updating the network parameter by using the updated value, and repeating the steps until the iteration times or the network loss is reduced to a preset value, and stopping training.

Specifically, referring to fig. 13, the present step includes the following substeps;

s441, calculating the length-width ratio of the human body region detection frame;

s442, judging whether the calculated length-width ratio is within a preset threshold interval; if the current time is within the predetermined threshold interval, the process proceeds to step S443, and if the current time is not within the predetermined threshold interval, the process proceeds to step S444; in this step, the set threshold interval is greater than 0.1 and less than 0.6;

s443, clockwise rotating the human body area detection frame by 90 degrees;

s444, the global feature extraction module performs global feature extraction on the human body region detection frame to obtain global features, and the local feature extraction module performs local feature extraction on the human body region detection frame to obtain local features; in this step, the local features are extracted through the following substeps:

s4441, processing the human body region detection frame to obtain a corresponding characteristic diagram;

s4442, horizontally dividing the feature map into a plurality of blocks to obtain a plurality of feature vectors f ═ f₁,f₂,…,f_n]^T；

In this embodiment, a local feature extraction branch is added after the conv4 module, and the feature map output by the conv4 module is horizontally divided into 4 blocks to obtain 4 feature vectors f ═ f₁,f₂,f₃,f₄]^T。

S4443, performing generalized average pooling operation on the plurality of feature vectors to obtain feature vectors subjected to generalized pooling operation

Obtaining the characteristic vector subjected to the generalized average pooling operation by the following generalized average pooling formula:

wherein p isⁱThe pooling for the generalized average contains a learnable parameter with pi set to 1.0 at its initial value and x representing the input feature, i.e., the feature vector f. The average pooling averages the input features, rather than the maximum pooling taking the maximum value for the input features, the generalized average pooling contains a learnable parameter pⁱFirst, p is obtained for the input featuresⁱTo the power of the order of p, then taking the mean value, then pⁱA second order, more compact features can be obtained, and the matching precision, p, is improvedⁱThe initial value is set to 1.0. In the present embodiment, n is 4.

S4445, vertically splicing the plurality of feature vectors into a single feature vector serving as the local feature f _ local. And respectively sending the 4 features with different scales into a unified aggregation gate for dynamic feature weighting, finally fusing the 4 weighted features and performing dimension raising on the number of feature channels to obtain a local feature vector which is consistent with the input size, namely the local feature f _ local.

The global feature is extracted through the following sub-steps:

s4441', processing the human body region detection frame to obtain a corresponding characteristic diagram;

s4442', performing generalized average pooling operation on the feature map to obtain a global feature f _ global after the generalized pooling operation;

in this embodiment, the average pooling in gap modules within a replacement global branch is generalized average pooling, where p in generalized average poolingⁱThe initial value was set to 6.5.

s45, judging whether the abnormal posture is the abnormal posture which appears in the same human body for the first time by judging whether the abnormal target list is empty or not; if the abnormal target list is empty, the step S46 is carried out, otherwise, the step S47 is carried out;

s46, storing Reid characteristics, abnormal posture types and abnormal times corresponding to the abnormal postures of the human body and the area position of the human body in the target image as abnormal target data, recording the positions, posture types and abnormal times of the abnormal postures stored in the abnormal target list and the last updating time, and turning to the step S41;

s47, finding out the area with the largest intersection ratio of the area positions corresponding to the abnormal posture from the abnormal target data, and comparing the Reid characteristic similarity of the areas;

in this step, the cosine distances of their reid features are compared as the similarity between the features. Referring to fig. 12, fig. 12 is a diagram illustrating the effect of feature extraction and feature retrieval performed by the present pedestrian re-identification module. After the pedestrian re-identification module obtains the one-dimensional characteristics of each picture, the direct similarity (between 0 and 1) of the pictures can be obtained through a residual similarity calculation formula, and the retrieval process is completed by sequencing according to the similarity.

S48, judging whether the similarity exceeds a first preset threshold value; if the first preset threshold value is not exceeded, directly switching to the step S46, and using the current abnormal target data as new abnormal target data (i.e. another new abnormal target data packet);

s48a, if the similarity exceeds a first preset threshold, turning to the step S46 to update the abnormal target data (namely, the abnormal target data in the original abnormal target data packet) and judging whether the abnormal frequency of the abnormal posture exceeds a second preset threshold; if the result exceeds the preset value, the step S49 is carried out, otherwise, the step S41 is carried out;

and S49, if the second preset threshold value is exceeded, generating an abnormal posture early warning.

The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A human body abnormal posture detection method comprises the following steps:

s11, acquiring a target image;

s13, judging whether the human body posture is in an abnormal posture or not;

s18, judging whether the similarity exceeds a first preset threshold value;

2. The human body abnormal posture detecting method as claimed in claim 1, wherein said step of S10 includes the substeps of:

3. The human body abnormal posture detecting method as claimed in claim 1, wherein: in the step S12, detecting the region position and the body posture of the body in the target image through an abnormal posture detection network, where the abnormal posture detection network includes a feature extraction network and a detection head network, and the feature extraction network includes several layers of convolutional layers; the detection head network comprises a positioner and a classifier; the step of S12 includes the following substeps:

4. The method for detecting an abnormal posture of a human body according to claim 3, further comprising, after the step of S13:

s143, rotating the human body area detection frame clockwise by 90 degrees;

5. The human body abnormal posture detecting method as claimed in claim 4, wherein in the step S144, the substeps of:

s1442, horizontally dividing the feature map into a plurality of blocks to obtain a plurality of feature vectors f ═ f₁，f₂，...，f_n]^T；

6. The abnormal human body posture detecting method of claim 5, wherein in the step S1442, the generalized mean pooling operation is obtained by the following formula:

7. The human body abnormal posture detecting method as claimed in claim 5, wherein in the step S144, the global feature extraction includes the sub-steps of:

8. The method for detecting the abnormal posture of the human body according to any one of the claims 1 to 6, wherein in the step of S18, if the first preset threshold is not exceeded, the method proceeds to the step of S16.

9. The method for detecting an abnormal posture of a human body according to claim 7, further comprising, after said step of S18:

10. The method for detecting an abnormal posture of a human body according to claim 7, further comprising, before the step of S11:

s10, initializing an abnormal target list;

in step S16, storing the abnormal posture category and the abnormal frequency corresponding to the abnormal posture of the human body and the area position of the human body in the target image into the abnormal target list as abnormal target data, recording the position, posture category, abnormal frequency and last update time of the abnormal posture stored in the abnormal target list, and proceeding to step S11;

in step S17, a region having the largest intersection ratio of the region positions corresponding to the abnormal posture is found from the abnormal target data in the abnormal target list, and the similarity of the abnormal postures is compared.

11. The method for detecting an abnormal posture of a human body according to claim 9, further comprising, after the step of S13:

12. The human body abnormal posture detecting method as claimed in claim 3, wherein in said step S15, it is determined whether the abnormal posture is an abnormal posture in which the same human body appears for the first time by determining whether the abnormal target list is empty or whether similar abnormal target feature data is not found in the abnormal target list.