CN113743257A - Construction aerial work instability state detection method integrating space-time characteristics - Google Patents
Construction aerial work instability state detection method integrating space-time characteristics Download PDFInfo
- Publication number
- CN113743257A CN113743257A CN202110962120.XA CN202110962120A CN113743257A CN 113743257 A CN113743257 A CN 113743257A CN 202110962120 A CN202110962120 A CN 202110962120A CN 113743257 A CN113743257 A CN 113743257A
- Authority
- CN
- China
- Prior art keywords
- construction
- network model
- network
- human body
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010276 construction Methods 0.000 title claims abstract description 94
- 238000001514 detection method Methods 0.000 title claims abstract description 64
- 230000002776 aggregation Effects 0.000 claims abstract description 19
- 238000004220 aggregation Methods 0.000 claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000003062 neural network model Methods 0.000 claims abstract description 3
- 210000000544 articulatio talocruralis Anatomy 0.000 claims description 26
- 210000004394 hip joint Anatomy 0.000 claims description 24
- 230000004927 fusion Effects 0.000 claims description 13
- 210000000746 body region Anatomy 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 210000001503 joint Anatomy 0.000 claims description 8
- 210000003625 skull Anatomy 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 2
- 230000002093 peripheral effect Effects 0.000 claims description 2
- 210000000323 shoulder joint Anatomy 0.000 claims 1
- 210000000988 bone and bone Anatomy 0.000 abstract description 8
- 238000012549 training Methods 0.000 description 17
- 238000010586 diagram Methods 0.000 description 6
- 210000003141 lower extremity Anatomy 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000005452 bending Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention discloses a construction high-altitude operation instability state detection method fused with space-time characteristics, which comprises the following steps: extracting the time motion characteristics of the detected target, and calculating the motion information of the pixels between frames of the image sequence; extracting the space positioning characteristics of the detection target, and fusing the time movement characteristics to obtain the space coordinate information of the skeletal joint points of the construction workers; and the instability state detection of the aerial work is realized according to the spatial position of the skeletal joint of the construction worker. The method is realized by a time sequence characteristic aggregation network model, a human body space positioning network model and a safety state detection network model cascaded neural network model. The invention integrates space-time characteristics, enriches the spatial relation of human body bone joint points on continuous frames of images by utilizing the time sequence relation of image pixels, and realizes the instability state detection of the construction high-altitude operation by taking the gradient of the connecting line of the bone joint points as a threshold value. The invention has high intelligent level and better application prospect and practical value.
Description
Technical Field
The invention relates to the technical field of computer vision, comprises a deep learning technology, and particularly relates to a construction high-altitude operation instability state detection method integrating space-time characteristics.
Background
The building industry occupies an important position in national economy of China, but has high risk and multiple accidents compared with other industries. Although a large number of safety control measures are established in construction sites, the construction safety of the construction engineering is still serious. Among many types of safety accidents, high-altitude falling is the most important accident type and occupies more than half of the initial accidents. According to statistics, high-altitude falling accidents are easily caused by unstable construction states such as high-altitude body probing operation and the like. If the instability state of the construction worker during the high-altitude operation can be timely and accurately detected and early warned, the high-altitude falling accident can be effectively prevented, and the life safety of the construction worker is guaranteed.
The purpose of safety state detection is to discover high-altitude dangerous states in time, but the current construction site still mainly adopts manual detection, the manual detection efficiency is low, the frequency is low, all-weather and full-coverage detection is difficult to achieve, and a written detection report submitted by the manual detection is difficult to reflect the detection result in detail, so that the potential safety hazard is not discovered and eliminated.
Therefore, in view of the above problems, it is desirable to design an intelligent method for detecting the instability state of overhead work, so as to implement real-time and dynamic detection of the construction work state. And the visual basis is improved for safety management and performance assessment of a construction site.
Disclosure of Invention
Aiming at the problems and the defects in the prior art, the invention aims to provide a construction high-altitude operation instability state detection method integrating space-time characteristics, which is used for realizing the real-time detection of a construction operation state in a dynamic video streaming state.
The technical scheme adopted by the invention is as follows: the method realizes detection through a neural network model cascaded by a time sequence feature aggregation network model, a human body space positioning network model and a safety state detection network model; comprises the following steps:
step S1: extracting the time motion characteristics of a detection target through a time sequence characteristic aggregation network model, and calculating the motion information of the pixels between frames of the image sequence;
step S2: extracting the space positioning characteristics of the detection target through a human body space positioning network model, and fusing the time movement characteristics obtained in the step S1 to obtain the space coordinate information of the skeletal joint points of the construction workers;
step S3: and according to the space coordinate information of the skeletal joints of the construction workers obtained in the step S2, realizing the detection of the instability state of the aerial work through the safety state detection network model.
Preferably, the time sequence feature aggregation network model and the human body space positioning network model in steps S1 and S2 are main feature extraction networks, are connected in parallel, and output continuous frame images with coordinate information and defining the human body area of the construction worker; the safety state detection network model in step S3 is a result judgment output model and is connected in series after the time sequence feature aggregation network model and the human body space positioning network model.
Preferably, the cascade model replaces the normal convolutional layer with a deep separable convolutional layer in addition to the lightweight network structure. The model is a lightweight model and can be loaded into a storage medium for use.
Preferably, the construction high-altitude operation instability state detection method fusing the space-time characteristics inputs the suggested inter-frame relation between the image of the human body area of the construction worker containing the coordinate information and the image into a safety state detection network model, performs integral target characteristic extraction and enhancement on the transmitted image and information, and then realizes construction operation state classification.
Preferably, the time-series feature aggregation network model is composed of an inter-frame connection network and a feature compensation network. The inter-frame contact network calculates the relative relation between adjacent frames according to the change condition of the image pixels on the discrete time sequence; and the characteristic compensation network carries out pixel prediction and compensation on the image loss frame according to the normal image frame and the inter-frame motion to acquire the motion information of the point pixel.
Preferably, the human body space positioning network model consists of a skeleton joint positioning network, a space-time feature fusion network and a human body region suggestion network. Firstly, positioning the human body area of the construction worker in the image by using the skeletal joint points, and then realizing optimal area prediction and key point coordinate output by using a convolutional neural network.
The bone joint positioning network positions and extracts bone joints of construction workers working aloft, predicts 9 key points of the skull, the uncinate vertebra joint, the left and right scapular joints, the sacrum, the left and right hip joints and the left and right ankle joints, and positions the spatial positions of the key points respectively; the space-time feature fusion network realizes the fusion of the time sequence features and the spatial positions of the key points in the step S1, rechecks the spatial positions of the human body key points positioned by each frame of image by combining the adjacent frame pixel motion information input by the time sequence feature aggregation network model, and predicts the positions of the skeletal joint points of the construction workers in the image loss frame; the human body region suggestion network utilizes the spatial position information of 7 key points of the skull, the left and right scapular joints, the left and right hip joints and the left and right ankle joints to return to a peripheral frame of a human body region of a construction worker, a pixel coordinate system is established by taking the lower left corner of a human body region image selected by the frame as an origin, and the coordinate information of each skeletal joint point is extracted.
Preferably, the security state detection network model is composed of a feature enhancement network and a classifier.
The feature enhancement network adopts a mode of superposing feature pyramids in an upside-down inverted manner to realize target feature extraction and enhancement, and the classifier realizes construction state judgment.
Preferably, the construction state classifier is a threshold classifier. Selecting a construction worker construction operation side image, and judging the relative spatial positions of the hip joint and the ankle joint in the image by utilizing the hip joint and ankle joint spatial coordinate information extracted by the human body spatial positioning network model acquired in the step S2, wherein the hip joint coordinate on each side is (x)i,yi) The ankle joint coordinate is (x)j,yj) And when the spatial positions of the left ankle joint and the right ankle joint are positioned at two sides of the two hip joints or in the same vertical direction, judging that the construction operation is normal.
xj1≤xi1≤xj2
xj1≤xi2≤xj2
When the spatial positions of the left and right ankle joints are on the same side relative to the two hip joints, the inclination of the connecting line between the left and right hip joints and the corresponding side ankle joint is used as a construction state judgment index, and when the inclination angle alpha is smaller than an inclination threshold value, the unstable construction operation is judged, otherwise, the normal construction operation is judged.
Preferably, the inclination threshold is obtained by a clustering method, and the inclination threshold is 60-70 degrees.
Preferably, the detection method is used for dynamic video detection, the human body area of a construction worker framed by a human body area suggestion network is utilized, and construction operation state result judgment is realized in the video, and the method comprises two state labels of Normal-operation and Unstable-operation.
The invention has the beneficial effects that:
1. the invention can realize the real-time dynamic detection of the construction operation state under the high-altitude operation environment. Compared with the prior art, the detection range is wider, and the detection precision is higher. The efficiency and the intelligent level of construction safety detection are improved.
2. The invention fuses the space-time characteristic information on the basis of target characteristic extraction. The spatial relation of human body bone joint points on continuous frames of the image is enriched through the motion information of the image pixels on the time sequence, and the detection precision of the method is improved. The construction safety state is detected by utilizing the inclined threshold value of the bone joint point, and detection means are enriched.
3. The invention uses the lightweight detection module, accelerates the detection speed of the method, and can be carried on the mobile equipment configured by the CPU for use.
Drawings
FIG. 1 is a schematic diagram of a detection model cascade structure of a construction aerial work instability state detection method with fusion of space-time characteristics according to an embodiment of the present invention;
FIG. 2 is a flow chart of a detection model training process of a construction high-altitude operation instability state detection method with fusion of space-time characteristics according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating the operation of a method for detecting instability of construction aerial work incorporating spatio-temporal features according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a construction work state discrimination threshold calculation according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a single-frame image detection result according to an embodiment of the present invention.
Detailed Description
The present invention is further described in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are provided for illustration only and are not intended to limit the present invention.
Fig. 1 shows a detection model cascade connection diagram of a method for detecting a instability state of a construction aerial work according to an embodiment of the present invention, which is detailed as follows:
the detection model is formed by cascading a time sequence characteristic aggregation network model, a human body space positioning network model and a safety state detection network model.
The time sequence feature gathering network model and the human body space positioning network model are main feature extraction networks and are connected in parallel. The time sequence characteristic gathering network model is used for extracting time motion information characteristics and outputting the inter-frame relation of the images and the motion information of the pixel points; the human body space positioning network model is used for extracting space coordinate positioning characteristics, outputting continuous frame images with coordinate information and defining human body areas of construction workers. The safety state detection network model is a result judgment output model and is connected in series behind the time sequence feature aggregation network model and the human body space positioning network model to realize feature extraction, enhancement and result prediction.
The time sequence feature aggregation network model is formed by connecting an inter-frame connection network and a feature compensation network in series. In the embodiment of the invention, the interframe connection network adopts a contraction part structure of a FlowNet C optical flow network, and the relationship between adjacent interframes is calculated according to the change condition of image pixels on a discrete time sequence; the characteristic compensation network consists of a plurality of layers of transposition convolution and expansion convolution layers, and pixel compensation is carried out on the construction worker image frames with the shelters or the blurs by combining the obtained interframe relation with the normal image frames to obtain the motion information of the point pixels. And predicting target pixel information of the image loss frame and outputting a complete inter-frame time sequence characteristic relation.
The human body space positioning network model is formed by connecting a skeleton joint positioning network, a space-time characteristic fusion network and a human body region suggestion network in series. In the embodiment of the invention, the skeleton joint positioning network adopts an Openpos key point prediction network, and replaces an original VGG-19 network with a MobileNet V2 light convolutional neural network, so as to predict a 2D confidence map of 9 key point positions of a construction worker skull, a uncinate joint, left and right scapular joints, a sacrum, left and right hip joints and left and right ankle joints, and position the optimal confidence point position of each key point; the spatial-temporal feature fusion network firstly adopts an SPP structure, enlarges the receptive field by a multi-scale pooling mode, realizes feature fusion by combining a Concat tensor splicing mode, rechecks the human body key point space position positioned by each frame of image by combining adjacent frame pixel motion information input by a time sequence feature aggregation network model, and predicts the bone joint point position of a construction worker in an image loss frame; the human body region suggestion network is composed of a convolutional layer and a full connection layer, the full connection layer is used for conducting outer frame regression on the whole body outline of a construction worker on the obtained spatial position information of 7 key points of the skull, the left and right scapular joints, the left and right hip joints and the left and right ankle joints, then the optimal result of human body region prediction is output by combining non-maximum value suppression, a pixel coordinate system is established by taking the lower left corner of a human body region image divided by the frame as an origin, and the coordinate information of each bone key point is extracted.
The safety state detection network model is formed by connecting a feature enhancement network and a classifier in series. In the embodiment of the invention, the characteristic enhancement network realizes repeated extraction of target characteristics by utilizing a PANet structure and in a mode of superposing characteristic pyramids in an upside-down inverted manner, thereby playing a characteristic enhancement role; the classifier uses an SVM (support vector machine) to realize construction state judgment.
In the embodiment of the invention, besides introducing a light network structure, the cascade model also uses a depth separable convolutional layer to replace a common 3 × 3 convolutional layer, and reduces model parameters in a way of channel-by-channel convolution and point-by-point convolution. The cascade model is a lightweight model and can be loaded into storage media such as a hard disk, a U disk and the like for use.
Before the model is operated, the model is required to be trained in a network mode by combining an image data set. Fig. 2 shows a model training flowchart of the method for detecting a instability state of construction aerial work according to the embodiment of the present invention, which includes the following detailed steps:
step S1: and collecting high-altitude operation videos of construction workers, and preprocessing the images.
The embodiment of the invention obtains the high-altitude operation video of the construction worker in a network retrieval and field acquisition mode, and the video requires to display the whole body of the construction worker. Dividing the video into sequence images according to frames, and enhancing the image data quality by utilizing a light homogenizing processing mode and a denoising processing mode.
Step S2: constructing an image dataset
According to the embodiment of the invention, the processed image is labeled by using a Labelme tool to generate a json file to complete the construction of the image data set.
Step S3: detection model building, training and weight saving
According to the embodiment of the invention, the time sequence feature aggregation network model, the human body space positioning network model and the safety state detection network model are connected and built according to the cascade mode. And loading the constructed image data set to train the network model, dividing the data set into a test set and a verification set according to a proportion, verifying the training effect of the verification set after finishing the training of each batch, and adjusting the model parameters to train again if the verification effect is not good. The network parameters were adjusted before training, setting the Batch size (Batch-size) to 32 and the training round (Epoch) to 200 rounds. In the training process, a Transfer Learning training method (Transfer Learning) is adopted, and the network weights trained by the model in other training data sets are used as the initial weights of the classification network for training, so that the training time and the memory consumption are reduced, and the classification precision of the algorithm on a small data set is improved; the training is set to Early Stopping (Early Stopping), the process takes the variation condition of the loss value as a progress measuring index, and when the loss value is converged, the training is completely represented, namely the training is stopped, and the overfitting of the training is prevented. And finally, saving the optimal training weight for method detection.
Fig. 3 shows an operation flow chart of the method for detecting the instability state of the construction aerial work, which is provided by the embodiment of the invention, and the detailed steps are as follows:
firstly, loading collected side videos of high-altitude operation of construction workers into a cascade detection model, and converting the videos into sequence images by the model according to frames; the sequence image enters a human body space positioning network model to predict the space positions of 9 skeletal joint points, simultaneously, a time sequence feature aggregation network model performs pixel compensation and motion information extraction on the sequence image, a space-time feature fusion network rechecks the predicted space positions of human body key points of each frame of image according to the pixel motion information, and predicts the skeletal joint point positions of construction workers in the image loss frame. The human body space positioning network model realizes the extraction of skeleton key points and the segmentation of human body image areas according to space-time characteristics, outputs continuous frame images with coordinate information and defines the human body areas of construction workers. The safety state detection network model realizes target feature enhancement on the input features, and realizes construction operation state discrimination by calculating the position relation and the inclination threshold of the joint points.
Fig. 4 shows a schematic diagram of calculating a construction operation state threshold according to an embodiment of the present invention, which is detailed as follows:
in the embodiment of the invention, a coordinate system is established by taking the lower left corner of the suggested human body image of the construction worker as a coordinate origin, the coordinate system takes pixels as unit length, and the relative spatial positions of hip joints and ankle joints in the image are judged by utilizing hip joint and ankle joint coordinate information of the side image of the construction worker extracted by a human body space positioning network model, wherein the coordinate of each hip joint is (x)i,yi) The ankle joint coordinate is (x)j,yj) When the spatial positions of the left ankle joint and the right ankle joint are positioned on two sides of the two hip joints or in the same vertical direction, the human body can realize the support balance, and then the normal construction operation is judged.
xj1≤xi1≤xj2
xj1≤xi2≤xj2
When the spatial positions of the left and right ankle joints are on the same side relative to the two hip joints, the inclination of the connecting line between the left and right hip joints and the corresponding side ankle joint is used as a construction state judgment index, and the calculation formula of the inclination angle alpha is as follows:
according to the embodiment of the invention, the inclination threshold is obtained in a clustering mode, the inclination threshold is set to be 65 degrees, when the lower limbs on two sides of a worker are superposed in the side image (the lower limbs on two sides in the side image are combined and the same bending angle is kept), judgment is carried out by using one-side inclination, and when the inclination angle is smaller than the inclination threshold, unstable construction operation is judged. And when the lower limbs on the two sides of the worker do not coincide in the side image, judging by the inclination angles on the two sides together, judging as unstable construction operation when the inclination angles are smaller than the inclination threshold value, and otherwise, judging as normal construction operation.
Fig. 5 shows a schematic diagram of a single-frame image detection result provided by the embodiment of the present invention, which is detailed as follows:
the embodiment of the invention realizes the visual display of the judgment result of the construction operation state. The construction operation state judgment result comprises two state labels of Normal-operation and Unstable-operation, and the human body area is selected by the frame to suggest the human body area of the construction worker defined by the network to realize result output.
According to the embodiment of the invention, different detection results are output in different color frames in a distinguishing manner, yellow represents normal construction operation, and red represents unstable construction operation.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.
Claims (6)
1. A construction high-altitude operation instability state detection method fused with space-time characteristics is realized by a time sequence characteristic aggregation network model, a human body space positioning network model and a neural network model cascaded with a safety state detection network model, and the method comprises the following steps:
step S1: extracting the time motion characteristics of a detection target through a time sequence characteristic aggregation network model, and calculating the motion information of the pixels between frames of the image sequence;
step S2: extracting the space positioning characteristics of the detection target through a human body space positioning network model, and fusing the time movement characteristics obtained in the step S1 to obtain the space coordinate information of the skeletal joint points of the construction workers;
step S3: and according to the space coordinate information of the skeletal joints of the construction workers obtained in the step S2, realizing the detection of the instability state of the aerial work through the safety state detection network model.
2. The method for detecting the instability state of construction high-altitude operation with fusion of spatio-temporal features as claimed in claim 1, wherein in steps S1 and S2, the time series feature aggregation network model and the human body space positioning network model are used as main feature extraction networks and are connected in parallel; in step S3, the security state detection network model is a result determination output model and is connected in series after the timing characteristic aggregation network model and the human body space positioning network model.
3. The model for detecting the instability state of construction high-altitude operation fused with spatio-temporal characteristics according to claim 1, wherein the time-series characteristic aggregation network model is composed of an inter-frame connection network and a characteristic compensation network;
the inter-frame contact network calculates the relative relation between adjacent frames according to the change condition of the image pixels on the discrete time sequence;
and the characteristic compensation network carries out pixel prediction and compensation on the image loss frame according to the relation between the normal image frame and the frame to acquire the motion information of the point pixel.
4. The model for detecting the instability state of construction high-altitude operation fused with spatio-temporal characteristics according to claim 1, wherein the human space positioning network model consists of a skeletal joint positioning network, a spatio-temporal characteristic fusion network and a human region suggestion network;
the skeletal joint positioning network positions and extracts skeletal joint points of construction workers in the sequence images, predicts 9 key points of the skull, the uncinate vertebra joint, the left and right scapular joints, the sacrum, the left and right hip joints and the left and right ankle joints, and positions the spatial positions of the key points respectively;
the space-time feature fusion network realizes the fusion of the time sequence features and the spatial positions of the key points in the step S1, performs rechecking and adjustment on the spatial positions of the human key points positioned by each frame of image by combining the adjacent frame pixel motion information input by the time sequence feature aggregation network model, and predicts the positions of the skeletal joint points of construction workers in the lost image;
the human body region suggestion network utilizes the spatial position information of 7 key points of the skull, the left and right shoulder joints, the left and right hip joints and the left and right ankle joints to return to a peripheral frame of a human body region of a construction worker, a pixel coordinate system is established by taking the lower left corner of a human body region image selected by the frame as an origin, and the spatial coordinate information of each skeletal joint point is extracted.
5. The construction high-altitude operation instability state detection model fused with spatio-temporal features as claimed in claim 1, wherein the safety state detection network model is composed of a feature enhancement network and a construction state classifier;
the construction state classifier is a threshold classifier, a construction worker construction operation side image is selected, the hip joint and ankle joint space coordinate information extracted by the human body space positioning network model obtained in the step S2 is utilized, the relative space positions of the hip joint and the ankle joint in the image are judged, wherein the hip joint coordinate on each side is (x) coordinatei,yi) The ankle joint coordinate is (x)j,yj) When the spatial positions of the left and right ankle joints are positioned at both sides of the two hip joints or in the same vertical directionJudging the construction is normal;
xj1≤xi1≤xj2
xj1≤xi2≤xj2
when the spatial positions of the left and right ankle joints are on the same side relative to the two hip joints, the inclination of the connecting line between the left and right hip joints and the corresponding side ankle joint is used as a construction state judgment index, and when the inclination angle alpha is smaller than an inclination threshold value, the instability construction operation is judged, otherwise, the normal construction operation is carried out;
6. the construction high-altitude operation instability state detection model fused with the spatio-temporal characteristics according to claim 5, wherein the tilt threshold is obtained through a clustering mode, and is 60-70 degrees.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110962120.XA CN113743257B (en) | 2021-08-20 | Construction overhead operation instability state detection method integrating space-time characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110962120.XA CN113743257B (en) | 2021-08-20 | Construction overhead operation instability state detection method integrating space-time characteristics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113743257A true CN113743257A (en) | 2021-12-03 |
| CN113743257B CN113743257B (en) | 2024-05-14 |
Family
ID=
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084161A (en) * | 2019-04-17 | 2019-08-02 | 中山大学 | A kind of rapid detection method and system of skeleton key point |
| CN110135319A (en) * | 2019-05-09 | 2019-08-16 | 广州大学 | A kind of anomaly detection method and its system |
| WO2021057027A1 (en) * | 2019-09-27 | 2021-04-01 | 北京市商汤科技开发有限公司 | Human body detection method and apparatus, computer device, and storage medium |
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084161A (en) * | 2019-04-17 | 2019-08-02 | 中山大学 | A kind of rapid detection method and system of skeleton key point |
| CN110135319A (en) * | 2019-05-09 | 2019-08-16 | 广州大学 | A kind of anomaly detection method and its system |
| WO2021057027A1 (en) * | 2019-09-27 | 2021-04-01 | 北京市商汤科技开发有限公司 | Human body detection method and apparatus, computer device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103839065B (en) | Extraction method for dynamic crowd gathering characteristics | |
| CN103886308B (en) | A kind of pedestrian detection method of use converging channels feature and soft cascade grader | |
| CN107038422A (en) | The fatigue state recognition method of deep learning is constrained based on space geometry | |
| CN104951773A (en) | Real-time face recognizing and monitoring system | |
| CN112990077B (en) | Face action unit identification method and device based on joint learning and optical flow estimation | |
| CN109886159B (en) | Face detection method under non-limited condition | |
| CN113093726A (en) | Target detection and tracking method based on Yolo _ v4 algorithm | |
| CN113553979B (en) | Safety clothing detection method and system based on improved YOLO V5 | |
| CN103729620B (en) | A kind of multi-view pedestrian detection method based on multi-view Bayesian network | |
| CN106056624A (en) | Unmanned aerial vehicle high-definition image small target detecting and tracking system and detecting and tracking method thereof | |
| WO2023087420A1 (en) | Thermal infrared vision-based parking apron human body action recognition method and system | |
| Chen et al. | An intelligent sewer defect detection method based on convolutional neural network | |
| CN111008994A (en) | Moving target real-time detection and tracking system and method based on MPSoC | |
| CN111738336A (en) | Image detection method based on multi-scale feature fusion | |
| CN109902613A (en) | A kind of human body feature extraction method based on transfer learning and image enhancement | |
| CN114689600A (en) | Method and system for detecting surface crack of bridge concrete structure | |
| CN114495068A (en) | Road surface health detection 'element' method based on man-machine interaction and deep learning | |
| CN114399734A (en) | Forest fire early warning method based on visual information | |
| Daogang et al. | Anomaly identification of critical power plant facilities based on YOLOX-CBAM | |
| CN113743257A (en) | Construction aerial work instability state detection method integrating space-time characteristics | |
| CN113743257B (en) | Construction overhead operation instability state detection method integrating space-time characteristics | |
| CN101877135A (en) | Moving target detecting method based on background reconstruction | |
| CN116310967A (en) | Chemical plant safety helmet wearing detection method based on improved YOLOv5 | |
| CN112819849B (en) | Mark point-free visual motion capture method based on three eyes | |
| CN111402223B (en) | Transformer substation defect problem detection method using transformer substation video image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |