CN112990149A - Multi-mode-based high-altitude safety belt detection method, device, equipment and storage medium - Google Patents
Multi-mode-based high-altitude safety belt detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112990149A CN112990149A CN202110498036.7A CN202110498036A CN112990149A CN 112990149 A CN112990149 A CN 112990149A CN 202110498036 A CN202110498036 A CN 202110498036A CN 112990149 A CN112990149 A CN 112990149A
- Authority
- CN
- China
- Prior art keywords
- image
- altitude
- personnel
- safety belt
- thermal imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The application provides a multi-mode-based high-altitude safety belt detection method, a multi-mode-based high-altitude safety belt detection device, high-altitude safety belt detection equipment and a storage medium, wherein the method comprises the following steps: acquiring an aloft personnel working diagram acquired by a first imaging device and an aloft personnel thermal imaging diagram acquired by a second imaging device; calibrating the high-altitude personnel thermal imaging image according to the RGB image to obtain a calibrated thermal imaging image; extracting image features of the aerial worker working diagram from a network according to preset image features and taking the image features as first image features; extracting image features of the calibrated thermal imaging image and taking the image features as second image features; determining a human-shaped image of the high-altitude personnel according to the first image characteristic and the second image characteristic; determining images of key parts of high-altitude personnel according to the human-shaped images of the high-altitude personnel; and detecting whether the high-altitude personnel wear the safety belt or not according to the image of the key part of the high-altitude personnel. The safety belt identification method and the safety belt identification device can quickly and accurately identify whether the safety belt of the high-altitude personnel is in standard wearing or not, and have better robustness.
Description
Technical Field
The application relates to the technical field of computers, in particular to a multi-mode-based high-altitude safety belt detection method, device, equipment and storage medium.
Background
With the development of society, the requirements of enterprises on safety production and safety construction are more and more clear, and safety production is regulated, so that safety helmets and safety belts must be worn during overhead operation. Wherein the safety belt is because of the shape is changeable, wears the rule requirement occasionally, if wear the unnormal, when high altitude construction, has life potential safety hazard, and when high altitude construction, because of the safety belt bearing problem, the condition that certain root belt fracture of safety belt and staff are unknown takes place occasionally. Therefore, the situation of wearing the safety belt of the high-altitude worker constantly monitored is about the life and property safety of the worker and is not small.
At present, for safety belt detection, manual detection is mainly used, or through traditional visual detection, although research exists, the detection precision is not high, so with the development of deep learning, more and more researchers consider using target detection to detect the wearing and wearing specifications of safety belts, and in high altitude, the posture change of people is large, and if the detection is only through the target detection, missing detection and false detection easily occur.
Further, in the conventional method, algorithms such as an SVM (support vector machine), kmean and the like are used for target detection, but the detection robustness is poor, and when the angle or illumination changes, the detection accuracy is greatly reduced. When the thermal imaging graph of the infrared thermal imaging instrument is only used for detection, false detection easily occurs only through the temperature due to the outdoor complicated environment. In recent years, with the development of deep learning, more and more researchers consider using deep convolutional networks to perform detection tasks, such as SSD [1], YOLO [2], fast RCNN [3], etc., and the accuracy rate of safety belt detection based on neural networks is greatly improved compared with the traditional algorithm. However, if the safety belt is directly detected, the safety belt is directly detected due to the complex outdoor environment, particularly in a construction site and in the power industry, false detection is easy, in addition, in high altitude, the posture change of people is large, the characteristics are easy to be covered, and false detection and missing detection are easy to occur.
Disclosure of Invention
The embodiments of the present application provide a method, an apparatus, a device, and a storage medium for detecting a high-altitude safety belt based on multiple modes, so as to quickly and accurately identify whether a safety belt of a high-altitude person is normal or not, and have better robustness.
To this end, the present application discloses in a first aspect a multi-modal based high altitude seat belt detection method, the method comprising:
acquiring an aloft personnel working diagram acquired by a first imaging device and an aloft personnel thermal imaging diagram acquired by a second imaging device;
calibrating the high-altitude personnel thermal imaging image according to the RGB image to obtain a calibrated thermal imaging image;
extracting image features of the high-altitude personnel working diagram according to a preset image feature extraction network and taking the image features as first image features;
extracting image features of the calibrated thermal imaging image and taking the image features as second image features;
determining a human-shaped image of the high-altitude personnel according to the first image characteristic and the second image characteristic;
determining images of key parts of the high-altitude personnel according to the human-shaped images of the high-altitude personnel;
and detecting whether the high-altitude personnel wear the safety belt or not according to the image of the key part of the high-altitude personnel.
Compared with the prior art, the method can integrate the RGB image and the thermal imaging image through multi-mode input, greatly improve the detection precision of people through complementarity, and the prior art only performs people detection through a certain mode, so that the detection omission is easily subjected to false detection only through input of a certain mode on the premise of complicated outdoor high-altitude environment. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, the key points of the key points are obtained by detecting the key points of the person firstly, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt detecting method has better detecting accuracy and robustness.
In the first aspect of the present application, as an optional implementation manner, the determining a human-shaped image of the high-altitude person according to the first image feature and the second image feature includes:
based on an attention mechanism, performing feature fusion on the first image feature and the second image feature to obtain a fusion feature;
and determining the human-shaped image of the high-altitude personnel according to the fusion characteristics.
In the optional embodiment, the attention mechanism can adapt to the condition that human-shaped image detection under different scenes depends on different RGB images and thermal imaging graphs, and further adapt to the detection of people under different scenes through the fusion of modal feature graphs with different weights, so that the detection accuracy is further improved.
In the first aspect of the present application, as an optional implementation manner, the determining the humanoid image of the high-altitude person according to the fusion feature includes:
determining the regional position of the high-altitude personnel according to the fusion characteristics;
and cutting out the image part of the regional position of the high-altitude personnel to be used as a human-shaped image of the high-altitude personnel.
In this optional embodiment, the region position of the high-altitude person can be determined according to the fusion feature, and then the image part where the region position of the high-altitude person is located is cut out and used as the human-shaped image of the high-altitude person.
In the first aspect of the present application, as an optional implementation manner, the preset image feature extraction network is one of VGG, google lenet, ResNet, ResNeXt, and densneet.
In the first aspect of the present application, as an optional implementation manner, after the detecting whether the high-altitude person wears a seat belt according to the image of the key part of the high-altitude person, the method further includes:
when the high-altitude personnel wear the safety belt, detecting whether the wearing mode of the safety belt worn by the high-altitude personnel meets the preset condition according to the image of the key part of the high-altitude personnel.
In the optional embodiment, the detection of the wearing mode of the safety belt worn by the high-altitude personnel can be realized.
In the first aspect of the present application, as an optional implementation manner, the determining an image of the key part of the high-altitude person according to the humanoid image of the high-altitude person includes:
and identifying the human-shaped image of the high-altitude personnel according to a human body key part detection network in CPM, alpha Pose, OpenPose and Retinaface to obtain an image of the high-altitude personnel key part, wherein the image of the high-altitude personnel key part comprises images of a human body left shoulder, a human body right shoulder, a human body left crotch, a human body right crotch and a human body center.
In the optional embodiment, the human figure image of the high-altitude personnel can be identified according to one human body key part detection network of CPM, alpha Pose, OpenPose and Retinaface.
In the first aspect of the present application, as an optional implementation manner, the detecting whether the high-altitude person wears a safety belt according to the image of the key part of the high-altitude person includes;
the method comprises the steps of taking an image of a key part of a high-altitude person as an input of a safety belt detection model, so that the safety belt detection model identifies whether the high-altitude person wears a safety belt, wherein the safety belt detection model is one of a network mode of YOLO, SSD and fast-RCNN.
In this alternative embodiment, it is possible to identify whether the high-altitude person wears a seat belt by a seat belt detection model.
A second aspect of the present application discloses a multi-modality based high altitude safety belt detection apparatus, the apparatus comprising:
the acquiring module is used for acquiring an overhead personnel working diagram acquired by the first imaging device and an overhead personnel thermal imaging diagram acquired by the second imaging device;
the calibration module is used for calibrating the high-altitude personnel thermal imaging image according to the RGB image to obtain a calibrated thermal imaging image;
the first feature extraction module is used for extracting image features of the high-altitude personnel working diagram according to a preset image feature extraction network and taking the image features as first image features;
the second feature extraction module is used for extracting the image features of the calibrated thermal imaging image and taking the image features as second image features;
the first determining module is used for determining a human-shaped image of the high-altitude personnel according to the first image characteristic and the second image characteristic;
the second determining module is used for determining the image of the key part of the high-altitude personnel according to the human-shaped image of the high-altitude personnel;
and the detection module is used for detecting whether the high-altitude personnel wear the safety belt or not according to the image of the key part of the high-altitude personnel.
Compared with the prior art, the device can be used for integrating the RGB image and the thermal imaging image through multi-mode input, greatly improving the detection precision of people through complementarity, and the prior art only performs detection of people through a certain mode, so that the detection omission is easily subjected to false detection only through input of a certain mode on the premise of complicated outdoor high-altitude environment. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, the key points of the key points are obtained by detecting the key points of the person firstly, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt detecting method has better detecting accuracy and robustness.
A third aspect of the present application discloses a multi-modality based high altitude safety belt detection apparatus, the apparatus comprising:
a processor; and
a memory configured to store machine readable instructions which, when executed by the processor, cause the processor to perform a multi-modality based high altitude seat belt detection method as disclosed in the first aspect of the present application.
Compared with the prior art, the device can be used for integrating the RGB image and the thermal imaging image through multi-mode input, greatly improving the detection precision of people through complementarity, and the prior art only performs people detection through a certain mode, so that the device is easy to perform false detection and miss detection only through input of a certain mode on the premise of complicated outdoor high-altitude environment. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, the key points of the key points are obtained by detecting the key points of the person firstly, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt detecting method has better detecting accuracy and robustness.
A fourth aspect of the present application discloses a storage medium storing a computer program for execution by a processor of the multi-modality based high altitude seat belt detection method of the first aspect of the present application.
Compared with the prior art, the detection precision of people is greatly improved by integrating the RGB image and the thermal imaging image through multi-mode input in the method, the detection of people is only performed through a certain mode in the prior art, and then the detection is easy to perform false detection and missed detection only through a certain mode input on the premise of complicated outdoor high-altitude environment. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, the key points of the key points are obtained by detecting the key points of the person firstly, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt detecting method has better detecting accuracy and robustness.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic flow chart of a multi-modal-based high-altitude safety belt detection method according to an embodiment of the present application;
fig. 2 is a schematic view of a seat belt identification scenario provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a multi-modal-based high-altitude safety belt detection device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a multi-modal-based high-altitude safety belt detection device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
Example one
Referring to fig. 1, fig. 1 is a schematic flow chart of a multi-modal-based high-altitude safety belt detection method disclosed in an embodiment of the present application. As shown in fig. 1, an embodiment of the present application discloses a high-altitude safety belt detection method based on multiple modalities, including the steps of:
101. acquiring an aloft personnel working diagram acquired by a first imaging device and an aloft personnel thermal imaging diagram acquired by a second imaging device;
102. calibrating the high-altitude personnel thermal imaging image according to the RGB image to obtain a calibrated thermal imaging image;
103. extracting image features of the aerial worker working diagram from a network according to preset image features and taking the image features as first image features;
104. extracting image features of the calibrated thermal imaging image and taking the image features as second image features;
105. determining a human-shaped image of the high-altitude personnel according to the first image characteristic and the second image characteristic;
106. determining images of key parts of high-altitude personnel according to the human-shaped images of the high-altitude personnel;
107. and detecting whether the high-altitude personnel wear the safety belt or not according to the image of the key part of the high-altitude personnel.
As shown in fig. 2, compared with the prior art, the method of the embodiment of the present application can integrate RGB images and thermal imaging images through multi-modal input, and greatly improve human detection accuracy through complementarity, whereas the prior art only performs human detection through a certain modality, and further easily performs false detection and missing detection through only one modality input on the premise of complicated outdoor high-altitude environment. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, the key points of the key points are obtained by detecting the key points of the person firstly, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt detecting method has better detecting accuracy and robustness.
In the embodiment of the present application, as an optional implementation manner, step 105: determining a humanoid image of the high-altitude person from the first image feature and the second image feature, comprising:
based on an attention mechanism, performing feature fusion on the first image feature and the second image feature to obtain fusion features;
and determining the human-shaped image of the high-altitude personnel according to the fusion characteristics.
In this alternative embodiment, based on the attention mechanism, the weights of the first image feature and the second image feature in the fusion process may be determined according to the degree of dependence on the first image feature and the second image feature in different scenes.
In the optional embodiment, the attention mechanism can adapt to the condition that human-shaped image detection under different scenes depends on different RGB images and thermal imaging graphs, and further adapt to the detection of people under different scenes through the fusion of modal feature graphs with different weights, so that the detection accuracy is further improved.
In the embodiment of the present application, as an optional implementation manner, the steps of: determining a human-shaped image of the high-altitude personnel according to the fusion characteristics, comprising:
determining the region position of the high-altitude personnel according to the fusion characteristics;
and cutting out the image part according to the area position of the high-altitude personnel and using the image part as a human-shaped image of the high-altitude personnel.
In this optional embodiment, the region position of the high-altitude person can be determined according to the fusion feature, and then the image part where the region position of the high-altitude person is located is cut out and used as the human-shaped image of the high-altitude person.
In the embodiment of the present application, as an optional implementation manner, the preset image feature extraction network is one of VGG, google lenet, ResNet, ResNeXt, and DenseNet. It should be noted that reference is made to the prior art for the description and training process of VGG, google lenet, ResNet, rennext, and densneet.
In this embodiment, as an optional implementation manner, after detecting whether the high-altitude person wears the seat belt according to the image of the key part of the high-altitude person in step 107, the method of this embodiment further includes the steps of:
when the high-altitude personnel wear the safety belt, detecting whether the wearing mode of the safety belt worn by the high-altitude personnel meets the preset condition according to the image of the key part of the high-altitude personnel.
In the optional embodiment, the detection of the wearing mode of the safety belt worn by the high-altitude personnel can be realized.
In the embodiment of the present application, as an optional implementation manner, step 105: determining images of key parts of high-altitude personnel according to the human-shaped images of the high-altitude personnel, wherein the images comprise:
the human figure image of the high-altitude personnel is identified according to a human body key part detection network in CPM, AlphaPose, OpenPose and Retinaface to obtain an image of a key part of the high-altitude personnel, wherein the image of the key part of the high-altitude personnel comprises images of a left shoulder, a right shoulder, a left crotch, a right crotch and a center of a human body.
In the optional embodiment, the human figure image of the high-altitude personnel can be identified according to one human body key part detection network of CPM, alpha Pose, OpenPose and Retinaface.
It should be noted that, please refer to the prior art for a detailed description of a human body key site detection network in CPM, alphapos, openpos, Retinaface.
In the embodiment of the application, as an optional implementation manner, detecting whether a high-altitude person wears a safety belt according to an image of a key part of the high-altitude person includes;
and taking the image of the key part of the high-altitude personnel as the input of a safety belt detection model so that the safety belt detection model identifies whether the high-altitude personnel wear a safety belt or not, wherein the safety belt detection model is one of a network mode of YOLO, SSD and fast-RCNN.
In this alternative embodiment, whether the high-altitude person wears the safety belt can be identified by the safety belt detection model.
Example two
Referring to fig. 3, fig. 3 is a schematic structural diagram of a multi-modal-based high-altitude safety belt detection device according to an embodiment of the present application. As shown in fig. 3, the apparatus of the embodiment of the present application includes:
the acquiring module 201 is configured to acquire an aloft personnel working diagram acquired by a first imaging device and an aloft personnel thermal imaging diagram acquired by a second imaging device;
the calibration module 202 is configured to calibrate a high-altitude personnel thermal imaging image according to the RGB image to obtain a calibrated thermal imaging image;
the first feature extraction module 203 is used for extracting image features of the aerial worker working diagram according to a preset image feature extraction network and taking the image features as first image features;
a second feature extraction module 204, configured to extract an image feature of the calibrated thermal imaging map and use the image feature as a second image feature;
a first determining module 205, configured to determine a humanoid image of the high-altitude person according to the first image feature and the second image feature;
a second determining module 206, configured to determine an image of a key part of the high-altitude person according to the human-shaped image of the high-altitude person;
and the detection module 207 is used for detecting whether the high-altitude personnel wear the safety belt or not according to the image of the key part of the high-altitude personnel.
Compared with the prior art, the device of the embodiment of the application can be used for integrating the RGB image and the thermal imaging image through multi-mode input, the detection precision of people is greatly improved through complementarity, the detection of people is only performed through a certain mode in the prior art, and then the detection is easy to perform false detection and missed detection only through input of a certain mode on the premise that the outdoor high-altitude environment is complicated. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, the key points of the key points are obtained by detecting the key points of the person firstly, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt detecting method has better detecting accuracy and robustness.
EXAMPLE III
Referring to fig. 4, fig. 4 is a schematic structural diagram of a multi-modal-based high-altitude safety belt detection device according to an embodiment of the present application. As shown in fig. 4, the apparatus of the embodiment of the present application includes:
a processor 301; and
the memory 302 is configured to store machine readable instructions, which when executed by the processor 301, cause the processor 301 to perform a multi-modality based high altitude seat belt detection method disclosed in an embodiment of the present application.
Compared with the prior art, the device provided by the embodiment of the application can integrate RGB images and thermal imaging images through multi-mode input, the detection precision of people is greatly improved through complementarity, the detection of people is only performed through a certain mode in the prior art, and then the detection is easy to perform false detection and missed detection only through input of a certain mode on the premise of complicated outdoor high-altitude environment. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, in the embodiment of the application, the key positions of the person are detected firstly, the key points of the key positions are obtained, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt has better detection accuracy and robustness.
Example four
In a fourth aspect of the present application, a storage medium stores a computer program, and the computer program is executed by a processor to perform the multi-modal-based high altitude seat belt detection method disclosed in the first embodiment of the present application.
Compared with the prior art, the detection precision of people is greatly improved through the complementarity of the RGB image and the thermal imaging image which can be integrated through multi-mode input in the embodiment of the application, and the detection of people is only performed through a certain mode in the prior art, so that the detection omission is easily subjected to false detection only through input of a certain mode on the premise of complicated outdoor high-altitude environment. On the other hand, in the prior art, on the premise that the posture of a person in high altitude is changed greatly, the accuracy of directly detecting the safety belt is not high, in the embodiment of the application, the key positions of the person are detected firstly, the key points of the key positions are obtained, then the safety belt is detected through the regional image of the person, whether the safety belt is worn or not is judged by detecting whether the safety belt exists in the category, and the safety wearing normalization is judged through the coincidence condition of the key points and the positions of the safety belt, so that the safety belt has better detection accuracy and robustness.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, a division of a unit is merely a division of one logic function, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
It should be noted that the functions, if implemented in the form of software functional modules and sold or used as independent products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A multi-mode-based high altitude safety belt detection method is characterized by comprising the following steps:
acquiring an aloft personnel working diagram acquired by a first imaging device and an aloft personnel thermal imaging diagram acquired by a second imaging device;
calibrating the high-altitude personnel thermal imaging image according to the RGB image to obtain a calibrated thermal imaging image;
extracting image features of the high-altitude personnel working diagram according to a preset image feature extraction network and taking the image features as first image features;
extracting image features of the calibrated thermal imaging image and taking the image features as second image features;
determining a human-shaped image of the high-altitude personnel according to the first image characteristic and the second image characteristic;
determining images of key parts of the high-altitude personnel according to the human-shaped images of the high-altitude personnel;
and detecting whether the high-altitude personnel wear the safety belt or not according to the image of the key part of the high-altitude personnel.
2. The method of claim 1, wherein determining the humanoid image of the high altitude person from the first image feature and the second image feature comprises:
based on an attention mechanism, performing feature fusion on the first image feature and the second image feature to obtain a fusion feature;
and determining the human-shaped image of the high-altitude personnel according to the fusion characteristics.
3. The method of claim 2, wherein the determining the humanoid image of the high altitude person from the fused features comprises:
determining the regional position of the high-altitude personnel according to the fusion characteristics;
and cutting out the image part of the regional position of the high-altitude personnel to be used as a human-shaped image of the high-altitude personnel.
4. The method of claim 1, wherein the predetermined image feature extraction network is one of VGG, google lenet, ResNet, rennext, DenseNet.
5. The method of claim 1, wherein after the detecting whether a high-altitude person wears a seat belt from the image of the high-altitude person critical portion, the method further comprises:
when the high-altitude personnel wear the safety belt, detecting whether the wearing mode of the safety belt worn by the high-altitude personnel meets the preset condition according to the image of the key part of the high-altitude personnel.
6. The method of claim 1, wherein the determining the image of the key part of the high altitude personnel from the humanoid image of the high altitude personnel comprises:
and identifying the human-shaped image of the high-altitude personnel according to a human body key part detection network in CPM, alpha Pose, OpenPose and Retinaface to obtain an image of the high-altitude personnel key part, wherein the image of the high-altitude personnel key part comprises images of a human body left shoulder, a human body right shoulder, a human body left crotch, a human body right crotch and a human body center.
7. The method of claim 1, wherein the detecting whether the high-altitude person wears a seat belt according to the image of the high-altitude person key part comprises;
the method comprises the steps of taking an image of a key part of a high-altitude person as an input of a safety belt detection model, so that the safety belt detection model identifies whether the high-altitude person wears a safety belt, wherein the safety belt detection model is one of a network mode of YOLO, SSD and fast-RCNN.
8. A multi-modality based high altitude seat belt detection apparatus, the apparatus comprising:
the acquiring module is used for acquiring an overhead personnel working diagram acquired by the first imaging device and an overhead personnel thermal imaging diagram acquired by the second imaging device;
the calibration module is used for calibrating the high-altitude personnel thermal imaging image according to the RGB image to obtain a calibrated thermal imaging image;
the first feature extraction module is used for extracting image features of the high-altitude personnel working diagram according to a preset image feature extraction network and taking the image features as first image features;
the second feature extraction module is used for extracting the image features of the calibrated thermal imaging image and taking the image features as second image features;
the first determining module is used for determining a human-shaped image of the high-altitude personnel according to the first image characteristic and the second image characteristic;
the second determining module is used for determining an image of a key part of the high-altitude personnel according to the human-shaped image of the high-altitude personnel;
and the detection module is used for detecting whether the high-altitude personnel wear the safety belt or not according to the image of the key part of the high-altitude personnel.
9. A multi-modality based high altitude seat belt detection apparatus, the apparatus comprising:
a processor; and
a memory configured to store machine readable instructions that, when executed by the processor, cause the processor to perform a multi-modality based high altitude seat belt detection method as claimed in any one of claims 1-7.
10. A storage medium, characterized in that the storage medium stores a computer program which is executed by a processor to perform a multi-modality based high altitude seat belt detection method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110498036.7A CN112990149A (en) | 2021-05-08 | 2021-05-08 | Multi-mode-based high-altitude safety belt detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110498036.7A CN112990149A (en) | 2021-05-08 | 2021-05-08 | Multi-mode-based high-altitude safety belt detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112990149A true CN112990149A (en) | 2021-06-18 |
Family
ID=76337224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110498036.7A Pending CN112990149A (en) | 2021-05-08 | 2021-05-08 | Multi-mode-based high-altitude safety belt detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990149A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822242A (en) * | 2021-11-19 | 2021-12-21 | 中化学交通建设集团有限公司 | Image recognition technology-based helmet wearing recognition method and device |
| CN117830956A (en) * | 2024-01-26 | 2024-04-05 | 南京关宁电子信息科技有限公司 | High-altitude live working safety protection method, system, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491781A (en) * | 2017-07-21 | 2017-12-19 | 国家电网公司 | A kind of crusing robot visible ray and infrared sensor data fusion method |
| CN108229440A (en) * | 2018-02-06 | 2018-06-29 | 北京奥开信息科技有限公司 | One kind is based on Multi-sensor Fusion indoor human body gesture recognition method |
| CN110889376A (en) * | 2019-11-28 | 2020-03-17 | 创新奇智(南京)科技有限公司 | Safety helmet wearing detection system and method based on deep learning |
| CN111382683A (en) * | 2020-03-02 | 2020-07-07 | 东南大学 | Target detection method based on feature fusion of color camera and infrared thermal imager |
| CN111986240A (en) * | 2020-09-01 | 2020-11-24 | 交通运输部水运科学研究所 | Drowning person detection method and system based on visible light and thermal imaging data fusion |
-
2021
- 2021-05-08 CN CN202110498036.7A patent/CN112990149A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491781A (en) * | 2017-07-21 | 2017-12-19 | 国家电网公司 | A kind of crusing robot visible ray and infrared sensor data fusion method |
| CN108229440A (en) * | 2018-02-06 | 2018-06-29 | 北京奥开信息科技有限公司 | One kind is based on Multi-sensor Fusion indoor human body gesture recognition method |
| CN110889376A (en) * | 2019-11-28 | 2020-03-17 | 创新奇智(南京)科技有限公司 | Safety helmet wearing detection system and method based on deep learning |
| CN111382683A (en) * | 2020-03-02 | 2020-07-07 | 东南大学 | Target detection method based on feature fusion of color camera and infrared thermal imager |
| CN111986240A (en) * | 2020-09-01 | 2020-11-24 | 交通运输部水运科学研究所 | Drowning person detection method and system based on visible light and thermal imaging data fusion |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822242A (en) * | 2021-11-19 | 2021-12-21 | 中化学交通建设集团有限公司 | Image recognition technology-based helmet wearing recognition method and device |
| CN113822242B (en) * | 2021-11-19 | 2022-02-25 | 中化学交通建设集团有限公司 | Image recognition technology-based helmet wearing recognition method and device |
| CN117830956A (en) * | 2024-01-26 | 2024-04-05 | 南京关宁电子信息科技有限公司 | High-altitude live working safety protection method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110312951B (en) | Personal protective equipment management system and method for integrated monitoring, warning and preventative safety event avoidance using optical articles | |
| CN107194361B (en) | Two-dimensional posture detection method and device | |
| CN109313659B (en) | Anomaly detection for revisions to web documents | |
| US11308620B1 (en) | Systems and methods for enhanced real-time image analysis with a dimensional convolution concept net | |
| US20140099623A1 (en) | Social graphs based on user bioresponse data | |
| CN112990149A (en) | Multi-mode-based high-altitude safety belt detection method, device, equipment and storage medium | |
| CN110443976A (en) | Safety prompt function method, apparatus and storage medium based on safety cap | |
| US20180053055A1 (en) | Integrating augmented reality content and thermal imagery | |
| Ahmed et al. | Gait recognition based on Kinect sensor | |
| KR20140104661A (en) | Method and apparatus for user interface using gaze interaction | |
| TW201222292A (en) | Facilitating people search in video surveillance | |
| US20200250224A1 (en) | Projecting visual aspects into a vector space | |
| US20240249555A1 (en) | Method for detecting human behavior, electronic device, and storage medium | |
| WO2021076628A1 (en) | Automatic pressure ulcer measurement | |
| CN111539253A (en) | Face detection method, device, terminal and storage medium | |
| Rasouli et al. | Smart Personal Protective Equipment (PPE) for construction safety: A literature review | |
| US11727710B2 (en) | Weakly supervised semantic parsing | |
| CN116681923A (en) | Automatic ophthalmic disease classification method and system based on artificial intelligence | |
| CN114220117A (en) | Wearing compliance detection method and device and computer readable storage medium | |
| AU2021101372A4 (en) | Human Body Shape Prediction System Based on Human Body Size Database | |
| CN103870814A (en) | Non-contact real-time eye movement identification method based on intelligent camera | |
| US20240058071A1 (en) | Left atrial appendage closure pre-procedure system and methods | |
| US11896375B2 (en) | Assessment of facial paralysis and gaze deviation | |
| CN113314230A (en) | Intelligent epidemic prevention method, device, equipment and storage medium based on big data | |
| CN107195163A (en) | Alarm method and device and wearable device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210618 |