CN116863232A - Steel structure safe hoisting method and device based on gesture image recognition monitoring - Google Patents
Steel structure safe hoisting method and device based on gesture image recognition monitoring Download PDFInfo
- Publication number
- CN116863232A CN116863232A CN202310857124.0A CN202310857124A CN116863232A CN 116863232 A CN116863232 A CN 116863232A CN 202310857124 A CN202310857124 A CN 202310857124A CN 116863232 A CN116863232 A CN 116863232A
- Authority
- CN
- China
- Prior art keywords
- hoisting
- safe
- gesture
- lifting
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000012544 monitoring process Methods 0.000 title claims abstract description 41
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 40
- 239000010959 steel Substances 0.000 title claims abstract description 40
- 238000001514 detection method Methods 0.000 claims abstract description 71
- 238000012549 training Methods 0.000 claims abstract description 24
- 238000010276 construction Methods 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims description 30
- 230000036544 posture Effects 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 22
- 230000006399 behavior Effects 0.000 claims description 19
- 239000000725 suspension Substances 0.000 claims description 19
- 238000012216 screening Methods 0.000 claims description 13
- 238000013145 classification model Methods 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000006073 displacement reaction Methods 0.000 claims description 5
- 238000012937 correction Methods 0.000 claims description 4
- 238000013461 design Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000011900 installation process Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000009435 building construction Methods 0.000 description 1
- 239000010720 hydraulic oil Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a steel structure safe hoisting method and a device based on gesture image recognition monitoring, wherein the method comprises the following steps: s10, setting key points, and taking a plurality of connection points of hoisting equipment as key points; s20, designing a lifting device target detection algorithm, and training by using a key point data set of the mobile enhanced lifting device; s30, designing a lifting device gesture detection algorithm for identifying position information of key points of the lifting device in a target frame of the lifting device, and giving gesture detection to extract position coordinates and confidence coefficients of a plurality of key points in the lifting device; s40, judging whether the hoisting equipment exceeds a safe hoisting range or not; s50, judging based on a safe hoisting detection method of the projection length between vertical relative key points: and (3) comparing the coordinates of each key point with the coordinates of the training model, and distinguishing the safe hoisting behavior from the unsafe hoisting behavior. The method improves the accuracy of full hoisting and unsafe hoisting judgment, and is beneficial to safe operation of hoisting construction.
Description
Technical Field
The application relates to the technical field of building construction, and particularly provides steel structure hoisting equipment and method based on gesture image recognition monitoring.
Background
The steel structure of large-scale building divide into frame construction and large-span truss structure, and main part component contains steel column, girder steel, roofing truss, and large-scale building's span often is more than 100 meters, need carry out the truss construction of sliding after accomplishing frame construction integral hoisting.
The existing hoisting and sliding construction mostly adopts a crawler hoisting machine. The crawler crane adopts the independent driving of chassis running and boom swinging and telescoping, the independent power source is not arranged on the boom, the power is obtained from the main machine master pump through the hydraulic oil pipe, the chassis and the boom driving can be linked and can be single-acting, and the crawler crane can be freely switched under different operation modes. In the construction process, the crawler-type hoisting machine can be installed while walking, and the efficiency is high. However, in the existing steel structure installation process, the state of the hoisting equipment is judged according to experience by a driver of the hoisting equipment, so that the state of the hoisting equipment is slightly careless, and once the postures of the suspension arm and the hoisting equipment are out of a reasonable range, the overturning accidents easily occur.
Disclosure of Invention
Aiming at the problems that in the existing steel structure installation process, a driver judges the state of the hoisting equipment according to experience, and the boom and the hoisting equipment easily exceed a reasonable range to easily cause overturning accidents, the application provides the steel structure hoisting equipment and the method based on gesture image recognition monitoring, so as to realize real-time monitoring on the gesture of the hoisting equipment and avoid the problems that the overturning problems are not found in time and caused because the gesture of the hoisting equipment exceeds the reasonable range.
In order to achieve the above object, in a first aspect, a method for safely hoisting a steel structure based on gesture image recognition and monitoring is provided, which comprises the following steps:
s10, design key points: taking a chassis center point of the hoisting equipment, a main arm and chassis hinge point of the hoisting equipment, a main arm and auxiliary arm hinge point and an auxiliary arm front end point as key points;
s20, designing a lifting device target detection algorithm training model: firstly, training a new lifting device target detection single classification model, and performing movement enhancement training by using a key point data set of the lifting device to obtain a boundary coordinate set of safe lifting and dangerous lifting;
the target detection algorithm is more stable in a scene with changeable positions and postures, and has stronger robustness, and the single classification model for the target detection of the lifting equipment only detects the chassis and the suspension arm of the lifting equipment, so that the size of the single classification model for the target detection of the lifting equipment is reduced;
s30, designing a lifting device posture detection algorithm to obtain key point information: the gesture detection algorithm is used for identifying chassis position information and suspension arm gesture information of the hoisting equipment in a target frame of the hoisting equipment, and giving coordinates and confidence of each key point; obtaining key point information of the hoisting equipment through a hoisting equipment target detection algorithm and a gesture detection algorithm, wherein the key point information is used for judging whether the hoisting equipment exceeds a safety detection condition in a threshold value or not in a displacement process;
s40, judging whether the hoisting equipment exceeds a safe hoisting range: in the hoisting construction process, coordinate information of each key point is obtained in real time, and the following logic judgment is carried out: 1) Judging that the coordinate of the center of the chassis of the lifting equipment is larger than a threshold value relative to the target coordinate after the target detection, and 2) carrying out subsequent procedures if the coordinate of each key point of the lifting equipment is larger than the threshold value relative to the target coordinate after the gesture detection and the condition is met;
s50, judging whether hoisting is safe or not based on a safe hoisting detection method of each key point: after the inclination occurs, the coordinates of each key point of the hoisting equipment are changed, and the posture of the suspension arm is obtained through the coordinates of the key points, so that a safe hoisting detection judging method based on the coordinates of the key points is designed, and safe hoisting and non-safe hoisting behaviors are distinguished by judging whether the coordinates of each key point are in the range of the safe hoisting state or not.
Optionally, the method for acquiring the key point position of the hoisting equipment comprises the steps of acquiring and processing an image in the hoisting process of the hoisting equipment, acquiring a real-time image of the hoisting equipment in real time by an image acquisition unit, and processing the acquired image by an image processing unit to acquire the contour information of the hoisting equipment.
Optionally, the method for obtaining the positions of the key points of the hoisting equipment includes performing profile information lattice processing of the hoisting equipment, and then extracting coordinate information of the key points.
Optionally, the step S40 further includes step S401 hoisting load screening logic: based on the fact that the safe hoisting problem cannot occur when the hoisting load is far smaller than the rated load, the hoisting load is compared with the rated load, and subsequent detection is not conducted when the hoisting load is far smaller than the rated load.
Optionally, the step S40 further includes step S402 special location screening logic: based on the safe lifting behavior, the lifting equipment targets at the positions can not be filtered out at the special positions such as the lifting initial point, the lifting end point and the like, and the situations of false alarm missing report and the like in the safe lifting detection are reduced.
Optionally, the step S50 further includes step S501 of multi-frame joint security movement judgment: in order to avoid that part of images are misjudged to exceed the safe lifting behavior, misjudgment flash screening is carried out, namely more than half of the images in the continuous multi-frame images are judged to exceed the safe lifting behavior by meeting the requirement, the images are finally judged to exceed the safe lifting behavior, and the images after the frames exceeding the safe lifting frame number are generated are finally judged to exceed the safe lifting behavior.
Optionally, S50 further includes S502 determining, based on the main arm inclination angle, safe hoisting, and selecting different determining logics according to the main arm inclination angle of the hoisting device: when the inclination angle of the main arm is large, coordinates of each key point at two ends of the main arm are selected to meet a threshold value, and when the inclination angle of the main arm is small, the auxiliary arm is selected to meet the threshold value.
Optionally, when the posture of the hoisting equipment is judged to be out of the safety range in step S50, performing posture correction in step S60, and controlling posture change of the hoisting equipment until coordinates of each key point are smaller than a threshold value relative to corresponding target coordinates.
In order to achieve the above purpose, a second aspect provides a steel structure hoisting device based on gesture image recognition and monitoring, the hoisting device comprises a chassis and a suspension arm, and further comprises a gesture image recognition and monitoring system;
the gesture image recognition monitoring system comprises an image acquisition unit, an image processing unit, a model training unit and a comparison judging unit;
the image acquisition unit is used for acquiring real-time images of the hoisting equipment;
the image processing unit is used for processing the outline of the real-time image of the hoisting equipment, amplifying signals, dot matrix and acquiring dot matrix outline, and the key point extractor is used for extracting coordinate information of key points;
the model training unit is used for training a new lifting device target detection single classification model, and training is carried out by using a key point data set of the lifting device with enhanced movement, so that a target detection algorithm is more stable in a scene with changeable angle and posture;
the comparison judging unit is used for comparing and identifying the coordinates of the key points of the chassis and the suspension arm with the coordinates of the key points of the trained model respectively so as to determine whether the position of the chassis exceeds a set range and determine whether the posture of the suspension arm exceeds the set range.
Optionally, the steel structure hoisting equipment based on gesture image recognition monitoring further comprises a gesture control system, wherein the gesture control system is connected with the output end of the image processing unit and is used for controlling the position of the chassis and the gesture of the suspension arm.
Compared with the prior art, the application has the following advantages:
1. through the obtained key point information of the hoisting equipment, a hoisting equipment target detection algorithm and a hoisting equipment gesture detection algorithm are designed, the hoisting equipment movement is classified in a follow-up safe hoisting detection algorithm, and the hoisting equipment movement detection algorithm is used for judging whether the hoisting equipment exceeds safe detection conditions in a threshold value or not in a displacement process, and the accuracy and efficiency of safe hoisting and unsafe hoisting judgment are improved, so that safe operation of hoisting construction is facilitated.
2. Based on the safety hoisting process characteristics, a plurality of algorithm screening logics such as a hoisting load screening method, a combined target detection algorithm, multi-frame combined safety movement judgment and the like are designed, and the false alarm missing report conditions caused by multiple hoisting loads, multiple positions and the like in the safety hoisting detection process are effectively discharged by combining the actual condition of the hoisting process, so that the method has higher robustness compared with the method for predicting actions by only relying on a neural network, the operation efficiency is improved, and the requirement of a system on the calculation power is reduced.
Drawings
The application will be described in further detail with reference to the accompanying drawings and detailed description. The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application.
Fig. 1 is a flow chart of a preferred embodiment of a steel structure hoisting method based on gesture image recognition monitoring.
Fig. 2 is a flow chart of another preferred embodiment of the steel structure hoisting method based on gesture image recognition monitoring provided by the application.
Fig. 3 is a system configuration diagram of a preferred embodiment of a steel structure hoisting device based on gesture image recognition monitoring provided by the application.
Fig. 4 is a schematic structural view of a preferred embodiment of the hoisting device provided by the application.
In the figure, a 1-image acquisition unit, a 11-first camera, a 12-second camera and a 13-third camera; 2-image processing unit, 3-model training unit; 4-comparison judging unit, 5-attitude control system, 61-chassis, 62-main arm and 63-auxiliary arm.
Detailed Description
The steel structure of large-scale building divide into frame construction and large-span truss structure, and main part component contains steel column, girder steel, roofing truss, and large-scale building's span often is more than 100 meters, need carry out the truss construction of sliding after accomplishing frame construction integral hoisting. The existing hoisting and sliding construction mostly adopts a crawler hoisting machine. In the construction process, the crawler-type hoisting machine can be installed while walking, and the efficiency is high. However, in the existing steel structure installation process, the state of the hoisting equipment is judged according to experience by a driver of the hoisting equipment, so that the state of the hoisting equipment is slightly careless, and once the postures of the suspension arm and the hoisting equipment are out of a reasonable range, the overturning accidents easily occur.
Therefore, the application provides a steel structure hoisting method and equipment based on gesture image recognition monitoring, and a preferred embodiment of the application is described in detail below with reference to the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the application, its application, or uses.
Fig. 1 is a flow chart of a preferred embodiment of a steel structure hoisting method based on gesture image recognition monitoring.
As shown in fig. 1, in an embodiment, the method for safely hoisting a steel structure based on gesture image recognition monitoring provided by the application includes:
s10, design key points: taking a chassis center point of the hoisting equipment, a main arm and chassis hinge point of the hoisting equipment, a main arm and auxiliary arm hinge point and an auxiliary arm front end point as key points;
s20, designing a lifting device target detection algorithm training model: firstly, training a new lifting device target detection single classification model, and performing movement enhancement training by using a key point data set of the lifting device to obtain a boundary coordinate set of safe lifting and dangerous lifting;
the target detection algorithm is more stable in a scene with changeable positions and postures, and has stronger robustness, and the single classification model for the target detection of the lifting equipment only detects the chassis and the suspension arm of the lifting equipment, so that the size of the single classification model for the target detection of the lifting equipment is reduced;
s30, designing a lifting device posture detection algorithm to obtain key point information: the gesture detection algorithm is used for identifying chassis position information and suspension arm gesture information of the hoisting equipment in a target frame of the hoisting equipment, and giving coordinates and confidence of each key point; obtaining key point information of the hoisting equipment through a hoisting equipment target detection algorithm and a gesture detection algorithm, wherein the key point information is used for judging whether the hoisting equipment exceeds a safety detection condition in a threshold value or not in a displacement process;
s40, judging whether the hoisting equipment exceeds a safe hoisting range: in the hoisting construction process, coordinate information of each key point is obtained in real time, and the following logic judgment is carried out: 1) Judging that the coordinate of the center of the chassis of the lifting equipment is larger than a threshold value relative to the target coordinate after the target detection, and 2) carrying out subsequent procedures if the coordinate of each key point of the lifting equipment is larger than the threshold value relative to the target coordinate after the gesture detection and the condition is met;
s50, judging whether hoisting is safe or not based on a safe hoisting detection method of each key point: after the inclination occurs, the coordinates of each key point of the hoisting equipment are changed, and the posture of the suspension arm is obtained through the coordinates of the key points, so that a safe hoisting detection judging method based on the coordinates of the key points is designed, and safe hoisting and non-safe hoisting behaviors are distinguished by judging whether the coordinates of each key point are in the range of the safe hoisting state or not.
Compared with the prior art, the steel structure safe hoisting method based on gesture image recognition monitoring provided by the embodiment designs a hoisting equipment target detection algorithm and a hoisting equipment gesture detection algorithm through the obtained hoisting equipment key point information, is used for classifying the hoisting equipment movement in the follow-up safe hoisting detection algorithm, is used for judging whether the hoisting equipment exceeds the safe detection condition in a threshold value in the displacement process or not, improves the accuracy and efficiency of safe hoisting and unsafe hoisting judgment, and is beneficial to safe operation of hoisting construction.
Aiming at the structure and the size of the hoisting equipment are larger, if all the images are processed, larger calculation is needed, and the processing efficiency is influenced, in one embodiment, the method for acquiring the positions of the key points of the hoisting equipment comprises the steps of acquiring and processing the images in the hoisting process of the hoisting equipment, acquiring the real-time images of the hoisting equipment in real time by an image acquisition unit, processing the acquired images by an image processing unit, and acquiring the contour information of the hoisting equipment, wherein the method for acquiring the positions of the key points of the hoisting equipment preferably comprises the steps of firstly carrying out the contour information lattice processing of the hoisting equipment, and then extracting the coordinate information of the key points. According to the embodiment, the outline information is obtained, dot matrix processing is carried out, and only the coordinate information of the key points is extracted and used, so that the data processing amount is effectively reduced, the requirement on the computing power of the system is reduced, and the processing efficiency is improved.
When the hoisting load is much smaller than the rated load, for example 1/5 of the rated load, or even lower, the hoisting device safety profile is high and no calculation is necessary, so in an embodiment, the S40 further comprises S401 hoisting load screening logic: based on the fact that the safe hoisting problem cannot occur when the hoisting load is far smaller than the rated load, the hoisting load is compared with the rated load, and subsequent detection is not conducted when the hoisting load is far smaller than the rated load. Therefore, the identification monitoring range is reduced, and the processing efficiency is improved.
At the initial point of hoisting, the hoisting end point, the hoisting load is supported by the building structure without regard to hoisting safety, for which reason, in some embodiments, the S40 further comprises S402 special position screening logic: based on the safe lifting behavior, the lifting equipment targets at the positions can not be filtered out at the special positions such as the lifting initial point and the lifting end point, the occurrence of the situations such as false alarm missing report and the like in the safe lifting detection is reduced, and meanwhile, the processing efficiency is effectively improved.
In the implementation process, the obtained partial single-frame image has distortion to cause erroneous judgment, so in an embodiment, the step S50 further includes step S501 of performing a multi-frame joint safe movement judgment: in order to avoid that part of images are misjudged to exceed the safe lifting behavior, misjudgment flash screening is carried out, namely more than half of the images in the continuous multi-frame images are judged to exceed the safe lifting behavior, and finally the images are judged to exceed the safe lifting behavior, and in a plurality of frames after the number of frames exceeding the safe lifting behavior occurs, the images are finally judged to exceed the safe lifting behavior, so that the judgment accuracy is improved.
According to the embodiments, based on the characteristics of the safe hoisting process, a plurality of algorithm screening logics such as a hoisting load screening method, a combined target detection algorithm, multi-frame combined safe movement judgment and the like are designed, and the false alarm missing report conditions caused by multiple hoisting loads, multiple positions and the like in the safe hoisting detection process are effectively discharged by combining the actual conditions of the hoisting process, so that the method has higher robustness compared with the method for predicting actions by only relying on a neural network, the operation efficiency is improved, and the requirement of a system on the calculation power is reduced.
Fig. 2 is a flow chart of another preferred embodiment of the steel structure hoisting method based on gesture image recognition monitoring provided by the application.
As shown in fig. 2, in an embodiment, S50 further includes S502 performing a safe hoisting judgment based on the main arm inclination angle, and selecting different judgment logics according to the main arm inclination angle of the hoisting device: when the inclination angle of the main arm is large, the coordinates of each key point at the two ends of the main arm 62 are selected to meet the threshold value, when the inclination angle of the main arm is small, the auxiliary arm 63 is selected to meet the threshold value, and the judgment of safe hoisting is performed through different judgment logics in a mode of assisting the division of the inclination angle of the arm, so that the judgment process is simplified, and the judgment efficiency is improved.
In an embodiment, when the posture of the hoisting equipment is judged to be beyond the safety range through the step S50, the posture correction of the step S60 is performed, and the posture change of the hoisting equipment is controlled until the coordinates of each key point are smaller than a threshold value relative to the corresponding target coordinates, so that the correction of the chassis and the suspension arm of the safety hoisting is automatically completed, and the automatic implementation of the safety hoisting is facilitated.
Fig. 3 is a system configuration diagram of a preferred embodiment of a steel structure hoisting device based on gesture image recognition monitoring provided by the application. Fig. 4 is a schematic structural view of a preferred embodiment of the hoisting device provided by the application.
As shown in fig. 3 and 4, in one embodiment of the present application, a steel structure hoisting device based on gesture image recognition monitoring is provided, where the hoisting device includes a chassis 61 and a boom, and further includes a gesture image recognition monitoring system; the gesture image recognition monitoring system comprises an image acquisition unit 1, an image processing unit 2, a model training unit 3 and a comparison judging unit 4; the image acquisition unit 1 is configured to acquire a real-time image of the hoisting device; the image processing unit 2 is configured to process the contour of the real-time image of the hoisting device, amplify the signal, dot matrix and obtain the dot matrix contour, and the key point extractor is used for extracting the coordinate information of the key points; the model training unit 3 is used for training a new lifting device target detection single classification model, and training by using a key point data set of the lifting device with enhanced movement so as to enable a target detection algorithm to be more stable in a scene with changeable angle and posture; the comparison and judgment unit 4 is configured to compare and identify the coordinates of the key points of the chassis and the boom with the coordinates of the key points of the trained model, respectively, so as to determine whether the chassis position exceeds a set range and determine whether the boom gesture exceeds the set range. Fig. 2 is a schematic diagram of the structure of the image acquisition unit. As shown in fig. 2, in some preferred embodiments, the cameras include a first camera 11 located at the top of the boom, a second camera 12 located at the middle and outer side of the boom, and a third camera 13 disposed on the chassis, where the first camera 11 and the second camera 12 are used to obtain real-time images of the boom pose, and the boom pose includes the rotation angle and elevation of the boom. The third camera 13 is used to acquire a real-time image of the chassis pose, which includes the position of the chassis. Through installing first camera, second camera and third camera respectively at davit top, lateral part and chassis, carry out effective control to lifting device's gesture, hoist and mount progress, slip and march etc. and then improve monitoring scope and monitoring accuracy. Preferably, the camera can adopt a video camera, a visual detector and an ultrasonic detector.
In an embodiment, the steel structure hoisting device based on gesture image recognition monitoring further comprises a gesture control system 4, wherein the gesture control system 4 is connected with the output end of the image processing unit 2, and the gesture control system 4 is used for controlling the position of the chassis and the gesture of the boom.
According to the steel structure hoisting equipment and the method based on gesture image recognition monitoring, provided by the embodiment, the gesture of the hoisting equipment can be monitored in real time, the problem that the hoisting equipment is not found in time and is overturned due to the fact that the gesture exceeds a reasonable range is avoided, the accuracy and the efficiency of safe hoisting and unsafe hoisting judgment are improved, and safe operation of hoisting construction is facilitated.
The above embodiments are only for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to examples, it should be understood by those skilled in the art that modifications and equivalents may be made thereto without departing from the spirit and scope of the technical solution of the present application, which is intended to be covered by the scope of the claims of the present application.
Claims (10)
1. The steel structure safe hoisting method based on gesture image recognition monitoring is characterized by comprising the following steps of:
s10, design key points: taking a chassis center point of the hoisting equipment, a main arm and chassis hinge point of the hoisting equipment, a main arm and auxiliary arm hinge point and an auxiliary arm front end point as key points;
s20, designing a lifting device target detection algorithm training model: firstly, training a new lifting device target detection single classification model, and performing movement enhancement training by using a key point data set of the lifting device to obtain a boundary coordinate set of safe lifting and dangerous lifting;
the target detection algorithm is more stable in a scene with changeable positions and postures, and has stronger robustness, and the single classification model for the target detection of the lifting equipment only detects the chassis and the suspension arm of the lifting equipment, so that the size of the single classification model for the target detection of the lifting equipment is reduced;
s30, designing a lifting device posture detection algorithm to obtain key point information: the gesture detection algorithm is used for identifying chassis position information and suspension arm gesture information of the hoisting equipment in a target frame of the hoisting equipment, and giving coordinates and confidence of each key point; obtaining key point information of the hoisting equipment through a hoisting equipment target detection algorithm and a gesture detection algorithm, wherein the key point information is used for judging whether the hoisting equipment exceeds a safety detection condition in a threshold value or not in a displacement process;
s40, judging whether the hoisting equipment exceeds a safe hoisting range: in the hoisting construction process, coordinate information of each key point is obtained in real time, and the following logic judgment is carried out: 1) Judging that the coordinate of the center of the chassis of the lifting equipment is larger than a threshold value relative to the target coordinate after the target detection, and 2) carrying out subsequent procedures if the coordinate of each key point of the lifting equipment is larger than the threshold value relative to the target coordinate after the gesture detection and the condition is met;
s50, judging whether hoisting is safe or not based on a safe hoisting detection method of each key point: after the inclination occurs, the coordinates of each key point of the hoisting equipment are changed, and the posture of the suspension arm is obtained through the coordinates of the key points, so that a safe hoisting detection judging method based on the coordinates of the key points is designed, and safe hoisting and non-safe hoisting behaviors are distinguished by judging whether the coordinates of each key point are in the range of the safe hoisting state or not.
2. The steel structure safe hoisting method based on gesture image recognition monitoring according to claim 1, wherein the method for acquiring the key point positions of the hoisting equipment comprises the steps of acquiring and processing images in the hoisting process of the hoisting equipment, acquiring real-time images of the hoisting equipment in real time by an image acquisition unit, and processing the acquired images by an image processing unit to acquire profile information of the hoisting equipment.
3. The steel structure safe hoisting method based on gesture image recognition monitoring according to claim 2, wherein the method for acquiring the positions of the key points of the hoisting equipment comprises the steps of firstly performing profile information lattice processing of the hoisting equipment and then extracting coordinate information of the key points.
4. The method for safely hoisting a steel structure based on gesture image recognition monitoring of claim 1, wherein S40 further comprises S401 hoisting load screening logic: based on the fact that the safe hoisting problem cannot occur when the hoisting load is far smaller than the rated load, the hoisting load is compared with the rated load, and subsequent detection is not conducted when the hoisting load is far smaller than the rated load.
5. The method for safely hoisting the steel structure based on the gesture image recognition monitoring of claim 1, wherein the step S40 further comprises step S402 of special position screening logic: based on the safe lifting behavior, the lifting equipment targets at the positions can not be filtered out at the special positions such as the lifting initial point, the lifting end point and the like, and the situations of false alarm missing report and the like in the safe lifting detection are reduced.
6. The steel structure safe hoisting method based on gesture image recognition monitoring of claim 1, wherein the step S50 further comprises step S501 of multi-frame joint safe movement judgment: in order to avoid that part of images are misjudged to exceed the safe lifting behavior, misjudgment flash screening is carried out, namely more than half of the images in the continuous multi-frame images are judged to exceed the safe lifting behavior by meeting the requirement, the images are finally judged to exceed the safe lifting behavior, and the images after the frames exceeding the safe lifting frame number are generated are finally judged to exceed the safe lifting behavior.
7. The safe hoisting method of the steel structure based on the gesture image recognition monitoring according to claim 1, wherein the step S50 further comprises the step S502 of performing safe hoisting judgment based on the inclination angle of the main arm (62), and selecting different judgment logics according to the inclination angle of the main arm of the hoisting device: when the inclination angle of the main arm is large, coordinates of each key point at two ends of the main arm (62) are selected to meet a threshold value, and when the inclination angle of the main arm is small, the auxiliary arm (63) is selected to meet the threshold value.
8. The steel structure safe hoisting method based on gesture image recognition monitoring according to claim 1, wherein when the gesture of the hoisting equipment is judged to be out of a safe range through the step S50, the gesture correction is performed in the step S60, and the gesture change of the hoisting equipment is controlled until the coordinates of each key point are smaller than a threshold value relative to the corresponding target coordinates.
9. Steel construction lifting device based on gesture image recognition monitoring, lifting device includes chassis and davit, its characterized in that:
the system also comprises a gesture image recognition monitoring system;
the gesture image recognition monitoring system comprises an image acquisition unit (1), an image processing unit (2), a model training unit (3) and a comparison judging unit (4);
the image acquisition unit (1) is used for acquiring real-time images of the hoisting equipment;
the image processing unit (2) is used for processing the outline of the real-time image of the hoisting equipment, amplifying signals, dot matrix and acquiring dot matrix outline, and the key point extractor is used for extracting coordinate information of key points;
the model training unit (3) is used for training a new lifting device target detection single classification model, and training is carried out by using a key point data set of the lifting device with enhanced movement, so that a target detection algorithm is more stable in a scene with changeable angle and posture;
the comparison judging unit (4) is used for comparing and identifying the coordinates of the key points of the chassis and the suspension arm with the coordinates of the key points of the trained model respectively so as to determine whether the position of the chassis exceeds a set range and determine whether the posture of the suspension arm exceeds the set range.
10. The steel structure hoisting equipment based on gesture image recognition monitoring as claimed in claim 7, wherein: the steel structure hoisting equipment based on gesture image recognition monitoring further comprises a gesture control system (5), wherein the gesture control system (5) is connected with the output end of the image processing unit (2), and the gesture control system (5) is used for controlling the position of the chassis and the gesture of the suspension arm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310857124.0A CN116863232A (en) | 2023-07-12 | 2023-07-12 | Steel structure safe hoisting method and device based on gesture image recognition monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310857124.0A CN116863232A (en) | 2023-07-12 | 2023-07-12 | Steel structure safe hoisting method and device based on gesture image recognition monitoring |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116863232A true CN116863232A (en) | 2023-10-10 |
Family
ID=88235533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310857124.0A Pending CN116863232A (en) | 2023-07-12 | 2023-07-12 | Steel structure safe hoisting method and device based on gesture image recognition monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116863232A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117776065A (en) * | 2024-02-27 | 2024-03-29 | 河北圣丰自动化科技有限公司 | Construction lifting platform safety state monitoring method and system |
-
2023
- 2023-07-12 CN CN202310857124.0A patent/CN116863232A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117776065A (en) * | 2024-02-27 | 2024-03-29 | 河北圣丰自动化科技有限公司 | Construction lifting platform safety state monitoring method and system |
| CN117776065B (en) * | 2024-02-27 | 2024-04-30 | 河北圣丰自动化科技有限公司 | Construction lifting platform safety state monitoring method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101748121B1 (en) | System and method for detecting image in real-time based on object recognition | |
| CN116863232A (en) | Steel structure safe hoisting method and device based on gesture image recognition monitoring | |
| JP4884331B2 (en) | Moving object tracking device | |
| CN107784652B (en) | Tower rapid detection method based on unmanned aerial vehicle image | |
| WO2018164273A1 (en) | Human detection system for work vehicle, and work vehicle equipped with same | |
| CN111506011A (en) | Construction safety monitoring method and device | |
| CN115082849B (en) | Intelligent template support safety monitoring method based on deep learning | |
| CN103663188A (en) | Luffing jib tower crane as well as operation control method, device and system thereof | |
| CN111062373A (en) | Hoisting process danger identification method and system based on deep learning | |
| CN107200274A (en) | A kind of anti-container truck based on machine vision is lifted method | |
| CN104866842A (en) | Human body target intrusion detection method | |
| CN112595730A (en) | Cable breakage identification method and device and computer equipment | |
| CN111879308A (en) | Intelligent tower crane safety monitoring system based on pose perception technology and implementation method | |
| CN114326625A (en) | System and method for monitoring potential safety risks in power grid infrastructure construction | |
| CN115082850A (en) | Template support safety risk identification method based on computer vision | |
| CN108805890A (en) | A kind of arc hammer measurement method based on power transmission line image characteristic point | |
| JP2005173649A (en) | Template matching processing method and processor for image processing | |
| CN113014894B (en) | Petroleum underground operation safety staring control method based on artificial intelligence | |
| CN113194284B (en) | Intelligent monitoring system and method for tower crane | |
| KR101754137B1 (en) | Methode for monitoring structure and structure monitoring system | |
| CN113569730B (en) | Protection state detection method and device and electronic equipment | |
| CN114511551A (en) | Ground damage identification system based on machine vision | |
| CN212516078U (en) | Intelligent monitoring and alarming system for loss of electric shovel teeth | |
| CN112634232A (en) | Detection method, device and system for violation border crossing of tunneller personnel | |
| CN117474321B (en) | BIM model-based construction site risk intelligent identification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |