CN110745704B - Tower crane early warning method and device - Google Patents
Tower crane early warning method and device Download PDFInfo
- Publication number
- CN110745704B CN110745704B CN201911326235.9A CN201911326235A CN110745704B CN 110745704 B CN110745704 B CN 110745704B CN 201911326235 A CN201911326235 A CN 201911326235A CN 110745704 B CN110745704 B CN 110745704B
- Authority
- CN
- China
- Prior art keywords
- image
- tower crane
- data
- image data
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66C—CRANES; LOAD-ENGAGING ELEMENTS OR DEVICES FOR CRANES, CAPSTANS, WINCHES, OR TACKLES
- B66C15/00—Safety gear
- B66C15/06—Arrangements or use of warning devices
- B66C15/065—Arrangements or use of warning devices electrical
Abstract
The invention provides a tower crane early warning method and a tower crane early warning device, wherein the method comprises the following steps: monitoring gravity sensor data of a tower crane, and acquiring image data to be identified of a lifting hook on the tower crane when the gravity sensor data changes and the change duration time exceeds preset time; analyzing the image data to be recognized to obtain characteristic data in the image data to be recognized; the characteristic data comprises hook characteristic data and hanging object characteristic data on a hook; determining a dangerous warning area of the tower crane based on the characteristic data; the positions of personnel around the tower crane are tracked so as to carry out safety alarm when the personnel enter a danger warning area. Based on the method provided by the invention, the personnel identification and the hanging object identification are independently separated, so that the mutual interference is reduced, the resource waste is avoided, and the personnel passing through the tower crane can be reminded in time.
Description
Technical Field
The invention relates to the technical field of cranes, in particular to a tower crane early warning method and device.
Background
At present, the tower crane is widely applied to the construction industry, so the potential safety hazards in all aspects of the tower crane are gradually attracted to all parties. The tower crane has the dangerous characteristics of high-altitude operation and heavy hanging weight, and ground workers are easy to hang at high altitude and are not easy to perceive dangerous hidden dangers.
In the existing tower crane operation systems, some of the tower crane operation systems are used for identifying personnel and lifting hooks, the early warning effect is achieved by confirming the distance between the personnel and the lifting hooks, and the problem that the magnitude of the hanging danger of lifting hook weights is improved in a working state is ignored; some devices acquire the height data of the lifting hook through a height sensor so as to confirm operation danger area alarm, ignore different danger areas caused by different hanging object types and sizes, and change the alarm area required to respond; in other aspects, partial technologies pay more attention to detect the self safety of the tower crane through the sensor, the safety monitoring of the working state of the tower crane is avoided, a similar projection warning area is adopted in some parts, the sensor personnel detection and the like confirm the dangerous range in an empirical mode, and the error early warning condition is easy to occur. Therefore, how to remind people in the ground operation area to improve vigilance is an urgent problem to be solved.
Disclosure of Invention
In view of the above, the present invention provides a tower crane early warning method and apparatus that overcomes or at least partially solves the above-mentioned problems.
According to one aspect of the invention, a tower crane early warning method is provided, which is characterized by comprising the following steps:
monitoring gravity sensor data of a tower crane, and acquiring image data to be identified of a lifting hook on the tower crane when the gravity sensor data changes and the change duration time exceeds preset time;
analyzing the image data to be identified to obtain feature data in the image data to be identified; wherein the characteristic data comprises hook characteristic data and hanging object characteristic data on the hook;
determining a danger warning area of the tower crane based on the characteristic data;
and tracking the positions of personnel around the tower crane so that the personnel can enter the danger warning area to perform safety warning.
Optionally, the analyzing the image data to be recognized to obtain feature data in the image data to be recognized includes:
extracting a first image size of the hook and a second image size of the hanging object;
and comparing the first image size of the lifting hook and the second image size of the hanging object with data in a preset image retrieval database to determine the category of the hanging object.
Optionally, before analyzing the image data to be recognized to obtain the feature data in the image data to be recognized, the method further includes:
collecting a plurality of image data of various types of hoisted objects on the tower crane at different time periods and different heights;
analyzing each image data in the image retrieval database to obtain a region of interest in the image data;
extracting image feature data included in each piece of the image data;
establishing an image retrieval database based on the image characteristic data;
and performing deep learning on the image characteristic data and the corresponding relation among the sizes of the lifting hooks, the types and the sizes of the hanging objects through a deep learning model.
Optionally, determining the danger warning area of the tower crane based on the characteristic data includes:
acquiring the first image size, the second image size and a first actual size of the hook;
calculating a proportional relation between the first image size and the first actual size, and determining a second actual size of the suspended object based on the second image size and the proportional relation;
and determining the danger warning area of the tower crane based on the center coordinate of the hanging object and the second actual size of the hanging object.
Optionally, the danger warning area includes a danger area and an early warning area;
based on hang the central coordinate of thing and hang the second actual size of thing and confirm the danger warning area of tower crane includes:
determining a first circular area with the center coordinate of the hoisted object as the circle center and half of the second actual size as a first radius as a dangerous area of the tower crane;
determining an annular area except the first circular area in a second circular area with the center coordinate of the hoisted object as a circle center and a second radius as an early warning area of the tower crane;
the second radius is greater than the first radius;
the second physical dimension of the sling is the length of the sling.
Optionally, calculating the second radius according to the following manner;
acquiring height data of the hoisted object, multiplying the height by a preset coefficient, and adding the height data and the first radius to obtain a second radius;
wherein the preset coefficient is more than 0 and less than 1.
Optionally, track personnel's around the tower crane position to when personnel get into carry out safety alarm when the danger warning area includes:
acquiring environmental image data of the tower crane during working in real time, and acquiring real-time coordinates of at least one person included in the environmental image data based on a pre-constructed coordinate detection model;
and determining an alarm grade based on the real-time coordinates of the personnel and carrying out corresponding safety alarm.
Optionally, the determining an alarm level based on the real-time coordinates of the person and performing a corresponding safety alarm includes:
calculating the distance between the person and the circle center based on the real-time coordinates of the person;
if the distance is between the first radius and the second radius, a first-level safety alarm is sent out;
if the distance is smaller than the first radius, a second-level safety alarm is sent out;
if the distance is larger than the second radius, no alarm is given;
wherein the warning degree of the second level of safety alarm is higher than that of the first level of safety alarm.
Optionally, the acquiring of the environmental image data of the tower crane during operation in real time further includes before acquiring the real-time coordinate of at least one person included in the environmental image data:
constructing a coordinate detection model;
acquiring a person image set under various environments, and labeling persons wearing safety helmets in the image set to generate a training set;
training the coordinate detection model based on the training set.
Optionally, the acquiring of the image data to be identified of the lifting hook on the tower crane comprises:
acquiring a multi-frame image frame of a lifting hook on the tower crane through image acquisition equipment arranged on a tower arm of the tower crane;
and acquiring the image frame with the highest definition in the multi-frame image frames based on a hill climbing algorithm to serve as image data to be identified.
Optionally, the obtaining, as image data to be identified, an image frame with the highest definition in the multiple image frames based on the hill climbing algorithm includes:
acquiring an initial image frame, carrying out gray level processing on the initial image frame, and calculating each gray level statistic value in a gray level histogram after the gray level histogram is established;
calculating a variance, and judging whether the variance is lower than a preset threshold value;
if yes, updating the preset threshold value by using the variance of the image frame, and recording the serial number of the image frame;
and if not, continuing to process the next image frame in the same way until the sequence number of the image frame corresponding to the last updated preset threshold value is obtained after all the image frames are calculated, and taking the image frame as the image data to be identified.
According to another aspect of the invention, the invention further provides a tower crane early warning device, which is applied to a tower crane system and used for performing tower crane early warning by adopting any one of the tower crane early warning methods;
this tower crane early warning device of preferred includes:
the image acquisition module is configured to monitor gravity sensor data of the tower crane and acquire image data to be identified of a lifting hook on the tower crane when the gravity sensor data changes and the change duration time exceeds preset time;
the characteristic data acquisition module is configured to analyze the image data to be identified so as to acquire characteristic data in the image data to be identified; wherein the characteristic data comprises hook characteristic data and hanging object characteristic data on the hook;
a dangerous area determining module configured to determine a dangerous warning area of the tower crane based on the characteristic data;
and the alarm module is configured to track the positions of personnel around the tower crane so that the personnel can enter the danger warning area to perform safety alarm.
According to the method provided by the embodiment of the invention, after the change of the gravity sensor data of the tower crane is monitored for a certain time, the image data to be identified of the lifting hook on the tower crane can be collected, the image data to be identified is analyzed to obtain the data related to the lifting hook and the lifting object on the lifting hook, and then the danger warning area of the tower crane is determined, so that personnel can be reminded in time after entering the danger warning area. Based on the method provided by the invention, the personnel identification and the hanging object identification are independently separated, so that the mutual interference is reduced, and the waste of resources is avoided. Further, the danger warning area can be more accurately determined on the basis of the characteristic data of the image data to be recognized in the embodiment, so that people passing through the danger warning area can be timely reminded.
Furthermore, the invention also adopts the form of a determination database to effectively collect and classify the data with the same size, thereby providing a strong data base for the follow-up confirmation of the hanging object type. In addition, the embodiment of the invention can also divide the early warning area by combining the height data and the type of the hanging object so as to carry out danger early warning on personnel entering the early warning area and effectively guarantee the personal safety of the personnel.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a schematic flow diagram of a tower crane early warning method according to an embodiment of the invention;
FIG. 2 shows a schematic structural diagram of a tower crane according to an embodiment of the invention;
FIG. 3 illustrates a flow chart of a method of acquiring image data to be identified in accordance with an embodiment of the present invention;
FIG. 4 is a diagram illustrating an image search database creation process according to an embodiment of the present invention;
FIG. 5 shows a schematic flow diagram of a tower crane early warning method according to another embodiment of the invention;
FIG. 6 shows a schematic structural diagram of a tower crane early warning device according to an embodiment of the invention; and
fig. 7 shows a schematic structural diagram of a tower crane early warning device according to another embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Fig. 1 shows a schematic flow diagram of a tower crane early warning method provided by an embodiment of the invention, and as can be seen from fig. 1, the tower crane early warning method provided by the embodiment of the invention may include:
step S101, monitoring gravity sensor data of a tower crane, and acquiring image data to be identified of a lifting hook on the tower crane when the gravity sensor data changes and the change duration time exceeds preset time;
step S102, analyzing image data to be recognized to obtain characteristic data in the image data to be recognized; the characteristic data comprises hook characteristic data and hanging object characteristic data on a hook;
step S103, determining a danger warning area of the tower crane based on the characteristic data;
and step S104, tracking the positions of personnel around the tower crane so as to perform safety alarm when the personnel enter a dangerous warning area.
According to the method provided by the embodiment of the invention, after the change of the gravity sensor data of the tower crane is monitored for a certain time, the image data to be identified of the lifting hook on the tower crane can be collected, the image data to be identified is analyzed to obtain the data related to the lifting hook and the lifting object on the lifting hook, and then the danger warning area of the tower crane is determined, so that personnel can be reminded in time after entering the danger warning area. According to the method provided by the embodiment of the invention, the personnel identification and the hanging object identification are independently separated, so that the mutual interference is reduced, and the waste of resources is avoided. Further, the danger warning area can be more accurately determined on the basis of the characteristic data of the image data to be recognized in the embodiment, so that people passing through the danger warning area can be timely reminded.
Fig. 2 shows a schematic structural diagram of a tower crane 20 according to an embodiment of the present invention, and as can be seen from fig. 2, the tower crane 20 may include: the tower crane body 21, the tower arm 22, the hook 23 connected to the tower arm 22, and various hangers 24 (not shown in the drawings) for the hook 23 to hang. Wherein, the tower arm 22 may be provided with an image collecting device 25 (for example, a camera, which may be above 400 ten thousand pixel level), which may collect an overall image overview of the tower crane operation area. In addition to the image acquisition device 25, it is also possible to have a transmission data server: and the video data stream is acquired by real-time butt joint with the image acquisition equipment 25. In addition, a gravity sensor (not shown in the figure) can be further arranged on the tower crane and used for acquiring the gravity data of the tower crane and further analyzing the working state of the tower crane. And the height sensor (not shown in the figure) is used for acquiring height data of the hoisted object 24 when the tower crane works. In fig. 2, area a is a danger area, and area B is an early warning area.
Referring to the step S102, the to-be-identified image data of the hook on the tower crane is acquired when the data of the gravity sensor of the tower crane is monitored to be changed and continues for a certain time. When the image acquisition device is used for acquiring in practical application, the multi-frame image frame of the lifting hook on the tower crane can be acquired through the image acquisition device arranged on the tower arm 22 of the tower crane 20; and acquiring the image frame with the highest definition in the multi-frame image frames based on a hill climbing algorithm to serve as image data to be identified.
That is, when the acquired data of the gravity sensor is obviously changed and is kept for a certain time (the time can be set according to different scenes, but the invention is not limited), the possibility of hanging objects on the lifting hook can be judged, and at the moment, the image acquired by the image acquisition equipment is acquired. Because image acquisition equipment sets up on tower arm 22, when the tower crane operation in the twinkling of an eye, image acquisition equipment and tower arm 22 do the follow-up, consequently just can lead to the image blurring at first, and video data is stable after a certain time, consequently, this embodiment adopts multiframe accumulative total mode, prevents that the image frame is because of the image instability that the data leads to when acquireing the image data of treating discernment suddenly. When acquiring multi-frame image data, about 10 frames may be acquired, or other number of frames may be selected as required, which is not limited in the present invention.
Further, the image with the highest definition can be selected by using a simple hill climbing algorithm, and the image characteristics are obtained by adopting an image processing mode consistent with the image retrieval. The hill climbing algorithm is a simple greedy search algorithm, the algorithm is more applied to automatic focusing of images in the 3A algorithm, and in the embodiment, image frames in a certain interval are obtained to find the clearest image. Optionally, when the image frame with the highest definition in the multiple image frames is acquired as the image data to be identified based on the hill climbing algorithm, the method may include: acquiring an initial image frame, carrying out gray level processing on the initial image frame, and calculating each gray level statistic value in a gray level histogram after establishing the gray level histogram; calculating the variance, and judging whether the variance is lower than a preset threshold value; if yes, updating a preset threshold value by using the variance of the image frame, and recording the serial number of the image frame; and if not, continuing to process the next image frame in the same way until the sequence number of the image frame corresponding to the last updated preset threshold value is obtained after all the image frames are calculated, and taking the image frame as the image data to be identified. The embodiment of the invention relates to the processing of the gray level image more during the image processing, so that the image with the highest definition can be acquired more quickly and the definition of the image can be accurately confirmed by collecting the gray level distribution of the image.
Fig. 3 shows a flowchart of a method for acquiring image data to be recognized according to an embodiment of the present invention, and as can be seen from fig. 3, the method may include:
s301, when the tower crane starts to work, acquiring image data of the tower crane continuously or at certain intervals by image acquisition equipment;
step S302, acquiring current frame image data;
step S303, carrying out gray processing on the current frame image;
step S304, establishing a gray level histogram;
step S305, calculating each gray scale statistic value of the histogram and calculating variance;
step S306, judging whether the variance obtained in step S305 is lower than an initial threshold value; if yes, go to step S307; if not, executing step S301, and continuing to analyze the next frame of image;
in step S307, the initial threshold is modified (the variance obtained in step S305 may be updated to the initial threshold), and the current frame number is recorded.
Referring to the above step S102, after the image data to be recognized is acquired, it may be analyzed to acquire hook characteristic data and hanging object characteristic data on the hook. Optionally, when the feature data is acquired, the first image size of the hook and the second image size of the suspended object may be extracted; and comparing the first image size of the lifting hook and the second image size of the hanging object with data in a preset image retrieval database to determine the category of the hanging object. The characteristic data of the hook and the characteristic data of the hanging object can be other characteristic data including the hook or the hanging object besides the respective size data. Optionally, before the feature data is acquired, the image data to be recognized may be preprocessed to classify each text image, that is, to determine the hook and the hanging object respectively. Optionally, in extracting the first image size of the hook and the second image size of the suspended object, a Scale-invariant feature transform (SIFT) algorithm may be adopted, and SIFT feature matching mainly includes 2 stages: the first stage is as follows: and (4) SIFT features are generated, namely feature vectors which are irrelevant to scale scaling, rotation and brightness change are extracted from a plurality of images. And a second stage: matching of SIFT feature vectors. The SIFT feature detection mainly comprises the following 4 basic steps:
1. detecting an extreme value of the scale space; the image locations are searched for on all scales. Potential scale-and rotation-invariant points of interest are identified by gaussian derivative functions.
2. Positioning key points; at each candidate location, the location and scale are determined by fitting a fine model. The selection of the key points depends on their degree of stability.
3. Determining the direction; one or more directions are assigned to each keypoint location based on the local gradient direction of the image. All subsequent operations on the image data are transformed with respect to the orientation, scale and location of the keypoints, providing invariance to these transformations.
4. Describing key points; local gradients of the image are measured at a selected scale in a neighborhood around each keypoint. These gradients are transformed into a representation that allows for relatively large local shape deformations and illumination variations.
Of course, in practical applications, besides the above-mentioned SIFT algorithm, a Speeded Up Robust Features (SURF) algorithm, an AKAZE (accelerated version of KAZE algorithm), an ORB (organized FAST and Rotated feature extraction and description) algorithm, or other algorithms for extracting a set of key invariant feature points from an RGB image may be used for size extraction, and the present invention is not limited thereto.
When the type of the hanging object is identified, the characteristic data is compared with the data in the preset image retrieval database. Optionally, in the embodiment of the present invention, an image retrieval database may be established first, and as can be seen from fig. 4, an establishment process of the image retrieval database provided in the embodiment of the present invention may be as follows:
step S401, collecting a plurality of image data of various types of hoisted objects on tower cranes at different time periods and different heights; the hanging object can comprise a lifting hook, a steel bar, a carrying trolley, wood with various shapes and sizes or other materials, and the invention is not limited;
step S402, analyzing each image data in the image retrieval database to obtain ROI (region of interest) in the image data;
step S403, extracting image feature data included in each piece of image data;
step S404, establishing an image retrieval database based on the image characteristic data;
step S405, performing deep learning on the image characteristic data and the corresponding relation among the sizes of the lifting hooks and the categories and sizes of the hanging objects through a deep learning model; after the characteristic data of the image data to be identified is acquired, such as the size of the hanging object, the hanging object can be searched in the image search database based on the size of the hanging object so as to quickly and accurately acquire the hanging object type corresponding to the size of the hanging object.
The embodiment of the invention provides a method for distinguishing and classifying by using shape and size similarity, which reduces the complexity of concrete classification and can better show the size characteristics. Based on the deep learning model, the present embodiment focuses more on the size characteristics and the hoisting instant category detection of the object, and proposes and acquires feature data and trains the feature data. Wherein, when above-mentioned step S401 collects image data, can install the camera in tower crane construction site commonly used, acquire each class that different time quantum was intercepted and hang the thing, do respectively: lifting hooks, steel bar sizes (different lengths), carrying trolley sizes and wood sizes, and establishing an image retrieval database. The image retrieval database contains image data of objects with different time periods and different heights, and the hanging object types can be identified more efficiently and quickly in the future.
Roi (region of interest). In machine vision and image processing, a region to be processed, called a region of interest, ROI, is delineated from a processed image in the form of a box, circle, ellipse, irregular polygon, or the like. Various operators (operators) and functions are commonly used in machine vision software such as Halcon, OpenCV, Matlab and the like to obtain a region of interest (ROI), and the image is processed in the next step. In the field of image processing, a region of interest (ROI) is an image region selected from an image, which is the focus of your image analysis. The area is delineated for further processing. The ROI is used for delineating the target which the user wants to read, so that the processing time can be reduced, and the precision can be increased. Optionally, when the step S402 acquires the ROI, the method may include:
1. determining the coordinates of a lifting hook and the size of the lifting hook;
2. establishing a by taking coordinates of a lifting hook as a center1*a1The rectangular frame of (2);
3. judging whether the maximum-size image except the lifting hook can be obtained based on the current rectangular frame;
4. if yes, obtaining the image category in the image retrieval database; if not, the value of a1 is changed and the rectangular frame is re-established until the largest size image can be obtained except for the hook.
Based on the scheme provided by the embodiment, the ROI can be acquired in a self-adaptive manner, so that the efficiency of acquiring the feature data in the image data to be identified is further improved.
Referring to the step S103, after the characteristic data is obtained, a danger warning area of the tower crane can be determined based on the characteristic data. As mentioned above, the characteristic data may include various types of data, and optionally, when determining the hazard warning area, may include: acquiring a first image size, a second image size and a first actual size of the lifting hook; and calculating the proportional relation between the first image size and the first actual size, and determining the second actual size of the suspended object based on the second image size and the proportional relation. For example, the second actual size of the sling may be determined according to the formula q2= (p 1/p 2) × q 1. Where q2 denotes the second actual size, p1 denotes the first image size, q1 denotes the second image size, and p2 denotes the first actual size. And finally, determining a danger warning area of the tower crane based on the center coordinate of the hoisted object and the second actual size of the hoisted object. The first actual size of the hook can be obtained and stored in advance based on standard size parameters of the hook which is produced from a factory, so that the hook can be used in subsequent calculation.
Optionally, since the type of the hanging object is known in the above step, after the type of the hanging object is known, a plurality of edge coordinates of the hanging object in the image data to be identified can be further obtained, and then the center coordinate of the hanging object can be determined after statistics is performed according to the plurality of edge coordinates.
In addition, the danger warning area in this embodiment may include a danger area and an early warning area, and optionally, when confirming the warning area, the method may include: determining a first circular area with the center coordinate of the hoisted object as the circle center and half of the second actual size as the first radius as a dangerous area of the tower crane; determining an annular area except the first circular area in a second circular area with the center coordinate of the hoisted object as the circle center and a second radius as an early warning area of the tower crane; the second radius is greater than the first radius. Wherein the second radius may be calculated according to the following: collecting height data of the hoisted object, multiplying the height by a preset coefficient, and adding the height data and the first radius to obtain a second radius; wherein the preset coefficient is more than 0 and less than 1.
For example, assume that a first image of the hook is taken at a size p1The first actual hook dimension of the hook is p2The second actual size of the image hanging object is q1A second actual dimension q of the suspended object2=(p1/p2)*q1Then, firstly, the dangerous area is taken as the center of a circle by taking the coordinate center of the hanging object as the center of the circle, q2Is a circle formed by the radius; the collected height data is assumed to be h, and according to the formula r = h 0.2, the center of the coordinate center of the hanging object is taken as the center of a circle (q)21/2+ r) is a radius, and the formed circle is an early warning area except a dangerous area. Second actual size q2May be the length of the sling.
Referring to step S104, after the danger warning area is determined, the positions of personnel around the tower crane can be tracked, so that safety warning can be performed when the personnel enter the danger warning area. In this embodiment, a coordinate detection model may be constructed first, and then the real-time coordinates of the person may be identified based on the coordinate detection model. Alternatively, the coordinate detection model may be constructed first; then, acquiring a person image set under various environments, and labeling persons wearing safety helmets in the image set to generate a training set; and finally, training the coordinate detection model based on the training set. In practical application, the construction site videos of workers in the past intensive time period can be collected in the daytime, at night and in rainy days; decomposing the obtained video into frame images, marking the frame images as personnel, and training the frame images through a certain deep learning model to obtain model parameters; and acquiring an image of the tower crane during operation in real time, and acquiring real-time coordinate information of personnel through model processing. The input data of the coordinate detection model may be image data in a pre-collected image set, and the output data may include coordinate information data and size data of a person in the image data. In the working process of the tower crane, personnel belong to image transverse motion detection. The mode of confirming the removal personnel that uses the detection safety helmet that this embodiment provided compares the human detail, and the characteristic of safety helmet is more outstanding. In addition, when the coordinate detection model is constructed, the coordinate detection model can be constructed based on network architectures such as a regional convolutional neural network R-CNN, Fast RCNN and FasterRCNN, and the invention is not limited.
The step S104 may further include: acquiring environmental image data of a tower crane during working in real time, and acquiring real-time coordinates of at least one person which can be included in the environmental image data based on a pre-constructed coordinate detection model; and determining the alarm grade based on the real-time coordinates of the personnel and carrying out corresponding safety alarm.
When the alarm level is determined and corresponding safety alarm is carried out, the distance between a person and the circle center can be calculated based on the real-time coordinates of the person; if the distance is between the first radius and the second radius, a first-level safety alarm is sent out; if the distance is smaller than the first radius, a second level of safety alarm is sent out; and if the distance is greater than the second radius, no alarm is given. And the warning degree of the safety alarm of the second level is higher than that of the safety alarm of the first level. The alarm information corresponding to the second-level safety alarm can attract the attention of related personnel more compared with the alarm information of the first-level safety alarm, and a stronger safety warning effect is achieved.
The center O (x) of the known danger area0,y0) The person coordinate is (x)i,yi) I =1,2 …, n, i-th person's distance from the center of the circle: r isi 2=(xi-x0)2+(yi-y0)2Then, there are:
if ri>(q21/2+ r), the state is safe, and no alarm is needed;
if ri>q2*1/2,ri<(q21/2+ r), an alarm prompt needs to be made for entering the early warning area;
if ri<q21/2, personnel enter the danger zone and need to make a strong alarm.
In practical application, different alarm audios and related audio parameters, such as content, frequency, sound size, etc., may be set for different alarms, and the present invention is not limited thereto.
The tower crane early warning method introduced in the above embodiment is explained through an embodiment.
Fig. 5 shows a schematic flow diagram of a tower crane early warning method according to another embodiment of the present invention, and as can be seen from fig. 5, the method provided in this embodiment may include:
step S501, when the data of the gravity sensor of the tower crane is monitored to be obviously changed (if the data is increased or decreased to exceed a certain value), 10 frames of image frames of the lifting hook are collected through a camera;
step S502, obtaining the clearest image frame from 10 image frames; for example, each frame of image can be read in sequence, whether the image is clearer than the previous frame of image or not is judged based on the gray value, if so, the image sequence number is recorded, and if not, the next frame of image is continuously judged;
step S503, acquiring the characteristic data in the clearest image frame in the step S502, comparing the characteristic data with the data in the image retrieval database, and acquiring the category of the hanging object on the hook;
step S504, determining the range of the dangerous area according to the related size and the category of the hanging object;
step S505, determining a wake-up early warning area based on the height data;
s506, acquiring the coordinates of personnel in the surrounding environment data of the tower crane in real time;
step S507, calculating the distance between the personnel coordinate and the center coordinate of the dangerous area;
and step S508, performing different alarm responses according to the distance.
Based on the same inventive concept, the embodiment of the invention also provides a tower crane early warning device 600, which is applied to the tower crane system shown in fig. 2, wherein the tower crane early warning device 600 is used for performing tower crane early warning by adopting the tower crane early warning method described in any one of the embodiments. As shown in fig. 6, tower crane early warning device 600 may include:
the image acquisition module 610 is configured to monitor gravity sensor data of the tower crane and acquire image data to be identified of a lifting hook on the tower crane when the gravity sensor data changes and the change duration time exceeds a preset time;
a feature data obtaining module 620 configured to analyze the image data to be recognized to obtain feature data in the image data to be recognized; the characteristic data can comprise hook characteristic data and hanging object characteristic data on a hook;
a dangerous area determining module 630 configured to determine a dangerous warning area of the tower crane based on the characteristic data;
and the alarm module 640 is configured to track the positions of personnel around the tower crane so as to perform safety alarm when the personnel enter a dangerous warning area.
In an optional embodiment of the present invention, the feature data obtaining module 620 is further configured to:
extracting a first image size of a lifting hook and a second image size of a hanging object;
and comparing the first image size of the lifting hook and the second image size of the hanging object with data in a preset image retrieval database to determine the category of the hanging object.
In an alternative embodiment of the present invention, as shown in fig. 7, the apparatus may further include:
the database establishing module 650 is configured to collect a plurality of image data of various types of hoisted objects on tower cranes at different time periods and different heights; wherein, the hanging object comprises at least one of a hook, a steel bar, a bearing trolley and wood;
analyzing each image data in the image retrieval database to obtain an ROI (region of interest) in the image data;
extracting image feature data included in each piece of image data;
establishing an image retrieval database based on the image characteristic data;
and performing deep learning on the image characteristic data and the corresponding relation among the sizes of the lifting hooks, the types and the sizes of the hanging objects through a deep learning model.
In an optional embodiment of the present invention, the hazardous area determining module 630 is further configured to:
acquiring a first image size, a second image size and a first actual size of the lifting hook;
calculating the proportional relation between the first image size and the first actual size, and determining the second actual size of the hanging object based on the second image size and the proportional relation;
and determining a danger warning area of the tower crane based on the center coordinate of the hoisted object and the second actual size of the hoisted object.
In an optional embodiment of the present invention, the hazard warning area may include a hazard area and an early warning area;
a hazardous area determination module 630, further configured to:
determining a first circular area with the center coordinate of the hoisted object as the circle center and half of the second actual size as the first radius as a dangerous area of the tower crane;
determining an annular area except the first circular area in a second circular area with the center coordinate of the hoisted object as the circle center and a second radius as an early warning area of the tower crane;
the second radius is greater than the first radius.
In an optional embodiment of the present invention, the hazardous area determining module 630 is further configured to calculate the second radius according to the following:
collecting height data of the hoisted object, multiplying the height by a preset coefficient, and adding the height data and the first radius to obtain a second radius;
wherein the preset coefficient is more than 0 and less than 1.
In an optional embodiment of the present invention, the alarm module 640 is further configured to:
acquiring environmental image data of a tower crane during working in real time, and acquiring real-time coordinates of at least one person included in the environmental image data based on a pre-constructed coordinate detection model;
and determining the alarm grade based on the real-time coordinates of the personnel and carrying out corresponding safety alarm.
In an optional embodiment of the present invention, the alarm module 640 is further configured to:
calculating the distance between the personnel and the circle center based on the real-time coordinates of the personnel;
if the distance is between the first radius and the second radius, a first-level safety alarm is sent out;
if the distance is smaller than the first radius, a second level of safety alarm is sent out;
if the distance is larger than the second radius, no alarm is given; wherein the warning degree of the second level of safety alarm is higher than that of the first level of safety alarm.
In an alternative embodiment of the present invention, as shown in fig. 7, the apparatus may further include:
a model construction module 660 configured to construct a coordinate detection model;
acquiring a person image set under various environments, and labeling persons wearing safety helmets in the image set to generate a training set;
the coordinate detection model is trained based on a training set.
In an optional embodiment of the present invention, the image capturing module 610 is further configured to:
collecting a multiframe image frame of a lifting hook on the tower crane through image collecting equipment arranged on a tower arm 22 of the tower crane;
and acquiring the image frame with the highest definition in the multi-frame image frames based on a hill climbing algorithm to serve as image data to be identified.
In an optional embodiment of the present invention, the image capturing module 610 is further configured to:
acquiring an initial image frame, carrying out gray level processing on the initial image frame, and calculating each gray level statistic value in a gray level histogram after establishing the gray level histogram;
calculating the variance, and judging whether the variance is lower than a preset threshold value;
if yes, updating a preset threshold value by using the variance of the image frame, and recording the serial number of the image frame;
and if not, continuing to process the next image frame in the same way until the sequence number of the image frame corresponding to the last updated preset threshold value is obtained after all the image frames are calculated, and taking the image frame as the image data to be identified.
According to the method provided by the embodiment of the invention, after the change of the gravity sensor data of the tower crane is monitored for a certain time, the image data to be identified of the lifting hook on the tower crane can be collected, the image data to be identified is analyzed to obtain the data related to the lifting hook and the lifting object on the lifting hook, and then the danger warning area of the tower crane is determined, so that personnel can be reminded in time after entering the danger warning area. According to the method provided by the embodiment of the invention, the personnel identification and the hanging object identification are independently separated, so that the mutual interference is reduced, and the waste of resources is avoided. Further, the danger warning area can be more accurately determined on the basis of the characteristic data of the image data to be recognized in the embodiment, so that people passing through the danger warning area can be timely reminded.
Furthermore, the embodiment of the invention also adopts the form of a determination database to effectively collect and classify the data with the same size, thereby providing a strong data base for subsequently confirming the category of the hanging objects. In addition, the embodiment of the invention can also divide the early warning area by combining the height data and the type of the hanging object so as to carry out danger early warning on personnel entering the early warning area and effectively guarantee the personal safety of the personnel.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.
Claims (13)
1. A tower crane early warning method is characterized by comprising the following steps:
monitoring gravity sensor data of a tower crane, and acquiring image data to be identified of a lifting hook on the tower crane when the gravity sensor data changes and the change duration time exceeds preset time;
analyzing the image data to be identified to obtain feature data in the image data to be identified; wherein the characteristic data comprises hook characteristic data and hanging object characteristic data on the hook;
determining a danger warning area of the tower crane based on the characteristic data;
and tracking the positions of personnel around the tower crane so that the personnel can enter the danger warning area to perform safety warning.
2. The method according to claim 1, wherein the analyzing the image data to be recognized to obtain feature data in the image data to be recognized comprises:
extracting a first image size of the hook and a second image size of the hanging object;
and comparing the first image size of the lifting hook and the second image size of the hanging object with data in a preset image retrieval database to determine the category of the hanging object.
3. The method of claim 2, wherein analyzing the image data to be recognized to obtain feature data in the image data to be recognized further comprises:
collecting a plurality of image data of various types of hoisted objects on the tower crane at different time periods and different heights; analyzing each image data in the image retrieval database to obtain a region of interest in the image data;
extracting image feature data included in each piece of the image data;
establishing an image retrieval database based on the image characteristic data;
and performing deep learning on the image characteristic data and the corresponding relation among the sizes of the lifting hooks, the types and the sizes of the hanging objects through a deep learning model.
4. The method of claim 2, wherein the determining a hazard warning area of the tower crane based on the characterization data comprises:
acquiring a first image size of the lifting hook, a second image size of a hanging object and a first actual size of the lifting hook;
calculating a proportional relation between the first image size and the first actual size, and determining a second actual size of the suspended object based on the second image size and the proportional relation;
and determining the danger warning area of the tower crane based on the center coordinate of the hanging object and the second actual size of the hanging object.
5. The method of claim 4, wherein the hazard warning area comprises a hazard area and a pre-warning area;
based on hang the central coordinate of thing and hang the second actual size of thing and confirm the danger warning area of tower crane includes:
determining a first circular area with the center coordinate of the hoisted object as the circle center and half of the second actual size as a first radius as a dangerous area of the tower crane;
determining an annular area except the first circular area in a second circular area with the center coordinate of the hoisted object as a circle center and a second radius as an early warning area of the tower crane;
the second radius is greater than the first radius;
the second physical dimension of the sling is the length of the sling.
6. The method of claim 5, wherein the second radius is calculated according to;
acquiring height data of the hoisted object, multiplying the height by a preset coefficient, and adding the height data and the first radius to obtain a second radius;
wherein the preset coefficient is more than 0 and less than 1.
7. The method of claim 5, wherein the tracking the location of personnel around the tower crane to alert safety when the personnel enter the hazard area comprises:
acquiring environmental image data of the tower crane during working in real time, and acquiring real-time coordinates of at least one person included in the environmental image data based on a pre-constructed coordinate detection model;
and determining an alarm grade based on the real-time coordinates of the personnel and carrying out corresponding safety alarm.
8. The method of claim 7, wherein determining an alarm level and issuing a corresponding safety alarm based on the real-time coordinates of the person comprises:
calculating the distance between the person and the circle center based on the real-time coordinates of the person;
if the distance is between the first radius and the second radius, a first-level safety alarm is sent out;
if the distance is smaller than the first radius, a second-level safety alarm is sent out;
if the distance is larger than the second radius, no alarm is given;
wherein the warning degree of the second level of safety alarm is higher than that of the first level of safety alarm.
9. The method according to claim 7, wherein the acquiring of the environment image data of the tower crane during operation in real time further comprises, before acquiring the real-time coordinates of at least one person included in the environment image data:
constructing a coordinate detection model;
acquiring a person image set under various environments, and labeling persons wearing safety helmets in the image set to generate a training set;
training the coordinate detection model based on the training set.
10. The method according to any one of claims 1 to 9, wherein the acquiring of the image data to be identified of the lifting hook on the tower crane comprises:
acquiring a multi-frame image frame of a lifting hook on the tower crane through image acquisition equipment arranged on a tower arm of the tower crane;
and acquiring the image frame with the highest definition in the multi-frame image frames based on a hill climbing algorithm to serve as image data to be identified.
11. The method according to claim 10, wherein the obtaining of the image frame with the highest definition from the plurality of image frames as the image data to be identified based on the hill climbing algorithm comprises:
acquiring an initial image frame, carrying out gray level processing on the initial image frame, and calculating each gray level statistic value in a gray level histogram after the gray level histogram is established;
calculating a variance, and judging whether the variance is lower than a preset threshold value;
if yes, updating the preset threshold value by using the variance of the image frame, and recording the serial number of the image frame;
and if not, continuing to process the next image frame in the same way until the sequence number of the image frame corresponding to the last updated preset threshold value is obtained after all the image frames are calculated, and taking the image frame as the image data to be identified.
12. A tower crane early warning device is applied to a tower crane system and used for carrying out tower crane early warning by adopting the tower crane early warning method as claimed in any one of claims 1 to 11.
13. The tower crane early warning device of claim 12, characterized by comprising:
the image acquisition module is configured to monitor gravity sensor data of the tower crane and acquire image data to be identified of a lifting hook on the tower crane when the gravity sensor data changes and the change duration time exceeds preset time;
the characteristic data acquisition module is configured to analyze the image data to be identified so as to acquire characteristic data in the image data to be identified; wherein the characteristic data comprises hook characteristic data and hanging object characteristic data on the hook;
a dangerous area determining module configured to determine a dangerous warning area of the tower crane based on the characteristic data;
and the alarm module is configured to track the positions of personnel around the tower crane so that the personnel can enter the danger warning area to perform safety alarm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911326235.9A CN110745704B (en) | 2019-12-20 | 2019-12-20 | Tower crane early warning method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911326235.9A CN110745704B (en) | 2019-12-20 | 2019-12-20 | Tower crane early warning method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110745704A CN110745704A (en) | 2020-02-04 |
| CN110745704B true CN110745704B (en) | 2020-04-10 |
Family
ID=69285994
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911326235.9A Active CN110745704B (en) | 2019-12-20 | 2019-12-20 | Tower crane early warning method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110745704B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111383253B (en) * | 2020-03-03 | 2020-12-15 | 清华大学 | Striking accident early warning system for construction site hoisting machinery |
| CN111062373A (en) * | 2020-03-18 | 2020-04-24 | 杭州鲁尔物联科技有限公司 | Hoisting process danger identification method and system based on deep learning |
| CN111392619B (en) * | 2020-03-25 | 2021-11-23 | 广东博智林机器人有限公司 | Tower crane early warning method, device and system and storage medium |
| JP2021163260A (en) * | 2020-03-31 | 2021-10-11 | 株式会社小松製作所 | Detection system and detection method |
| IT202000016051A1 (en) * | 2020-07-02 | 2022-01-02 | Itway S P A | ELECTRONIC SYSTEM FOR DETECTING THE PRESENCE OF A PERSON IN A LIMITED AREA |
| CN112520606A (en) * | 2020-10-21 | 2021-03-19 | 浙江大华技术股份有限公司 | Tower crane monitoring system and tower crane monitoring method |
| CN113780429B (en) * | 2021-09-14 | 2023-06-02 | 杭州大杰智能传动科技有限公司 | Tower crane material classification and identification method and system based on image analysis |
| CN114821982A (en) * | 2022-04-07 | 2022-07-29 | 中广核研究院有限公司 | Fuel assembly transportation alarm method, device, system, equipment and storage medium |
| CN114612866B (en) * | 2022-05-12 | 2022-09-02 | 东营固泰尔建筑科技有限责任公司 | Intelligent identification method, device and equipment for safety in building site |
| CN115082865B (en) * | 2022-07-27 | 2022-11-11 | 国能大渡河检修安装有限公司 | Bridge crane intrusion dangerous behavior early warning method and system based on visual image recognition |
| CN115588121B (en) * | 2022-11-03 | 2023-07-04 | 腾晖科技建筑智能(深圳)有限公司 | Tower crane object type detection method and system based on sensing data and image sequence |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2504903B (en) * | 2011-05-20 | 2016-05-25 | Optilift As | System, device and method for tracking position and orientation of vehicle, loading device and cargo in loading device operations |
| CN109071187B (en) * | 2015-12-01 | 2023-04-14 | 香港物流及供应链管理应用技术研发中心 | Safety system for machine |
| CN209038904U (en) * | 2018-08-31 | 2019-06-28 | 南京科技职业学院 | A kind of row hangs warning device |
| CN110217709B (en) * | 2019-06-14 | 2021-06-25 | 万翼科技有限公司 | Tower crane equipment, and projection prompting method and device for tower crane equipment hanging object area |
| CN110240074B (en) * | 2019-06-14 | 2021-06-25 | 万翼科技有限公司 | Tower crane equipment, and projection prompting method and device for tower crane equipment hanging object area |
-
2019
- 2019-12-20 CN CN201911326235.9A patent/CN110745704B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110745704A (en) | 2020-02-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110745704B (en) | Tower crane early warning method and device | |
| CN103576660B (en) | Smart Home method for supervising | |
| JP3785456B2 (en) | Safety monitoring device at station platform | |
| US20140369566A1 (en) | Perimeter Image Capture and Recognition System | |
| US20130170711A1 (en) | Edge detection image capture and recognition system | |
| CN106341661B (en) | Patrol robot | |
| CN106128022B (en) | A kind of wisdom gold eyeball identification violent action alarm method | |
| JP6764481B2 (en) | Monitoring device | |
| Hu et al. | Detection of workers without the helments in videos based on YOLO V3 | |
| US20130163823A1 (en) | Image Capture and Recognition System Having Real-Time Secure Communication | |
| CN109672863A (en) | A kind of construction personnel's safety equipment intelligent monitoring method based on image recognition | |
| CN112258573B (en) | Method and device for acquiring throwing position, computer equipment and storage medium | |
| CN107977639A (en) | A kind of face definition judgment method | |
| JP2012133665A (en) | Held object recognition device, held object recognition method and held object recognition program | |
| CN113743256A (en) | Construction site safety intelligent early warning method and device | |
| EP2000998B1 (en) | Flame detecting method and device | |
| KR20190050551A (en) | Apparatus and method for recognizing body motion based on depth map information | |
| CN110674753A (en) | Theft early warning method, terminal device and storage medium | |
| CN111383248A (en) | Method and device for judging red light running of pedestrian and electronic equipment | |
| CN111832450B (en) | Knife holding detection method based on image recognition | |
| CN104392201A (en) | Human fall identification method based on omnidirectional visual sense | |
| CN114821978B (en) | Method, device and medium for eliminating false alarm | |
| CN113762115B (en) | Distribution network operator behavior detection method based on key point detection | |
| JP6831396B2 (en) | Video monitoring device | |
| CN116523288A (en) | Base station constructor risk identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |