CN113536935A

CN113536935A - Safety monitoring method and equipment for engineering site

Info

Publication number: CN113536935A
Application number: CN202110671419.XA
Authority: CN
Inventors: 苏天晴; 牛卓; 胡乾坤; 李俊; 项羽升; 茆胜
Original assignee: Shenzhen Youxiang Zhilian Technology Co ltd
Current assignee: Shenzhen Youxiang Zhilian Technology Co ltd
Priority date: 2021-06-17
Filing date: 2021-06-17
Publication date: 2021-10-22

Abstract

The application is applicable to the technical field of computers, and provides a safety monitoring method and equipment for an engineering site, wherein the method comprises the following steps: identifying the target overhead view image to obtain the region type of each pixel in the target overhead view image; identifying the target ground image to obtain a target object and position information thereof in the target ground image; calculating a movement risk index between target objects; and determining a regional risk index; calculating a risk coefficient of the target object according to the movement risk index and the regional risk index; and carrying out safety monitoring on the engineering site according to the danger coefficient. According to the scheme, from the dynamic operation angle of the whole engineering field, the multidimensional information of the engineering field is comprehensively acquired, the danger coefficient of the target object is calculated through the movement danger index and the regional danger index, the safety early warning can be more comprehensively carried out through the danger coefficient, and the safety of the engineering field is more comprehensively monitored.

Description

Safety monitoring method and equipment for engineering site

Technical Field

The application belongs to the technical field of computers, and particularly relates to a safety monitoring method and equipment for an engineering site.

Background

The engineering site is a place with multiple safety accidents, so the safety of the engineering site needs to be monitored. Currently, when safety monitoring is performed on an engineering site, a computer vision or deep neural network method is adopted to detect whether a worker wears a safety helmet or not and warn the unworn worker. However, the scheme has low detection rate on whether the safety helmet is worn or not, cannot ensure enough safety, can only play a primary reminding role, has very limited safety early warning role, and cannot comprehensively monitor the safety of a project site.

Disclosure of Invention

The embodiment of the application provides a safety monitoring method and equipment for an engineering field, which can solve the problems.

In a first aspect, an embodiment of the present application provides a safety monitoring method for an engineering site, including:

acquiring a target overhead view image and a target ground image of an engineering site;

identifying the target overhead view image to obtain the region type of each pixel in the target overhead view image;

identifying the target ground image to obtain a target object and position information thereof in the target ground image; the number of the target objects is at least two;

calculating a movement risk index between the target objects according to the target objects and the position information thereof;

determining a target region to which the target object belongs according to the region type to which each pixel belongs, and determining a region risk index according to the target region;

calculating a risk coefficient of the target object according to the movement risk index and the regional risk index;

and carrying out safety monitoring on the engineering site according to the danger coefficient.

Further, the target object includes a target worker and a target vehicle; the location information of the target worker includes a first center coordinate and a first size of a first bounding box; the position information of the target vehicle includes a second center coordinate and a second size of a second enclosure frame;

the calculating of the motion risk index between the target objects according to the target objects and the position information thereof comprises:

calculating an expected time-to-collision for a target collision event between the target worker and the target vehicle based on the first center coordinate, the first dimension, the second center coordinate, and the second dimension;

and if the target collision event is a real collision event, calculating a movement risk index between the target worker and the target vehicle according to a preset movement risk index calculation rule.

Further, the target ground image comprises a plurality of image frame groups collected by the same image collecting device;

said calculating a predicted time to collision for a target collision event between said target worker and said target vehicle based on said first center coordinate, said first dimension, said second center coordinate, and said second dimension, comprising:

calculating an initial time-to-collision for a target collision event between the target worker and the target vehicle based on the first center coordinate, the first dimension, the second center coordinate, and the second dimension for each of the sets of image frames;

calculating an average of all of the initial collision times to obtain an expected collision time for a target collision event between the target worker and the target vehicle.

Further, each of the image frame groups includes at least three consecutive images;

said calculating an initial time-to-collision of a target collision event between the target worker and the target vehicle from the first center coordinate, the first dimension, the second center coordinate, and the second dimension of each of the sets of image frames comprises:

calculating a first acceleration and a first velocity of the target worker according to the first center coordinates and a first preset calculation rule of the continuous three-frame image, and calculating a second velocity according to the first acceleration and the first velocity; wherein the first speed is a speed of the target worker in a first image of the three consecutive images; the second speed is the speed of the target worker in a third frame of image of the three consecutive frames of images;

calculating a second acceleration and a third speed of the target vehicle according to the second center coordinates and a second preset calculation rule of the three continuous frame images, and calculating a fourth speed according to the second acceleration and the third speed; wherein the third speed is the speed of the target vehicle in the first frame image of the three consecutive frame images; the fourth speed is the speed of the target vehicle in a third frame image of the three continuous frame images;

calculating a target distance between the target worker and the target vehicle in a third one of the consecutive three images based on the first size, the second speed, and the fourth speed;

if it is determined from the target distance that there is a risk of collision between the target worker and the target vehicle, an initial collision time for a target collision event is calculated from the first acceleration, the second acceleration, the first dimension, the second velocity, and the fourth velocity.

Further, after the calculating a target distance between the target worker and the target vehicle in a third one of the consecutive three images according to the first size, the second speed, and the fourth speed, further comprising:

if it is determined from the target distance that there is no danger of collision between the target worker and the target vehicle, the predicted collision time of the target collision event is infinity.

Further, if the target collision event is a real collision event, calculating a movement risk index between the target worker and the target vehicle according to a preset movement risk index calculation rule, including:

if the predicted collision time is smaller than a first preset early warning time threshold value, acquiring standby collision time corresponding to the predicted collision time;

and if the standby collision time is greater than a second preset early warning time threshold and less than the first preset early warning time threshold, calculating a movement danger index between the target worker and the target vehicle according to a preset coefficient and the predicted collision time.

Further, the identifying the target top-view image to obtain the region type to which each pixel in the target top-view image belongs includes:

and inputting the target overhead view image into a trained region identification model for identification to obtain the region type of each pixel in the target overhead view image.

Further, before the inputting the target top-view image into the trained region identification model for identification to obtain the region type to which each pixel in the target top-view image belongs, the method further includes:

acquiring a sample training set; the sample training set comprises a sample top-view image and a sample region type to which each pixel corresponding to the sample top-view image belongs;

and training the initial recognition model by using the sample training set to obtain a trained region recognition model.

Further, the obtaining a training set of samples includes:

acquiring an initial overhead image, and processing the initial overhead image according to a preset image processing strategy to obtain a sample overhead image; the image processing strategy comprises one or more of a brightness adjustment strategy, a tone adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy and an image mixing strategy;

and acquiring the sample region type of each pixel corresponding to the sample top view image, and determining a sample training set according to the sample top view image and the sample region type of each pixel corresponding to the sample top view image.

In a second aspect, an embodiment of the present application provides a safety monitoring device for an engineering site, including:

the system comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring a target overhead image and a target ground image of an engineering site;

the first identification unit is used for identifying the target top view image to obtain the region type of each pixel in the target top view image;

the second identification unit is used for identifying the target ground image to obtain a target object and position information thereof in the target ground image; the number of the target objects is at least two;

the first calculation unit is used for calculating a movement risk index between the target objects according to the target objects and the position information of the target objects;

the first processing unit is used for determining a target area to which the target object belongs according to the area type to which each pixel belongs and determining an area danger index according to the target area;

the second calculation unit is used for calculating the danger coefficient of the target object according to the movement danger index and the region danger index;

and the second processing unit is used for carrying out safety monitoring on the engineering site according to the danger coefficient.

the first computing unit is specifically configured to:

Further, the first computing unit is specifically further configured to:

Further, the first calculating unit is specifically configured to:

Further, the first identification unit is specifically configured to:

Further, the first identification unit is specifically further configured to:

In a third aspect, an embodiment of the present application provides a safety monitoring device for an engineering site, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the computer program, implements the safety monitoring method for an engineering site as described in the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the method for monitoring safety of a project site according to the first aspect is implemented.

In the embodiment of the application, a target overhead view image and a target ground image of an engineering site are obtained; identifying the target overhead view image to obtain the region type of each pixel in the target overhead view image; identifying the target ground image to obtain a target object and position information thereof in the target ground image; calculating movement risk indexes among the target objects according to the target objects and the position information of the target objects; determining a target region to which the target object belongs according to the region type to which each pixel belongs, and determining a region risk index according to the target region; calculating a risk coefficient of the target object according to the movement risk index and the regional risk index; and carrying out safety monitoring on the engineering site according to the danger coefficient. According to the scheme, from the dynamic operation angle of the whole engineering field, the multidimensional information of the engineering field is comprehensively acquired, the danger coefficient of the target object is calculated through the movement danger index and the regional danger index, the safety early warning can be more comprehensively carried out through the danger coefficient, and the safety of the engineering field is more comprehensively monitored.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic flow chart of a safety monitoring method for an engineering site according to a first embodiment of the present disclosure;

fig. 2 is a schematic diagram illustrating a top view of a target in a safety monitoring method for an engineering site according to a first embodiment of the present disclosure;

fig. 3 is a schematic diagram of a region type to which each pixel in a target overhead view image belongs in a safety monitoring method for an engineering site according to a first embodiment of the present application;

fig. 4 is a schematic diagram illustrating an effect of a random scaling method in an image processing strategy in a safety monitoring method for an engineering site according to a first embodiment of the present application;

fig. 5 is a schematic diagram illustrating effects of a plurality of image processing strategies in a safety monitoring method for an engineering site according to a first embodiment of the present application;

fig. 6 is a schematic diagram illustrating effects of a plurality of image processing strategies in a safety monitoring method for an engineering site according to a first embodiment of the present application;

fig. 7 is a schematic diagram of a target object and position information thereof in a target ground image in a safety monitoring method for an engineering site according to a first embodiment of the present application;

fig. 8 is a schematic diagram of a ground camera arrangement in a safety monitoring method for an engineering site according to a first embodiment of the present application;

FIG. 9 is a schematic diagram of a safety monitoring device for a construction site provided in a second embodiment of the present application;

fig. 10 is a schematic diagram of a safety monitoring device of an engineering site according to a third embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

Referring to fig. 1, fig. 1 is a schematic flow chart of a safety monitoring method for a construction site according to a first embodiment of the present application. In this embodiment, an execution main body of the safety monitoring method for an engineering site is a device having a safety monitoring function for the engineering site, for example, a server, a desktop computer, or the like. The safety monitoring method for the engineering site as shown in fig. 1 may include:

s101: and acquiring a target overhead view image and a target ground image of the engineering site.

In this embodiment, two image capturing devices are installed at the engineering site to capture images, one is an image capturing device for high altitude overhead viewing angle, and the other is an image capturing device placed on the ground. The target overlook image is an image of an engineering site acquired by adopting a high-altitude overlook visual angle and can be acquired by an unmanned aerial vehicle or a tower crane camera and the like. As shown in fig. 2, fig. 2 is a schematic diagram of a target top view, and the target top view should include all areas in the whole project site.

The target ground image is formed by an image acquisition device placed on the ground, specifically, the image acquisition device can be placed around a project site, and when placed, the image acquisition device can be a camera placed 2 meters high from the ground. The height of 2 m is given here by way of example only and not as a limitation.

In this embodiment, the number of the target overhead view images and the target ground images acquired by the engineering site is not limited, and may be one or more. After the image acquisition device on the engineering site acquires the target overhead view image and the target ground image, the target overhead view image and the target ground image are sent to the local terminal equipment. The home terminal equipment acquires a target overhead image and a target ground image of an engineering site.

S102: and identifying the target overhead view image to obtain the region type of each pixel in the target overhead view image.

After the device acquires the target overhead view image, the device identifies the target overhead view image in a preset image identification mode, and the type of the area to which each pixel in the target overhead view image belongs is identified. In this embodiment, the preset image recognition method is not limited, as long as the type of the region to which each pixel in the target top view image belongs can be recognized.

As shown in fig. 3, fig. 3 is a schematic diagram of the region type to which each pixel belongs in the target top-view image. In general, the types of zones of a project site may include, but are not limited to, construction zones, facility zones, roadway zones, surface zones, office zones, and construction zones.

In one embodiment, in order to accurately identify the region type to which each pixel in the top view image of the target belongs, the top view image of the target may be identified by means of a neural network. The neural network mode can rapidly and accurately process the target overhead view image and output the region type of each pixel.

The equipment inputs the target overhead image into the trained region identification model for identification, and the region type of each pixel in the target overhead image is obtained. The trained region recognition model can be preset in the device, and can also be called from other devices. The trained region recognition model may include an input layer, a hidden layer, an output layer (a loss function layer). The input layer includes an input layer node for receiving an input of the object overhead image from the outside. The hidden layer is used for processing the target top view image and extracting the region type of each pixel in the target top view image. The output layer is used for outputting the region type of each pixel in the target top-view image.

In one possible implementation, the region identification model is trained in advance by the local device. The training method of the region identification model can be as follows:

the method comprises the steps that equipment obtains a sample training set, wherein the sample training set comprises a sample top view image and a sample region type to which each pixel corresponding to the sample top view image belongs; and training the initial recognition model by using a sample training set to obtain a trained region recognition model. In the training process, the sample top view image and the sample area type to which each pixel corresponding to the sample top view image belongs are used as training data, the training data are input into the initial recognition model, and the model is continuously perfected by adjusting a loss function of the initial recognition model, so that a final area recognition model is obtained.

In the process of model training, a large amount of sample data is needed to train so as to obtain a more accurate model. If the number of sample data sets is insufficient, it is likely that the model accuracy is not high enough. In the embodiment, in order to avoid the problem, the identification capability of the model on objects such as engineering machinery, workers, materials and the like under complicated and changeable construction site environment conditions, the anti-interference capability and the robustness of the model can be improved by expanding the limited samples. The basic idea of the filling is to fully consider the possible conditions on the construction site, so that the conditions which can be found on the construction site can be found in the training set, and the identification accuracy is improved.

The equipment acquires an initial overhead image, and processes the initial overhead image according to a preset image processing strategy to obtain a sample overhead image. The initial overlook image is a limited sample, and the device performs data enhancement on the limited sample through a preset image processing strategy. Then, the device obtains the sample region type to which each pixel corresponding to the sample top view image belongs, and determines a sample training set according to the sample top view image and the sample region type to which each pixel corresponding to the sample top view image belongs.

The image processing strategy comprises one or more of a brightness adjustment strategy, a tone adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy and an image mixing strategy. The image processing strategy will be described in detail below.

From the perspective of enhancing the anti-interference capability of the model, images acquired by the camera have obviously different brightness, saturation and contrast differences on the site of a construction site under different time and weather conditions; images acquired by cameras of different models have different hues; some noise may also be generated during the generation, transmission, etc. of the image. To avoid the interference of these factors on the model identification capability, the following method can be used to expand a limited number of samples in this embodiment:

1. random luminance method: in the HSV color space, the brightness components of all pixels in the image are added with a random value within a specific threshold value range, so that the brightness of the image is randomly adjusted to simulate different illumination differences of a construction site.

v_i′＝v_i+δ，δ[-Δ，Δ]，Δ∈[0,0.5]

Where P is the set of all pixels in the initial top-down image, v_iIs the luminance component of a pixel, v_i' is the processed pixel luminance component value, delta is the random increase in luminance, and delta is the luminanceA threshold for the amount of increase.

2. Random saturation method: in HSV color space, the saturation of the image is randomly adjusted by adding the saturation components of all pixels in the image to a random value within a specific threshold range so as to simulate different light environments of a construction site

s_i′＝s_i+γ，γ∈[-Γ，Γ]，Γ∈[0,0.5]

Where P is the set of all pixels in the initial top-down image, s_iIs the saturation component of a pixel, s_i' is a processed pixel saturation value, γ is a randomly increased value of saturation, and Γ is a threshold for the increase in saturation.

3. Random tone method: in the HSV color space, the tone components of all pixels in the image are added with a random value within a specific threshold value range, so that the tone of the image is randomly adjusted to simulate the tone difference of the image under different cameras and light conditions.

h_i′＝h_i+η，η∈[-H，H]，H∈[0°,180°]

Where P is the set of all pixels in the image, h_iIs the hue component of a pixel, h_i' is a processed pixel hue component value,. eta.is a randomly increased value of hue, and H is a threshold value of hue increase.

4. Random contrast method: in the RGB color space, the three components of red, green and blue are multiplied by factors within a certain threshold range to randomly enhance or reduce the contrast of the image, and the contrast difference of the image under different cameras and light conditions can be simulated.

r_i′＝r_i×α，g_i′＝g_i×α,b_i′＝b_i×α,α>0

Where r, g, b are the red, green, blue color components of a pixel, and α is the contrast factor.

5. Gaussian noise method: the noise value of two-dimensional Gaussian distribution with the mean value of 0 shown in the following formula is added to the original image, so that the anti-interference capability of the model can be effectively enhanced. In application, a noise matrix with corresponding Gaussian distribution can be obtained by selecting a proper variance, the matrix is used as an operator of convolution operation, convolution operation is carried out on an original image, and an image with Gaussian noise added can be obtained.

Wherein σ_WAnd middle sigma_HIs the variance of the horizontal axis and the vertical axis, and ρ is the correlation coefficient between W and H.

6. Salt and pepper noise method: also called impulse noise, it randomly sets some pixel points in the image as pure black or pure white. The noise can be used to simulate errors of the camera and the transmission device caused by sudden and strong interference, such as a failure of the sensor causing the pixel value to be a minimum value, namely a black point, and a saturation of the sensor causing the pixel value to be a maximum value, namely a white point.

7. Edge enhancement method: because the convolutional neural network is good at learning the texture features of the object, the method enhances the edge features in the image through the Sobel operator, so that the model can be helped to learn the effective features of the object more quickly and better, and the recognition capability of the model is improved.

From the perspective of enhancing the generalization ability of the model, in order to enable the model to identify a brand-new sample under the condition that the number of samples is limited, that is, to enable the model to be truly abstracted into the essential features of the object, the generalization ability of the model may be enhanced by using the following data enhancement method in this embodiment:

1. random zooming method: in order to enable the model to have strong recognition capability on objects with different sizes, the original image can be randomly zoomed in a certain range, for example, under the multiplying power of [0.5,1.5] to simulate the objects with different distances and sizes, so that the generalization capability of the model on the effective characteristics of the objects is improved. The specific effect is shown in fig. 4.

2. A mirror image method: by turning over the original image through the horizontal mirror image, effective samples with doubled quantity and unchanged quality can be obtained, and the method is a very effective data enhancement method. The specific effect is shown in fig. 5.

3. Random erase method: a number of rectangular areas of a particular size are randomly generated in the image, which are filled with the average pixel values of the entire image. The method can simulate the shielding effect on the object, thereby greatly improving the generalization capability of the model on the object. The effect is shown in fig. 5.

4. Random cutting method: a number of rectangular areas of a certain size are randomly generated in the image, which areas are filled with zero values, i.e. black. The method is similar to the random erasure method, derives from the idea of a common regularization method Cutout in neural network learning, can effectively avoid the problem that the model only identifies the local area of an object in the learning process, and improves the identification capability of the model. The effect is shown in fig. 5.

5. Random occlusion method: the method divides an image into S × S regions, and randomly sets the pixel values of the regions to 0. Similar to the random erasing method and the random cutting method, the method has the effect of simulating the shielding, and avoids the problem that the model only focuses on the local part of the object. Is an effective data enhancement method. The effect is shown in fig. 5.

6. Array occlusion method: the method uses a rectangular array with the pixel value of 0 to shield an original image, so that the model is forced to randomly learn each part of the object, and the recognition capability of the model to the object is improved. The effect is shown in fig. 6.

7. The mixing method comprises the following steps: the method blends two objects in an image together while evenly distributing labels. For example, in the output of the neural network, the label of the object a, i.e., the ideal output, is [ 10 ], the label of the object B, i.e., the ideal output, is [ 01 ], and the label of the fused image immediately after blending is [ 0.50.5 ]. The method uses the label smoothing thought of the neural network as a reference, is also a regularization strategy, and can effectively prevent overfitting when the neural network is trained. The effect is shown in fig. 6.

8. A shearing and mixing method: the method synthesizes a completely new image by cutting out a part from other images and pasting it to a target image. Forcing the learning of the model to have to be based on multiple features of the object, rather than some local feature that is easy to learn. Meanwhile, the label of the corresponding position of the synthesized new image is set to a ratio according to the size of the cut part and the remaining part of the target image object, such as 0.6: 0.4. the effect is shown in fig. 6.

9. Mosaic mixing method: the method combines four training images into one according to a certain proportion, and helps a model to learn how to recognize smaller objects. The method is similar to a random zooming method, but the learning efficiency is higher. The effect is shown in fig. 6.

It is to be understood that the above methods in the image processing strategy are only for illustration and are not to be construed as limitations on the image processing strategy.

S103: identifying the target ground image to obtain a target object and position information thereof in the target ground image; the number of the target objects is at least two.

The equipment identifies the target ground image to obtain a target object and position information thereof in the target ground image. In this embodiment, the method for identifying the target ground image is not limited as long as the target object and the position information thereof in the target ground image can be identified.

In particular, target objects may include workers and other work machines and vehicles; the position information may be the center coordinates (x, y) of the bounding box of the target object, and the size (w, h) of the bounding box.

As shown in fig. 7, fig. 7 is a schematic diagram of a target object and its position information in a target ground image. Wherein the number of target objects is at least two, a collision risk is only possible.

In one embodiment, in order to accurately identify the target object and the position information thereof in the target ground image, the target ground image may be identified by means of a neural network. The neural network mode can rapidly and accurately process the target object and the position information thereof in the target ground image.

And the equipment inputs the target ground image into the trained object recognition model for recognition to obtain the target object and the position information thereof in the target ground image. The trained object recognition model can be preset in the equipment, and can also be called from other equipment. The trained object recognition model may include an input layer, a hidden layer, an output layer (a loss function layer). The input layer includes an input layer node for receiving an input target ground image from the outside. The hidden layer is used for processing the target ground image and extracting the target object and the position information thereof in the target ground image. The output layer is used for outputting the target object and the position information thereof in the target ground image.

In one possible embodiment, the object recognition model is trained in advance by the home device. The training method of the object recognition model may refer to the training process of the region recognition model described in S102, and is not described herein again.

The target object and the position information thereof in the acquired target ground image can be added into a training set of a neural network to optimize the model, so that the optimized model can be better suitable for the engineering field, the anti-interference capability of the model is improved, and the accuracy of the field is improved.

S104: and calculating the movement danger index between the target objects according to the target objects and the position information thereof.

The equipment can analyze various factors influencing the safety of the target object and the degree of danger according to the motion information of dynamic entities such as the target object of the engineering site and the position information thereof obtained by the target ground image of the engineering site, thereby providing reliable warning information for the target object. The device calculates a movement risk index between the target objects according to the target objects and the position information thereof. And then, if the target collision event is a real collision event, calculating a movement risk index between the target worker and the target vehicle according to a preset movement risk index calculation rule.

Wherein, the larger the value of the motion risk index, the higher the risk, and the collision is likely to occur, and the smaller the value, the smaller the risk, and the collision occurrence probability is low or non-existent.

Specifically, the target object may include a target worker and a target vehicle, wherein the position information of the target worker includes a first center coordinate and a first size of the first enclosure; the position information of the target vehicle includes a second center coordinate and a second size of the second enclosure frame. The first center coordinate (x, y) of the first enclosure of the target worker is (w, h), the second center coordinate (x ', y') of the second enclosure of the target vehicle is (w ', h'), and the second dimension is (w ', h').

The apparatus calculates an expected time-to-collision for the target collision event between the target worker and the target vehicle based on the first center coordinate, the first dimension, the second center coordinate, and the second dimension. The position of any target object at any time can be obtained, so that the relative motion between any two target objects and the predicted collision time can be calculated.

Since the estimated collision time is calculated only once, which may have a large error and is not reliable enough, in the embodiment, a plurality of initial collision times may be calculated, and a more accurate estimated collision time may be obtained according to the plurality of initial collision times. Specifically, the target ground image includes a plurality of image frame sets captured by the same image capture device. Calculating an initial collision time for a target collision event between the target worker and the target vehicle based on the first center coordinates, the first size, the second center coordinates, and the second size of each set of image frames; an average of all initial collision times is calculated to obtain an expected collision time for a target collision event between a target worker and the target vehicle. For example, the device may be every n (0)<n<30) The initial collision time T is calculated once per m (2) for the frame<m<20) Next, the average value is calculated and the value is used as the predicted collisionTime T_collision。

In one embodiment, for more accurate calculation of the predicted time-to-collision, each group of image frames includes at least three consecutive images when calculating the initial time-to-collision;

the apparatus assumes that both the worker and the construction machine are in uniform acceleration motion when calculating the initial collision time of the target collision event between the target worker and the target vehicle based on the first center coordinates, the first size, the second center coordinates, and the second size of each set of image frames. The apparatus may first calculate a first acceleration and a first velocity of the target worker based on the first center coordinates of the consecutive three frames of images and a first preset calculation rule, and calculate a second velocity based on the first acceleration and the first velocity. Wherein the first speed is the speed of the target worker in the first frame of image in the three continuous frames of images; the second speed is the speed of the target worker in the third image of the three consecutive images.

Specifically, if the frame rate of three consecutive images is f, the time difference between two frames is Δ t equal to 1/f; setting the positions of the target worker and the target vehicle in the images in three continuous frames as

And

wherein the subscripts 1, 2, 3 herein represent three frames, respectively, and the velocities are v, respectively₁,v₂,v₃And v₁′,v₂′,v₃', acceleration is a and a' respectively; in three continuous frames of the target worker and the target vehicle, the average width and the average height of the surrounding frames are w, h and w ', h' respectively. The target worker's two-stage interframe movement can be described by the following equation:

v₂＝v₁+aΔt

wherein,

the first acceleration of the target worker is given by the above formula:

the first acceleration of the target worker is:

the second speed of the target worker is:

v₃＝v₁+2aΔt

then, the device calculates a second acceleration and a third speed of the target vehicle according to a second center coordinate of the continuous three frames of images and a second preset calculation rule, and calculates a fourth speed according to the second acceleration and the third speed; the third speed is the speed of the target vehicle in the first frame image in the three continuous frame images; the fourth speed is the speed of the target vehicle in the third frame image of the three consecutive frame images. The specific calculation method may refer to the above calculation methods of the first acceleration, the first velocity, and the second velocity, and is not described herein again.

The second acceleration, the third speed and the fourth speed are calculated according to the calculation method of the first acceleration, the first speed and the second speed, and are respectively as follows:

v′₃＝v′₁+2a′Δt

then, the apparatus calculates a target distance between the target worker and the target vehicle in a third frame image in the consecutive three frame images based on the first size, the second speed, and the fourth speed. Specifically, the target distance between the target worker and the target vehicle in the third frame image may be calculated by the following formula:

calculating a target distance X between the target worker and the target vehicle in the third frame of image₃Is to judge whether there is a collision risk between the target worker and the target vehicle, the target distance between the target worker and the target vehicle in the first frame image is X₁Before calculating the initial collision time, the relative movement direction of the two needs to be judged, if X is₁>X₃Then the two run in the same or opposite direction with a risk of collision and begin to calculate the predicted time to collision. That is, the apparatus determines that there is a risk of collision between the target worker and the target vehicle based on the target distance, calculates an initial collision time of the target collision event based on the first acceleration, the second acceleration, the first size, the second velocity, and the fourth velocity.

Specifically, the initial collision time for a target collision event may be calculated by the following equation:

when judging whether a collision danger exists between a target worker and a target vehicle, if the collision danger does not exist between the target worker and the target vehicle according to the target distance, namely X₁<X₃The two are traveling in either a backward or forward direction, and there is no risk of collision, and the predicted collision time for the target collision event is infinite.

After the predicted collision time is calculated, the construction machine or the vehicle may collide with the worker due to the fact that it is within the view angle of a specific camera, and the corresponding collision time. However, the two-dimensional planar view of the camera determines that such collision prediction may not be effective, for example, the two may be at different distances from the camera, and may only be staggered when the two are far away from the camera. Therefore, the device also needs to detect whether the target collision event is a real collision event. And if the target collision event is a real collision event, calculating a movement risk index between the target worker and the target vehicle according to a preset movement risk index calculation rule.

In this embodiment, a plurality of ground cameras may be provided to detect whether a real collision is occurring. As shown in fig. 8, fig. 8 is a schematic diagram of a ground camera arrangement, when the ground cameras are arranged in axial symmetry and the lenses are arranged in parallel and opposite, the two have completely equivalent functions in object motion analysis. Therefore, the ground cameras at the engineering site should avoid the axisymmetric arrangement, and when the number is small, an odd number of cameras should be used, thereby avoiding the waste of hardware resources.

Specifically, a first preset early warning time threshold and a second early warning time threshold are preset in the equipment, and if the predicted collision time is smaller than the first preset early warning time threshold, the standby collision time corresponding to the predicted collision time is acquired. Wherein, the standby collision time is the collision time obtained according to the target ground images collected by other cameras. And if the standby collision time is greater than the second preset early warning time threshold and less than the first preset early warning time threshold, calculating a movement risk index between the target worker and the target vehicle according to the preset coefficient and the predicted collision time. At this time, it is described that the risk of the collision event is small and the situation is not urgent, and the calculation of the index of risk of movement between the target worker and the target vehicle can be continued. The specific calculation of the risk index of movement between the target worker and the target vehicle is as follows:

α_j＝c₁exp(-c₂T_collision)+c₃

wherein in c₁、c₂And c₃Is a constant, T_collisionTo predict the time to collision.

If the standby collision time is smaller than the second preset early warning time threshold value, real collision can happen to the target workers and the target vehicle immediately, the situation is urgent, the movement danger index between the target workers and the target vehicle is not calculated continuously at the moment, collision early warning can be directly sent to the target workers, and the safety level is further improved.

S105: and determining a target region to which the target object belongs according to the region type to which each pixel belongs, and determining a region danger index according to the target region.

The device determines a target area to which the target object belongs according to the area type to which each pixel belongs, and in normal operation, the engineering machinery often appears in a certain area at a high frequency, namely a corresponding operation area. According to the analysis of the safety accident case of the construction site, areas near the working engineering machinery, construction site roads, parking lot exits, deep pits or sunken area edges and the like are high accident areas, and when workers appear in the areas, the areas generally have higher safety risks. The equipment determines the regional risk index according to the target region, and when the target object appears in a region with higher safety risk, the regional risk index is higher. For example, if the work machine is located in a work area, a road, or the like, the regional risk index value is large, and if the work machine is located in an office area, an idle area, or the like, the regional risk index value is small.

S106: and calculating the danger coefficient of the target object according to the movement danger index and the regional danger index.

And calculating the risk coefficient of the target object by the equipment to obtain the movement risk index and the regional risk index. Specifically, the motion risk index is alpha and the regional risk index is beta, and a reasonable weighting coefficient k is set_αAnd k_βThe risk factor of the target object can be calculated:

Γ_j＝k_α×α_j+k_β×β_w,h

wherein, gamma is_jIs the risk coefficient of the target object j; alpha is alpha_jThe motion risk index of the target object j indicates the collision risk of the target object j at the current moment, the larger the numerical value is, the higher the risk is, the collision is possible to happen, and the smaller the numerical value is, the smaller the risk is, the lower the collision possibility is or does not exist; beta is a_w,hThe index is an area danger index of a target object j facing an image position (w, h), if the target object j is in the range of an operation area, a road and the like of the engineering machine, the index value is larger, and the index value is smaller in the range of an office area, an idle area and the like; k is a radical of_αAnd k_βThe weighting coefficients of the motion risk index and the regional risk index are respectively.

S107: and carrying out safety monitoring on the engineering site according to the danger coefficient.

The equipment carries out safety monitoring on the engineering site according to the danger coefficient, namely, the early warning signals of corresponding levels can be output according to the calculated danger coefficient. Different danger coefficients can be preset in the equipment to correspond to different early warning signal levels, for example, when the danger coefficients are 0-30, the early warning signal levels are low risk; when the danger coefficient is 30-60, the early warning signal level is medium risk; when the danger coefficient is 60-80, the early warning signal level is high risk; the early warning signal level is a fatal risk when the risk factor is 80-100. And safety detection of the engineering site is realized through early warning signals of different degrees.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Referring to fig. 9, fig. 9 is a schematic view of a safety monitoring device in a construction site according to a second embodiment of the present application. The units are included for performing the steps in the corresponding embodiment of fig. 1. Please refer to fig. 1 for the related description of the corresponding embodiment. For convenience of explanation, only the portions related to the present embodiment are shown. Referring to fig. 9, the safety monitoring device 9 at the construction site includes:

a first obtaining unit 910, configured to obtain a target overhead view image and a target ground image of an engineering site;

a first identification unit 920, configured to identify the target top view image, to obtain a region type to which each pixel in the target top view image belongs;

a second identifying unit 930, configured to identify the target ground image to obtain a target object in the target ground image and position information thereof; the number of the target objects is at least two;

a first calculating unit 940, configured to calculate a risk index of movement between the target objects according to the target objects and the position information thereof;

a first processing unit 950, configured to determine a target region to which the target object belongs according to a region type to which each pixel belongs, and determine a regional risk index according to the target region;

a second calculating unit 960, configured to calculate a risk coefficient of the target object according to the motion risk index and the regional risk index;

and the second processing unit 970 is configured to perform safety monitoring on the engineering site according to the risk factor.

the first calculating unit 940 is specifically configured to:

Further, the first calculating unit 940 is specifically configured to:

Further, the first identifying unit 920 is specifically configured to:

Fig. 10 is a schematic diagram of a safety monitoring device of an engineering site according to a third embodiment of the present application. As shown in fig. 10, the safety monitoring device 10 of the construction site of this embodiment includes: a processor 100, a memory 101 and a computer program 102 stored in said memory 101 and operable on said processor 100, such as a safety monitoring program in an engineering site. The processor 100, when executing the computer program 102, implements the steps in the above-mentioned safety monitoring method embodiments of the engineering site, such as the steps 101 to 107 shown in fig. 1. Alternatively, the processor 100, when executing the computer program 102, implements the functions of each module/unit in the above-mentioned device embodiments, for example, the functions of the modules 910 to 970 shown in fig. 9.

Illustratively, the computer program 102 may be partitioned into one or more modules/units that are stored in the memory 101 and executed by the processor 100 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 102 in the safety monitoring device 10 at the engineering site. For example, the computer program 102 may be divided into a first acquiring unit, a first identifying unit, a second identifying unit, a first calculating unit, a first processing unit, a second calculating unit, and a second processing unit, and each unit has the following specific functions:

The safety monitoring device of the engineering site can include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will appreciate that fig. 10 is merely an example of the safety monitoring device 10 of the engineering site, and does not constitute a limitation of the safety monitoring device 10 of the engineering site, and may include more or less components than those shown, or some components in combination, or different components, for example, the safety monitoring device of the engineering site may further include an input-output device, a network access device, a bus, etc.

The Processor 100 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 101 may be an internal storage unit of the safety monitoring device 10 in the engineering field, for example, a hard disk or a memory of the safety monitoring device 10 in the engineering field. The memory 101 may also be an external storage device of the safety monitoring device 10 in the engineering site, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, which is equipped on the safety monitoring device 10 in the engineering site. Further, the safety monitoring device 10 of the engineering site may also include both an internal storage unit and an external storage device of the safety monitoring device 10 of the engineering site. The memory 101 is used for storing the computer program and other programs and data required by the safety monitoring equipment of the engineering field. The memory 101 may also be used to temporarily store data that has been output or is to be output.

It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.

An embodiment of the present application further provides a network device, where the network device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor implementing the steps of any of the various method embodiments described above when executing the computer program.

The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.

The embodiments of the present application provide a computer program product, which when running on a mobile terminal, enables the mobile terminal to implement the steps in the above method embodiments when executed.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other ways. For example, the above-described apparatus/network device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A safety monitoring method for an engineering site is characterized by comprising the following steps:

2. The safety monitoring method for a project site according to claim 1, wherein the target object includes a target worker and a target vehicle; the location information of the target worker includes a first center coordinate and a first size of a first bounding box; the position information of the target vehicle includes a second center coordinate and a second size of a second enclosure frame;

3. The safety monitoring method for engineering sites according to claim 2, wherein the target ground image comprises a plurality of image frame sets collected by the same image collecting device;

4. The safety monitoring method for engineering sites according to claim 3, wherein each group of the image frames comprises at least three continuous frames of images;

5. The safety monitoring method for a construction site according to claim 4, further comprising, after said calculating a target distance between the target worker and the target vehicle in a third image of the consecutive three images based on the first size, the second speed, and the fourth speed:

6. The safety monitoring method for engineering sites according to claim 4, wherein if the target collision event is a real collision event, calculating the moving risk index between the target worker and the target vehicle according to a preset moving risk index calculation rule comprises:

7. The safety monitoring method for the engineering site according to claim 1, wherein the identifying the target overhead view image to obtain the type of the region to which each pixel in the target overhead view image belongs comprises:

8. The method for monitoring the safety of the engineering site according to claim 7, wherein before the inputting the target overhead view image into the trained area recognition model for recognition and obtaining the area type to which each pixel in the target overhead view image belongs, the method further comprises:

9. The method for safety monitoring of an engineering site according to claim 8, wherein the obtaining of the training set of samples comprises:

10. A material quality detection apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 9 when executing the computer program.