CN113536935A - A kind of safety monitoring method and equipment for engineering site - Google Patents

Abstract

The present application is applicable to the field of computer technology, and provides a method and equipment for safety monitoring on an engineering site. The method includes: identifying a target overhead image to obtain the region type to which each pixel in the target overhead image belongs; Identify, obtain the target object and its position information in the target ground image; calculate the movement risk index between the target objects; and determine the regional risk index; calculate the risk factor of the target object according to the movement risk index and the regional risk index; The coefficient is used to monitor the safety of the project site. The above scheme, from the perspective of dynamic operation of the entire project site, comprehensively obtains the multi-dimensional information of the project site, and calculates the risk factor of the target object through the movement risk index and the regional risk index. More comprehensively monitor the safety of the project site.

Description

A kind of safety monitoring method and equipment for engineering site

technical field

The application belongs to the field of computer technology, and in particular relates to a safety monitoring method and device for a construction site.

Background technique

The construction site is a place with frequent safety accidents, so it is necessary to monitor the safety of the construction site. At present, computer vision or deep neural network methods are used in safety monitoring of engineering sites to detect whether workers wear safety helmets and warn those who do not wear them. However, this scheme has a low detection rate on whether to wear a helmet, cannot guarantee sufficient safety, and can only serve as a preliminary reminder. monitor.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a method and device for safety monitoring at a construction site, which can solve the above problems.

In the first aspect, the embodiments of the present application provide a safety monitoring method on a construction site, including:

Obtain the target overhead image and target ground image of the project site;

Identifying the target overhead image to obtain the region type to which each pixel in the target overhead image belongs;

Identifying the target ground image to obtain the target object and its position information in the target ground image; the number of the target objects is at least two;

Calculate the movement risk index between the target objects according to the target objects and their position information;

Determine the target area to which the target object belongs according to the area type to which each pixel belongs, and determine a regional risk index according to the target area;

calculating the risk factor of the target object according to the movement risk index and the regional risk index;

The safety monitoring of the engineering site is performed according to the risk factor.

Further, the target object includes a target worker and a target vehicle; the location information of the target worker includes a first center coordinate and a first size of a first bounding box; the location information of the target vehicle includes a second the second center coordinate and second size of the bounding box;

The calculating the movement risk index between the target objects according to the target objects and their position information includes:

Calculate an estimated time to collision for a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate, and the second size;

If the target collision event is a real collision event, the movement risk index between the target worker and the target vehicle is calculated according to a preset movement risk index calculation rule.

Further, the target ground image includes multiple groups of image frame groups collected by the same image collection device;

The calculating, according to the first center coordinate, the first size, the second center coordinate and the second size, an estimated time to collision for a target collision event between the target worker and the target vehicle, including :

Calculate a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate, and the second size of each set of the image frame groups the initial collision time;

An average of all the initial collision times is calculated to obtain an estimated collision time for a target collision event between the target worker and the target vehicle.

Further, each group of the image frame group includes at least three consecutive frames of images;

calculating the target between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate and the second size of each group of the image frame groups The initial collision time of the collision event, including:

Calculate the first acceleration and first speed of the target worker according to the first center coordinates of the three consecutive frames of images and the first preset calculation rule, and calculate according to the first acceleration and the first speed second speed; wherein, the first speed is the speed of the target worker in the first frame of the three consecutive frames of images; the second speed is the speed of the target worker in the third frame of the three consecutive frames of images describe the speed of the target worker;

Calculate the second acceleration and the third speed of the target vehicle according to the second center coordinate of the three consecutive frames of images and the second preset calculation rule, and calculate the second acceleration and the third speed according to the second acceleration and the third speed the fourth speed; wherein the third speed is the speed of the target vehicle in the first frame of the three consecutive frames of images; the fourth speed is the speed of the target vehicle in the third frame of the three consecutive frames of images the speed of the target vehicle;

Calculate the distance between the target worker and the target vehicle in the third frame of the three consecutive frames based on the first size, the second size, the second speed and the fourth speed target distance;

If it is determined according to the target distance that there is a danger of collision between the target worker and the target vehicle, then according to the first acceleration, the second acceleration, the first size, the second size, the The second velocity and the fourth velocity calculate the initial impact time of the target impact event.

Further, in the calculation according to the first size, the second size, the second speed and the fourth speed, the target worker and the target worker in the third frame of images in the three consecutive frames of images are calculated. After describing the target distance between the target vehicles, it also includes:

If it is determined according to the target distance that there is no danger of collision between the target worker and the target vehicle, the expected collision time of the target collision event is infinite.

Further, if the target collision event is a real collision event, calculating the movement risk index between the target worker and the target vehicle according to a preset movement risk index calculation rule, including:

If the estimated collision time is less than the first preset warning time threshold, acquiring the backup collision time corresponding to the estimated collision time;

If the backup collision time is greater than the second preset warning time threshold and less than the first preset warning time threshold, the distance between the target worker and the target vehicle is calculated according to the preset coefficient and the expected collision time exercise hazard index.

Further, identifying the target overhead image to obtain the region type to which each pixel in the target overhead image belongs, including:

The target bird's-eye view image is input into a trained region recognition model for identification, and the region type to which each pixel in the target bird's-eye view image belongs is obtained.

Further, before the input of the target bird's-eye view image into a trained region recognition model for identification, and obtaining the region type to which each pixel in the target bird's-eye view image belongs, the method further includes:

Obtain a sample training set; the sample training set includes the sample top-down image and the sample area type to which each corresponding pixel belongs;

The initial recognition model is trained using the sample training set to obtain a trained region recognition model.

Further, the obtained sample training set includes:

Obtain an initial bird's-eye view image, and process the initial bird's-eye view image according to a preset image processing strategy to obtain a sample bird's-eye view image; the image processing strategy includes a brightness adjustment strategy, a hue adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, and a noise adjustment strategy. One or more of adjustment strategies, edge enhancement strategies, image mirroring strategies, image scaling strategies, image removal strategies, and image mixing strategies;

The sample area type to which each pixel corresponding to the sample top-down image belongs is acquired, and the sample training set is determined according to the sample top-down image and the sample area type to which each corresponding pixel belongs.

In a second aspect, the embodiments of the present application provide a safety monitoring device on a construction site, including:

The first acquisition unit is used to acquire the target overhead image and the target ground image of the engineering site;

a first identifying unit, configured to identify the target bird's-eye view image, and obtain the region type to which each pixel in the target bird's-eye view image belongs;

a second identification unit, configured to identify the target ground image, and obtain the target object and its position information in the target ground image; the number of the target objects is at least two;

a first calculation unit, configured to calculate a movement risk index between the target objects according to the target objects and their position information;

a first processing unit, configured to determine the target area to which the target object belongs according to the area type to which each pixel belongs, and determine a regional risk index according to the target area;

a second calculation unit, configured to calculate the risk factor of the target object according to the movement risk index and the regional risk index;

The second processing unit is configured to perform safety monitoring on the engineering site according to the risk factor.

Further, the target object includes a target worker and a target vehicle; the location information of the target worker includes a first center coordinate and a first size of a first bounding box; the location information of the target vehicle includes a second the second center coordinate and second size of the bounding box;

The first computing unit is specifically used for:

Calculate an estimated time to collision for a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate, and the second size;

If the target collision event is a real collision event, the movement risk index between the target worker and the target vehicle is calculated according to a preset movement risk index calculation rule.

Further, the target ground image includes multiple groups of image frame groups collected by the same image collection device;

The first computing unit is specifically used for:

Calculate a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate, and the second size of each set of the image frame groups the initial collision time;

An average of all the initial collision times is calculated to obtain an estimated collision time for a target collision event between the target worker and the target vehicle.

Further, each group of the image frame group includes at least three consecutive frames of images;

The first computing unit is specifically used for:

Calculate the first acceleration and first speed of the target worker according to the first center coordinates of the three consecutive frames of images and the first preset calculation rule, and calculate according to the first acceleration and the first speed second speed; wherein, the first speed is the speed of the target worker in the first frame of the three consecutive frames of images; the second speed is the speed of the target worker in the third frame of the three consecutive frames of images describe the speed of the target worker;

Calculate the second acceleration and the third speed of the target vehicle according to the second center coordinate of the three consecutive frames of images and the second preset calculation rule, and calculate the second acceleration and the third speed according to the second acceleration and the third speed the fourth speed; wherein the third speed is the speed of the target vehicle in the first frame of the three consecutive frames of images; the fourth speed is the speed of the target vehicle in the third frame of the three consecutive frames of images the speed of the target vehicle;

Calculate the distance between the target worker and the target vehicle in the third frame of the three consecutive frames based on the first size, the second size, the second speed and the fourth speed target distance;

If it is determined according to the target distance that there is a danger of collision between the target worker and the target vehicle, then according to the first acceleration, the second acceleration, the first size, the second size, the The second velocity and the fourth velocity calculate the initial impact time of the target impact event.

Further, the first computing unit is specifically also used for:

If it is determined according to the target distance that there is no danger of collision between the target worker and the target vehicle, the expected collision time of the target collision event is infinite.

Further, the first computing unit is specifically used for:

If the estimated collision time is less than the first preset warning time threshold, acquiring the backup collision time corresponding to the estimated collision time;

If the backup collision time is greater than the second preset warning time threshold and less than the first preset warning time threshold, the distance between the target worker and the target vehicle is calculated according to the preset coefficient and the expected collision time exercise hazard index.

Further, the first identification unit is specifically used for:

The target bird's-eye view image is input into a trained region recognition model for identification, and the region type to which each pixel in the target bird's-eye view image belongs is obtained.

Further, the first identification unit is specifically also used for:

Obtain a sample training set; the sample training set includes the sample top-down image and the sample area type to which each corresponding pixel belongs;

The initial recognition model is trained using the sample training set to obtain a trained region recognition model.

Further, the first identification unit is specifically also used for:

Obtain an initial bird's-eye view image, and process the initial bird's-eye view image according to a preset image processing strategy to obtain a sample bird's-eye view image; the image processing strategy includes a brightness adjustment strategy, a hue adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, and a noise adjustment strategy. One or more of adjustment strategies, edge enhancement strategies, image mirroring strategies, image scaling strategies, image removal strategies, and image mixing strategies;

The sample area type to which each pixel corresponding to the sample top-down image belongs is acquired, and the sample training set is determined according to the sample top-down image and the sample area type to which each corresponding pixel belongs.

In a third aspect, embodiments of the present application provide a safety monitoring device on an engineering site, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing all When the computer program is used, the safety monitoring method of the engineering site as described in the first aspect above is realized.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the engineering site described in the first aspect above is implemented safety monitoring method.

In the embodiment of the present application, the target overhead image and the target ground image of the project site are acquired; the target overhead image is identified to obtain the area type to which each pixel in the target overhead image belongs; the target ground image is identified, and the target ground image is obtained. According to the target object and its position information, the movement risk index between the target objects is calculated; the target area to which the target object belongs is determined according to the area type to which each pixel belongs, and the regional risk index is determined according to the target area; According to the movement risk index and the regional risk index, the risk coefficient of the target object is calculated; the safety monitoring of the project site is carried out according to the risk coefficient. The above scheme, from the perspective of dynamic operation of the entire project site, comprehensively obtains the multi-dimensional information of the project site, and calculates the risk factor of the target object through the movement risk index and the regional risk index. More comprehensively monitor the safety of the project site.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present application. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic flowchart of a safety monitoring method on a construction site provided by the first embodiment of the present application;

2 is a schematic diagram of a top view of a target in a safety monitoring method on a construction site provided by the first embodiment of the present application;

3 is a schematic diagram of a region type to which each pixel in a target overhead image belongs in a safety monitoring method on a construction site provided by the first embodiment of the present application;

4 is a schematic diagram of the effect of a random scaling method in an image processing strategy in a safety monitoring method on a construction site provided by the first embodiment of the present application;

5 is a schematic diagram of the effect of multiple image processing strategies in a safety monitoring method on a construction site provided by the first embodiment of the present application;

6 is a schematic diagram of the effect of multiple image processing strategies in a safety monitoring method for a construction site provided by the first embodiment of the present application;

7 is a schematic diagram of a target object and its position information in a target ground image in a safety monitoring method on a construction site provided by the first embodiment of the present application;

8 is a schematic diagram of the arrangement of ground cameras in a safety monitoring method on a construction site provided by the first embodiment of the present application;

9 is a schematic diagram of a safety monitoring device on a construction site provided by the second embodiment of the present application;

FIG. 10 is a schematic diagram of a safety monitoring device on a construction site provided by the third embodiment of the present application.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It is to be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described feature, integer, step, operation, element and/or component, but does not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or sets thereof.

It will also be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items.

As used in the specification of this application and the appended claims, the term "if" may be contextually interpreted as "when" or "once" or "in response to determining" or "in response to detecting ". Similarly, the phrases "if it is determined" or "if the [described condition or event] is detected" may be interpreted, depending on the context, to mean "once it is determined" or "in response to the determination" or "once the [described condition or event] is detected. ]" or "in response to detection of the [described condition or event]".

In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and should not be construed as indicating or implying relative importance.

References in this specification to "one embodiment" or "some embodiments" and the like mean that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically emphasized otherwise. The terms "including", "including", "having" and their variants mean "including but not limited to" unless specifically emphasized otherwise.

Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of a safety monitoring method on a construction site provided by the first embodiment of the present application. In this embodiment, the execution subject of the method for safety monitoring on a project site is a device with a safety monitoring function on the project site, for example, a server, a desktop computer, and the like. As shown in Figure 1, the safety monitoring method on the project site can include:

S101: Acquire a target overhead image and a target ground image of the engineering site.

In this embodiment, two types of image acquisition devices are installed on the engineering site to acquire images, one is an image acquisition device from a high-altitude bird's-eye view, and the other is an image acquisition device placed on the ground. The target overhead image is the image of the engineering site collected from a high-altitude overhead view, which can be collected by a drone or a tower crane camera. Among them, as shown in FIG. 2, FIG. 2 is a schematic diagram of a top view of the target, and the top view of the target should include all areas in the entire project site.

The target ground image is an image acquisition device placed on the ground. Specifically, the image acquisition device can be placed around the engineering site, and when placed, the image acquisition device can be a camera placed 2 meters above the ground. Among them, the height of 2 meters here is just an example, not a limitation.

In this embodiment, the number of the target overhead image and the target ground image collected at the engineering site is not limited, which may be one or multiple. After the image acquisition device at the engineering site collects the target overhead image and the target ground image, it sends the target overhead image and the target ground image to the local device. The local device obtains the target overhead image and the target ground image of the project site.

S102: Identify the target bird's-eye view image to obtain a region type to which each pixel in the target bird's-eye view image belongs.

After the device obtains the target overhead image, it uses a preset image recognition method to identify the target overhead image, and recognizes the region type to which each pixel in the target overhead image belongs. In this embodiment, the preset image recognition method is not limited, as long as the region type to which each pixel in the target bird's-eye view image belongs can be recognized.

As shown in FIG. 3 , FIG. 3 is a schematic diagram of a region type to which each pixel in the target overhead image belongs. Generally speaking, the types of areas on a project site may include but are not limited to construction areas, facility areas, road areas, water surface areas, office areas, and construction areas.

In one embodiment, in order to accurately identify the region type to which each pixel in the target bird's-eye view image belongs, the target bird's-eye view image may be identified by means of a neural network. The neural network mode can quickly and accurately process the top-down image of the target, and output the region type to which each pixel belongs.

The device inputs the target bird's-eye view image into the trained region recognition model for recognition, and obtains the region type to which each pixel in the target bird's-eye view image belongs. The trained area recognition model can be preset in the device, or the trained area recognition model can be called from other devices. The trained region recognition model can include an input layer, a hidden layer, and an output layer (loss function layer). The input layer includes an input layer node that receives the input top-down image of the target from the outside. The hidden layer is used to process the target overhead image and extract the region type to which each pixel in the target overhead image belongs. The output layer is used to output the region type to which each pixel in the target overhead image belongs.

In a possible implementation, the region identification model is pre-trained by the local device. The training method of the region recognition model can be as follows:

The device obtains a sample training set, wherein the sample training set includes the sample top-down image and the sample area type to which each corresponding pixel belongs; the initial recognition model is trained by using the sample training set to obtain a trained area recognition model. In the training process, the sample top-down image and the sample area type to which each corresponding pixel belongs as the training data is input into the initial recognition model, and the model is continuously improved by adjusting the loss function of the initial recognition model to obtain the final area. Identify the model.

In the process of model training, a large amount of sample data is required for training to obtain a more accurate model. If the number of sample data sets is insufficient, it is likely that the model accuracy is not high enough. Acquiring a large amount of rich sample data will also consume a lot of resources. In this embodiment, in order to avoid this problem, the limited samples can be expanded to improve the model's ability to adapt to construction machinery under complex and changeable construction site environmental conditions. , workers, materials and other objects recognition ability, and its anti-interference ability and robustness. The basic idea of this filling is to fully consider the conditions that may occur on the construction site, so that the conditions that may be found on the construction site appear in the training set, thereby improving the accuracy of recognition.

The device acquires an initial bird's-eye view image, and processes the initial bird's-eye view image according to a preset image processing strategy to obtain a sample bird's-eye view image. The initial bird's-eye view image is a limited sample, and the device performs data enhancement on the limited sample through a preset image processing strategy. Then, the device acquires the sample area type to which each pixel corresponding to the sample overhead image belongs, and determines the sample training set according to the sample overhead image and the sample area type to which each corresponding pixel belongs.

The image processing strategy includes one of a brightness adjustment strategy, a hue adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy, and an image mixing strategy or more. The image processing strategy is described in detail below.

From the perspective of enhancing the anti-interference ability of the model, on the construction site, under different time and weather conditions, the images obtained by the camera will have significantly different brightness, saturation and contrast differences; the images obtained by different models of cameras will also have different hues. Differences; some noise may also be generated during image generation, transmission, etc. In order to avoid the interference of these factors on the model identification ability, the following methods can be used to expand the limited number of samples in this embodiment:

1. Random brightness method: In the HSV color space, the brightness components of all pixels in the image are added to a random value within a certain threshold range, so as to randomly adjust the brightness of the image to simulate different illumination differences on the construction site.

v _i ′ ₌ vi +δ,δ[-Δ,Δ],Δ∈[0,0.5]

Among them, P is the set of all pixels in the initial bird's-eye view image, vi is the luminance component of a certain pixel _{, v i '} is the value of the luminance component of the pixel after processing, δ is the random increase value of luminance, and Δ is the threshold value of luminance increase.

2. Random saturation method: In the HSV color space, the saturation components of all pixels in the image are added to a random value within a certain threshold range, so as to randomly adjust the saturation of the image to simulate different light environments on the construction site

s _i ′=s _i +γ, γ∈[-Γ,Γ], Γ∈[0,0.5]

where P is the set of all pixels in the initial top-down image, s _i is the saturation component of a certain pixel, s _i ′ is the value of the pixel saturation component after processing, γ is the random increase in saturation, and Γ is the increase in saturation volume threshold.

3. Random hue method: In the HSV color space, the hue components of all pixels in the image are added to a random value within a certain threshold range, so as to randomly adjust the hue of the image to simulate different camera and light conditions on the image. Hue difference.

h _i ′= _hi +η, η∈[-H, H], H∈[0°, 180°]

Among them, P is the set of all pixels in the image, _hi is the hue component of a certain pixel, _hi ' is the value of the pixel hue component after processing, η is the random increase of hue, and H is the threshold of hue increase.

4. Random contrast method: In the RGB color space, the three components of red, green and blue are multiplied by a factor within a specific threshold range to randomly enhance or weaken the contrast of the image, which can also simulate images under different camera and light conditions. contrast difference.

r _i ′=r _i ×α, g _i ′=g _i ×α,b _i ′=b _i ×α,α>0

Among them, r, g, b are the red, green, and blue color components of a pixel, and α is the contrast factor.

5. Gaussian noise method: Adding the noise value of the two-dimensional Gaussian distribution with the mean value of 0 as shown in the following formula to the original image can effectively enhance the anti-interference ability of the model. In the application, by selecting the appropriate variance, the corresponding noise matrix of Gaussian distribution can be obtained, and the matrix is used as the operator of the convolution operation to perform the convolution operation on the original image, and the image with the added Gaussian noise can be obtained.

Among them, σ _W and σ _H are the variances of the horizontal axis and the vertical axis, and ρ is the correlation coefficient between W and H.

6. Salt and pepper noise method: Also known as impulse noise, it randomly places some pixels in the image as pure black or pure white. This noise can be used to simulate the errors of cameras and transmission devices when they are subjected to sudden and strong interference. For example, a failed sensor results in a minimum pixel value, which is a black point, and a saturated sensor results in a maximum pixel value, which is a white point.

7. Edge enhancement method: Since the convolutional neural network is good at learning the texture features of objects, this method uses the Sobel operator to enhance the edge features in the image, thereby helping the model to learn the effective features of objects faster and better, thereby improving The recognition ability of the model.

From the perspective of enhancing the generalization ability of the model, in the case of a limited number of samples, in order to allow the model to recognize brand new samples, that is, to allow the model to truly abstract the essential features of objects, the following data enhancements can be used in this embodiment. method to improve the generalization ability of the model:

1. Random scaling method: In order to enable the model to have a strong ability to recognize objects of different sizes, the original image can be randomly scaled within a certain range, such as at [0.5, 1.5] magnification, to simulate different distances and sizes. object, so as to improve the generalization ability of the model to the effective features of the object. The specific effect is shown in Figure 4.

2. Mirroring method: Flip the original image by horizontal mirroring, and it is a very effective data enhancement method to obtain effective samples with double the quantity and the same quality. The specific effect is shown in Figure 5.

3. Random erasing method: randomly generate multiple rectangular areas of a specific size in the image, and fill the area with the average pixel value of the entire image. This method can simulate the occlusion effect of the object, thereby greatly improving the generalization ability of the model to the object. The effect is shown in Figure 5.

4. Random excision method: randomly generate a plurality of rectangular areas of a certain size in the image, and fill the area with a zero value, that is, black. This method is similar to the above random erasing method, which is derived from the idea of Cutout, a regularization method commonly used in neural network learning. The effect is shown in Figure 5.

5. Random occlusion method: This method divides the image into S×S regions, and randomly sets the pixel value of this region to 0. Similar to the random erasing method and the random excision method, there are simulated occlusion effects to avoid the problem that the model only pays attention to the local part of the object. It is an effective data augmentation method. The effect is shown in Figure 5.

6. Array occlusion method: This method uses a rectangular array with a pixel value of 0 to occlude the original image, thereby forcing the model to randomly learn each part of the object, thereby improving the model's ability to recognize objects. The effect is shown in Figure 6.

7. Hybrid method: This method mixes two objects in the image together, while assigning the labels equally. For example, in the output of the neural network, the label of object A, that is, the ideal output is [1 0], the label of object B, that is, the ideal output is [01], and the fused image label just after mixing is [0.5 0.5]. This method draws on the idea of label smoothing of neural networks, and is also a regularization strategy, which can effectively prevent overfitting of neural networks during training. The effect is shown in Figure 6.

8. Cut-Mix Method: This method synthesizes a completely new image by cutting parts from other images and pasting them on the target image. Therefore, the learning of the model must be based on multiple features of the object, rather than a local feature that is easy to learn. At the same time, the label of the corresponding position of the synthesized new image is set in a ratio according to the size of the cut part and the remaining part of the target image object, such as 0.6:0.4. The effect is shown in Figure 6.

9. Mosaic mixing method: This method combines four training images into one in a certain proportion to help the model learn how to recognize smaller objects. This method is similar to the random scaling method, but it is more efficient to learn. The effect is shown in Figure 6.

It can be understood that the methods in the above-mentioned image processing strategy are only for illustration, and do not constitute a limitation on the image processing strategy.

S103: Identify the target ground image to obtain target objects and their position information in the target ground image; the number of the target objects is at least two.

The device recognizes the target ground image, and obtains the target object and its position information in the target ground image. In this embodiment, there is no limitation on the identification method of the target ground image, as long as the target object and its position information in the target ground image can be identified.

Specifically, the target object may include workers and other construction machinery and vehicles; the location information may be the center coordinates (x, y) of the bounding box of the target object, and the size of the bounding box (w, h).

As shown in FIG. 7 , FIG. 7 is a schematic diagram of the target object and its position information in the target ground image. Among them, the number of target objects is at least two, and there is a possibility of a collision risk.

In one embodiment, in order to accurately identify the target object and its position information in the target ground image, the target ground image may be identified by means of a neural network. The neural network model can process the target object and its position information in the target ground image quickly and accurately.

The device inputs the target ground image into the trained object recognition model for recognition, and obtains the target object and its position information in the target ground image. The trained object recognition model can be preset in the device, or the trained object recognition model can be called from other devices. A trained object recognition model can include an input layer, a hidden layer, and an output layer (loss function layer). The input layer includes an input layer node that receives the input target ground image from the outside. The hidden layer is used to process the target ground image and extract the target object and its position information in the target ground image. The output layer is used to output the target object and its position information in the target ground image.

In a possible implementation, the object recognition model is pre-trained by the local device. For the training method of the object recognition model, reference may be made to the training process of the region recognition model described in S102, which will not be repeated here.

Among them, the target object and its position information in the target ground image obtained here can be added to the training set of the neural network to optimize the model, so that the optimized model can better adapt to the current engineering site, which is conducive to making the model anti-interference The ability has been increased, and the accuracy of the field has been improved.

S104: Calculate a movement risk index between the target objects according to the target objects and their position information.

According to the motion information of dynamic entities such as the target object and its position information on the project site obtained by the equipment based on the target ground image of the project site, we can analyze various factors that affect the safety of the target object and the level of danger, so as to provide reliable and reliable information for the target object. warning message. The device calculates the movement risk index between the target objects according to the target objects and their position information. Then, if the target collision event is a real collision event, the movement risk index between the target worker and the target vehicle is calculated according to the preset movement risk index calculation rule.

Among them, the larger the value of the movement risk index, the higher the risk, and the collision may be about to occur, and the smaller the value, the smaller the risk, and the possibility of collision is very low or does not exist.

Specifically, the target object may include a target worker and a target vehicle, wherein the location information of the target worker includes a first center coordinate and a first size of the first bounding box; the location information of the target vehicle includes a second center of the second bounding box Coordinates and second dimension. The first center coordinate (x, y) of the first bounding box of the target worker, the first size is (w, h), the second center coordinate (x', y') of the second bounding box of the target vehicle, the second The dimensions are (w',h').

The device calculates an estimated time to collision for a target collision event between the target worker and the target vehicle based on the first center coordinates, the first size, the second center coordinates, and the second size. We can obtain the position of any target object at any time, so as to calculate the relative motion and expected collision time between any two target objects.

Since calculating the estimated collision time only once may cause large errors and is not reliable enough, in this embodiment, multiple initial collision times can be calculated, and a more accurate estimated collision time can be obtained according to the multiple initial collision times. Specifically, the target ground image includes multiple sets of image frame groups collected by the same image collection device. Calculate the initial collision time of the target collision event between the target worker and the target vehicle according to the first center coordinate, first size, second center coordinate and second size of each group of image frame groups; calculate the average of all initial collision times , obtain the estimated collision time of the target collision event between the target worker and the target vehicle. For example, the device can calculate the initial collision time T every n (0<n<30) frames, every m (2<m<20) times as a group, calculate the average value, and use this value as the estimated collision time T _collision .

In one embodiment, in order to calculate the expected collision time more accurately, when calculating the initial collision time, each group of image frame groups includes at least three consecutive frames of images;

When the device calculates the initial collision time of the target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate and the second size of each group of image frame groups, here it is assumed that the worker and Construction machinery is a uniform acceleration motion. The device may first calculate the first acceleration and first speed of the target worker according to the first center coordinates of the three consecutive frames of images and the first preset calculation rule, and calculate the second speed according to the first acceleration and the first speed. Wherein, the first speed is the speed of the target worker in the first frame of the three consecutive frames of images; the second speed is the speed of the target worker in the third frame of the three consecutive frames of images.

和

其中，这里的下标1、2、3分别代表三个帧，速度分别为v₁,v₂,v₃和v₁′,v₂′,v₃′，加速度分别为a和a′；目标工人和目标车辆连续三帧内，包围框的平均宽、高分别为w,h和w′,h′。则目标工人的两段帧间运动可由下列公式描述：Specifically, the frame rate of three consecutive frames of images is f, then the time difference between the two frames is Δt=1/f; the positions of the target worker and the target vehicle in the three consecutive frames are respectively

and

Among them, the subscripts 1, 2, and 3 here represent three frames, respectively, the speeds are v ₁ , v ₂ , v ₃ and v ₁ ′, v ₂ ′, v ₃ ′, respectively, and the accelerations are a and a ′ respectively; the target The average width and height of the bounding boxes are w, h and w′, h′ in three consecutive frames of the worker and the target vehicle, respectively. Then the motion between the two frames of the target worker can be described by the following formula:

v ₂ =v ₁ +aΔt

in,

From the above formula, the first acceleration of the target worker can be obtained as:

The first acceleration of the target worker is:

The second velocity of the target worker is:

v ₃ =v ₁ +2aΔt

Then, the device calculates the second acceleration and the third speed of the target vehicle according to the second center coordinates of the three consecutive frames of images and the second preset calculation rule, and calculates the fourth speed according to the second acceleration and the third speed; The third speed is the speed of the target vehicle in the first frame of the three consecutive frames; the fourth speed is the speed of the target vehicle in the third frame of the three consecutive frames. For a specific calculation method, reference may be made to the above-mentioned calculation methods of the first acceleration, the first speed, and the second speed, which will not be repeated here.

According to the calculation method of the first acceleration, the first speed and the second speed above, the second acceleration, the third speed and the fourth speed are calculated as:

v' ₃ =v' ₁ +2a'Δt

Then, the device calculates the target distance between the target worker and the target vehicle in the third frame of images of the three consecutive frames according to the first size, the second size, the second speed, and the fourth speed. Specifically, the target distance between the target worker and the target vehicle in the third frame image can be calculated by the following formula:

Calculating the target distance X ₃ between the target worker and the target vehicle in the third frame image is to determine whether there is a danger of collision between the target worker and the target vehicle. The target distance between the target worker and the target vehicle in the first frame image is X _1. Before calculating the initial collision time, it is necessary to judge the relative movement direction of the two. If X ₁ >X ₃ , then the two move in the opposite or the same direction, and there is a danger of collision, and the estimated collision time begins to be calculated. That is, the device determines that there is a collision risk between the target worker and the target vehicle according to the target distance, and then calculates the initial collision time of the target collision event according to the first acceleration, the second acceleration, the first size, the second size, the second speed and the fourth speed. .

Specifically, the initial collision time of the target collision event can be calculated by the following formula:

When judging whether there is a danger of collision between the target worker and the target vehicle, if it is determined according to the target distance that there is no danger of collision between the target worker and the target vehicle, that is, X ₁ <X ₃ , then the two are facing away or the same If there is no collision risk, the expected collision time of the target collision event is infinite.

After calculating the expected collision time, the construction machinery or vehicle may collide with the worker due to the view of a specific camera, and the corresponding collision time. However, the two-dimensional plane viewing angle of the camera determines that this collision prediction may be invalid. For example, the distance between the two is different from the camera. One is far and the other is close, and they will only pass by staggered. Therefore, the device also needs to detect whether the target collision event is a real collision event. If the target collision event is a real collision event, the movement risk index between the target worker and the target vehicle is calculated according to the preset movement risk index calculation rule.

In this embodiment, multiple ground cameras may be set to detect whether it is a real collision. As shown in FIG. 8, FIG. 8 is a schematic diagram of the arrangement of the ground cameras. When the ground cameras are arranged axisymmetrically and the lenses are parallel to each other, the two have completely equivalent effects in the analysis of object motion. Therefore, the ground cameras on the engineering site should avoid axisymmetric arrangement, and when the number is small, an odd number of cameras should be used to avoid wasting hardware resources.

Specifically, a first preset warning time threshold and a second warning time threshold are preset in the device, and if the expected collision time is less than the first preset warning time threshold, the backup collision time corresponding to the expected collision time is obtained. The backup collision time is the collision time obtained according to the target ground images collected by other cameras. If the backup collision time is greater than the second preset warning time threshold and less than the first preset warning time threshold, the movement risk index between the target worker and the target vehicle is calculated according to the preset coefficient and the expected collision time. At this time, it means that the danger of the collision event is small and the situation is not urgent, and the movement risk index between the target worker and the target vehicle can be calculated continuously. The specific calculation method of the movement hazard index between the target worker and the target vehicle is as follows:

α _j =c ₁ exp(-c ₂ T _collision )+c ₃

Among them, c ₁ , c ₂ and c ₃ are constants, and T _collision is the expected collision time.

If the backup collision time is less than the second preset warning time threshold, the target worker and the target vehicle will have a real collision immediately, and the situation is more urgent. The target worker issued a collision warning, further enhancing the safety level.

S105: Determine the target area to which the target object belongs according to the area type to which each pixel belongs, and determine a regional risk index according to the target area.

The equipment determines the target area to which the target object belongs according to the area type to which each pixel belongs. During normal operation, construction machinery often appears in a certain area with high frequency, that is, the corresponding operation area. According to the analysis of construction site safety accident cases, the areas near the construction machinery at work, construction site roads, parking lot exits, deep pits or the edge of the depression area are high accident areas. When workers appear in these areas, they usually have high safety risks. . The device determines the regional hazard index according to the target area. When the target object appears in an area with higher security risks, the regional hazard index is also higher. For example, if you are in the working area of construction machinery, roads, etc., the value of the regional risk index is large, and in the range of office areas, idle areas, etc., the value of the regional risk index is small.

S106: Calculate the risk coefficient of the target object according to the movement risk index and the regional risk index.

The equipment calculates the movement risk index and the regional risk index, and calculates the risk factor of the target object. Specifically, if the movement risk index is α and the regional risk index is β, and reasonable weight coefficients k _α and k _β are set, the risk coefficient of the target object can be calculated:

Γ _j = k _α ×α _j +k _β ×β _w,h

Among them, Γ _j is the risk factor of the target object j; α _j is the movement risk index of the target object j, indicating the collision risk faced by the target object j at the current moment. _βw,h is the regional risk index faced by the target object j at the image position (w,h), if it is in the working area of construction machinery , roads, etc., the index value is larger, and in the office area, idle area, etc., the value is smaller; k _α and k _β are the weight coefficients of the movement risk index and the regional risk index, respectively.

S107: Perform safety monitoring on the engineering site according to the risk factor.

The equipment performs safety monitoring on the project site according to the risk factor, that is, it can output a corresponding level of early warning signal according to the calculated risk factor. Different risk factors can be preset in the device to correspond to different warning signal levels. For example, when the risk factor is 0-30, the warning signal level is low risk; when the risk factor is 30-60, the warning signal level is medium risk; When the risk factor is 60-80, the warning signal level is high risk; when the risk factor is 80-100, the warning signal level is fatal risk. Through different degrees of early warning signals, the safety detection of the project site is realized.

In the embodiment of the present application, the target overhead image and the target ground image of the project site are acquired; the target overhead image is identified to obtain the area type to which each pixel in the target overhead image belongs; the target ground image is identified, and the target ground image is obtained. According to the target object and its position information, the movement risk index between the target objects is calculated; the target area to which the target object belongs is determined according to the area type to which each pixel belongs, and the regional risk index is determined according to the target area; According to the movement risk index and the regional risk index, the risk coefficient of the target object is calculated; the safety monitoring of the project site is carried out according to the risk coefficient. The above scheme, from the perspective of dynamic operation of the entire project site, comprehensively obtains the multi-dimensional information of the project site, and calculates the risk factor of the target object through the movement risk index and the regional risk index. More comprehensively monitor the safety of the project site.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Please refer to FIG. 9. FIG. 9 is a schematic diagram of a safety monitoring device on a construction site provided by the second embodiment of the present application. The included units are used to execute the steps in the embodiment corresponding to FIG. 1 . For details, please refer to the relevant description in the embodiment corresponding to FIG. 1 . For convenience of explanation, only the parts related to this embodiment are shown. Referring to Figure 9, the safety monitoring device 9 on the engineering site includes:

a first acquiring unit 910, configured to acquire the target overhead image and the target ground image of the engineering site;

a first identifying unit 920, configured to identify the target bird's-eye view image, and obtain the region type to which each pixel in the target bird's-eye view image belongs;

The second identification unit 930 is configured to identify the target ground image, and obtain the target object and its position information in the target ground image; the number of the target objects is at least two;

a first calculation unit 940, configured to calculate a movement risk index between the target objects according to the target objects and their position information;

a first processing unit 950, configured to determine the target area to which the target object belongs according to the area type to which each pixel belongs, and determine a regional risk index according to the target area;

a second calculating unit 960, configured to calculate the risk factor of the target object according to the movement risk index and the regional risk index;

The second processing unit 970 is configured to perform safety monitoring on the engineering site according to the risk factor.

Further, the target object includes a target worker and a target vehicle; the location information of the target worker includes a first center coordinate and a first size of a first bounding box; the location information of the target vehicle includes a second the second center coordinate and second size of the bounding box;

The first computing unit 940 is specifically used for:

Calculate an estimated time to collision for a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate, and the second size;

If the target collision event is a real collision event, the movement risk index between the target worker and the target vehicle is calculated according to a preset movement risk index calculation rule.

Further, the target ground image includes multiple groups of image frame groups collected by the same image collection device;

The first computing unit 940 is specifically used for:

Calculate a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate, and the second size of each set of the image frame groups the initial collision time;

An average of all the initial collision times is calculated to obtain an estimated collision time for a target collision event between the target worker and the target vehicle.

Further, each group of the image frame group includes at least three consecutive frames of images;

The first computing unit 940 is specifically used for:

Calculate the first acceleration and first speed of the target worker according to the first center coordinates of the three consecutive frames of images and the first preset calculation rule, and calculate according to the first acceleration and the first speed second speed; wherein, the first speed is the speed of the target worker in the first frame of the three consecutive frames of images; the second speed is the speed of the target worker in the third frame of the three consecutive frames of images describe the speed of the target worker;

Calculate the second acceleration and the third speed of the target vehicle according to the second center coordinate of the three consecutive frames of images and the second preset calculation rule, and calculate the second acceleration and the third speed according to the second acceleration and the third speed the fourth speed; wherein the third speed is the speed of the target vehicle in the first frame of the three consecutive frames of images; the fourth speed is the speed of the target vehicle in the third frame of the three consecutive frames of images the speed of the target vehicle;

Calculate the distance between the target worker and the target vehicle in the third frame of the three consecutive frames based on the first size, the second size, the second speed and the fourth speed target distance;

If it is determined according to the target distance that there is a danger of collision between the target worker and the target vehicle, then according to the first acceleration, the second acceleration, the first size, the second size, the The second velocity and the fourth velocity calculate the initial impact time of the target impact event.

Further, the first computing unit 940 is further used for:

If it is determined according to the target distance that there is no danger of collision between the target worker and the target vehicle, the expected collision time of the target collision event is infinite.

Further, the first computing unit 940 is specifically used for:

If the estimated collision time is less than the first preset warning time threshold, acquiring the backup collision time corresponding to the estimated collision time;

If the backup collision time is greater than the second preset warning time threshold and less than the first preset warning time threshold, the distance between the target worker and the target vehicle is calculated according to the preset coefficient and the expected collision time exercise hazard index.

Further, the first identification unit 920 is specifically used for:

The target bird's-eye view image is input into a trained region recognition model for identification, and the region type to which each pixel in the target bird's-eye view image belongs is obtained.

Further, the first identification unit 920 is specifically also used for:

Obtain a sample training set; the sample training set includes the sample top-down image and the sample area type to which each corresponding pixel belongs;

The initial recognition model is trained using the sample training set to obtain a trained region recognition model.

Further, the first identification unit 920 is specifically also used for:

Obtain an initial bird's-eye view image, and process the initial bird's-eye view image according to a preset image processing strategy to obtain a sample bird's-eye view image; the image processing strategy includes a brightness adjustment strategy, a hue adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, and a noise adjustment strategy. One or more of adjustment strategies, edge enhancement strategies, image mirroring strategies, image scaling strategies, image removal strategies, and image mixing strategies;

The sample area type to which each pixel corresponding to the sample top-down image belongs is acquired, and the sample training set is determined according to the sample top-down image and the sample area type to which each corresponding pixel belongs.

FIG. 10 is a schematic diagram of a safety monitoring device on a construction site provided by the third embodiment of the present application. As shown in FIG. 10 , the safety monitoring device 10 on the engineering site in this embodiment includes: a processor 100 , a memory 101 , and a computer program 102 stored in the memory 101 and executable on the processor 100 , such as engineering On-site safety monitoring procedures. When the processor 100 executes the computer program 102 , the steps in the above-mentioned embodiments of the safety monitoring method for each engineering site are implemented, for example, steps 101 to 107 shown in FIG. 1 . Alternatively, when the processor 100 executes the computer program 102, the functions of the modules/units in each of the foregoing apparatus embodiments, such as the functions of the modules 910 to 970 shown in FIG. 9 , are implemented.

Exemplarily, the computer program 102 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 101 and executed by the processor 100 to complete the this application. The one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program 102 in the safety monitoring device 10 on the engineering site. For example, the computer program 102 can be divided into a first acquisition unit, a first identification unit, a second identification unit, a first calculation unit, a first processing unit, a second calculation unit, and a second processing unit, with specific functions of each unit as follows:

The first acquisition unit is used to acquire the target overhead image and the target ground image of the engineering site;

a first identifying unit, configured to identify the target bird's-eye view image, and obtain the region type to which each pixel in the target bird's-eye view image belongs;

a second identification unit, configured to identify the target ground image, and obtain the target object and its position information in the target ground image; the number of the target objects is at least two;

a first calculation unit, configured to calculate a movement risk index between the target objects according to the target objects and their position information;

a first processing unit, configured to determine the target area to which the target object belongs according to the area type to which each pixel belongs, and determine a regional risk index according to the target area;

a second calculation unit, configured to calculate the risk factor of the target object according to the movement risk index and the regional risk index;

The second processing unit is configured to perform safety monitoring on the engineering site according to the risk factor.

The safety monitoring device on the engineering site may include, but is not limited to, a processor 100 and a memory 101 . Those skilled in the art can understand that FIG. 10 is only an example of the safety monitoring device 10 on the engineering site, and does not constitute a limitation on the safety monitoring device 10 on the engineering site, and may include more or less components than those shown in the figure, or combinations thereof Some components, or different components, for example, the safety monitoring device on the engineering site may also include input and output devices, network access devices, buses, and the like.

The so-called processor 100 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 101 may be an internal storage unit of the safety monitoring device 10 on the engineering site, such as a hard disk or memory of the safety monitoring device 10 on the engineering site. The memory 101 may also be an external storage device of the safety monitoring device 10 on the engineering site, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC) equipped on the safety monitoring device 10 on the engineering site, Secure digital (Secure Digital, SD) card, flash memory card (FlashCard) and so on. Further, the safety monitoring device 10 on the engineering site may also include both an internal storage unit and an external storage device of the safety monitoring device 10 on the engineering site. The memory 101 is used for storing the computer program and other programs and data required by the safety monitoring equipment on the engineering site. The memory 101 may also be used to temporarily store data that has been output or will be output.

It should be noted that the information exchange, execution process and other contents between the above-mentioned devices/units are based on the same concept as the method embodiments of the present application. For specific functions and technical effects, please refer to the method embodiments section. It is not repeated here.

An embodiment of the present application also provides a network device, the network device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor executing The computer program implements the steps in any of the foregoing method embodiments.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

The embodiments of the present application provide a computer program product, when the computer program product runs on a mobile terminal, the steps in the foregoing method embodiments can be implemented when the mobile terminal executes the computer program product.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the methods of the above embodiments, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When executed by the processor, the steps of the above-mentioned various method embodiments may be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the photographing device/terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, U disk, mobile hard disk, disk or CD, etc. In some jurisdictions, under legislation and patent practice, computer readable media may not be electrical carrier signals and telecommunications signals.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/network device and method may be implemented in other manners. For example, the apparatus/network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units. Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it can still be used for the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application.

Claims (10)

1. A safety monitoring method for an engineering site is characterized by comprising the following steps:

acquiring a target overhead view image and a target ground image of an engineering site;

identifying the target overhead view image to obtain the region type of each pixel in the target overhead view image;

identifying the target ground image to obtain a target object and position information thereof in the target ground image; the number of the target objects is at least two;

calculating a movement risk index between the target objects according to the target objects and the position information thereof;

determining a target region to which the target object belongs according to the region type to which each pixel belongs, and determining a region risk index according to the target region;

calculating a risk coefficient of the target object according to the movement risk index and the regional risk index;

and carrying out safety monitoring on the engineering site according to the danger coefficient.

2. The safety monitoring method for a project site according to claim 1, wherein the target object includes a target worker and a target vehicle; the location information of the target worker includes a first center coordinate and a first size of a first bounding box; the position information of the target vehicle includes a second center coordinate and a second size of a second enclosure frame;

the calculating of the motion risk index between the target objects according to the target objects and the position information thereof comprises:

calculating an expected time-to-collision for a target collision event between the target worker and the target vehicle based on the first center coordinate, the first dimension, the second center coordinate, and the second dimension;

and if the target collision event is a real collision event, calculating a movement risk index between the target worker and the target vehicle according to a preset movement risk index calculation rule.

3. The safety monitoring method for engineering sites according to claim 2, wherein the target ground image comprises a plurality of image frame sets collected by the same image collecting device;

said calculating a predicted time to collision for a target collision event between said target worker and said target vehicle based on said first center coordinate, said first dimension, said second center coordinate, and said second dimension, comprising:

calculating an initial time-to-collision for a target collision event between the target worker and the target vehicle based on the first center coordinate, the first dimension, the second center coordinate, and the second dimension for each of the sets of image frames;

calculating an average of all of the initial collision times to obtain an expected collision time for a target collision event between the target worker and the target vehicle.

4. The safety monitoring method for engineering sites according to claim 3, wherein each group of the image frames comprises at least three continuous frames of images;

said calculating an initial time-to-collision of a target collision event between the target worker and the target vehicle from the first center coordinate, the first dimension, the second center coordinate, and the second dimension of each of the sets of image frames comprises:

calculating a first acceleration and a first velocity of the target worker according to the first center coordinates and a first preset calculation rule of the continuous three-frame image, and calculating a second velocity according to the first acceleration and the first velocity; wherein the first speed is a speed of the target worker in a first image of the three consecutive images; the second speed is the speed of the target worker in a third frame of image of the three consecutive frames of images;

calculating a second acceleration and a third speed of the target vehicle according to the second center coordinates and a second preset calculation rule of the three continuous frame images, and calculating a fourth speed according to the second acceleration and the third speed; wherein the third speed is the speed of the target vehicle in the first frame image of the three consecutive frame images; the fourth speed is the speed of the target vehicle in a third frame image of the three continuous frame images;

calculating a target distance between the target worker and the target vehicle in a third one of the consecutive three images based on the first size, the second speed, and the fourth speed;

if it is determined from the target distance that there is a risk of collision between the target worker and the target vehicle, an initial collision time for a target collision event is calculated from the first acceleration, the second acceleration, the first dimension, the second velocity, and the fourth velocity.

5. The safety monitoring method for a construction site according to claim 4, further comprising, after said calculating a target distance between the target worker and the target vehicle in a third image of the consecutive three images based on the first size, the second speed, and the fourth speed:

if it is determined from the target distance that there is no danger of collision between the target worker and the target vehicle, the predicted collision time of the target collision event is infinity.

6. The safety monitoring method for engineering sites according to claim 4, wherein if the target collision event is a real collision event, calculating the moving risk index between the target worker and the target vehicle according to a preset moving risk index calculation rule comprises:

if the predicted collision time is smaller than a first preset early warning time threshold value, acquiring standby collision time corresponding to the predicted collision time;

and if the standby collision time is greater than a second preset early warning time threshold and less than the first preset early warning time threshold, calculating a movement danger index between the target worker and the target vehicle according to a preset coefficient and the predicted collision time.

7. The safety monitoring method for the engineering site according to claim 1, wherein the identifying the target overhead view image to obtain the type of the region to which each pixel in the target overhead view image belongs comprises:

and inputting the target overhead view image into a trained region identification model for identification to obtain the region type of each pixel in the target overhead view image.

8. The method for monitoring the safety of the engineering site according to claim 7, wherein before the inputting the target overhead view image into the trained area recognition model for recognition and obtaining the area type to which each pixel in the target overhead view image belongs, the method further comprises:

acquiring a sample training set; the sample training set comprises a sample top-view image and a sample region type to which each pixel corresponding to the sample top-view image belongs;

and training the initial recognition model by using the sample training set to obtain a trained region recognition model.

9. The method for safety monitoring of an engineering site according to claim 8, wherein the obtaining of the training set of samples comprises:

acquiring an initial overhead image, and processing the initial overhead image according to a preset image processing strategy to obtain a sample overhead image; the image processing strategy comprises one or more of a brightness adjustment strategy, a tone adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy and an image mixing strategy;

and acquiring the sample region type of each pixel corresponding to the sample top view image, and determining a sample training set according to the sample top view image and the sample region type of each pixel corresponding to the sample top view image.

10. A material quality detection apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 9 when executing the computer program.

Images