CN111931706A

CN111931706A - Man-machine collision early warning method and system for construction site

Info

Publication number: CN111931706A
Application number: CN202010970370.3A
Authority: CN
Inventors: 方东平; 古博韬; 吴言坤; 张顶立; 郭红领; 黄玥诚; 苗春刚; 陈健正; 周予启
Original assignee: Tsinghua University; China Railway 14th Bureau Group Co Ltd
Current assignee: Tsinghua University; China Railway 14th Bureau Group Co Ltd
Priority date: 2020-09-16
Filing date: 2020-09-16
Publication date: 2020-11-13
Anticipated expiration: 2040-09-16
Also published as: CN111931706B

Abstract

The disclosure relates to a man-machine collision early warning method and a system for a construction site, wherein the system comprises the following steps: the data acquisition module is used for respectively acquiring current image data, depth image data and motion data of the mechanical equipment on a construction site through a plurality of binocular cameras and a plurality of sensors which are arranged on the movable mechanical equipment on the construction site; the processing module is used for analyzing and processing each current image data to determine the contour pixel coordinates of workers in each current image data, and determining the distance between each worker and the mechanical equipment and the color combination of the safety helmet of each worker in each current image data according to the contour pixel coordinates of the workers, the depth image data and the motion data of the corresponding mechanical equipment; the risk identification module is used for identifying the risk level corresponding to the current image data according to the distance between the worker and the mechanical equipment in each current image data and the color combination of the safety helmet of the worker; and the early warning module determines a corresponding human-computer collision early warning prompting mode to prompt according to the risk level.

Description

Man-machine collision early warning method and system for construction site

Technical Field

The disclosure relates to the technical field of safety monitoring of construction sites, in particular to a man-machine collision early warning method and system for a construction site.

Background

With the development of construction technology in the construction industry, more and more mobile construction machines are used at work sites, such as excavators, forklifts, cranes, and the like. The use of mobile construction machines introduces the risk of human-machine collision to the worksite. Accidents related to collisions at construction sites account for 22% of the total accidents, and man-machine collisions are becoming one of the main causes of construction site accidents.

The reasons for the occurrence of a man-machine collision accident at a construction site are as follows. The first is that the machine may be in a state of whole or partial movement due to the needs of work tasks, such as the forklift moving forward or backward integrally when shoveling soil, the crane extending boom or rotating crane when hoisting; secondly, workers work near the machine due to the working task requirements or site condition limitation, such as a ropeman binding the materials hoisted by the crane, the workers and the machine work in a narrow site at the same time. When the worker and the machine are close to each other in the same space-time range and are smaller than a certain threshold value, the worker enters a machine danger area to form a man-machine collision risk.

The construction site machines involve a large number of whole or partial movements during normal operation, so the core of controlling the risk of man-machine collision of mobile machines lies in controlling the combination of the number and the type of the operating personnel in the danger area. The types of workers are distinguished by the colors of safety helmets, a red safety helmet represents technicians and managers, a yellow safety helmet represents ordinary workers, a blue safety helmet represents technicians, and a white safety helmet represents supervision or Party A. Every kind of machinery corresponds required workman personnel combination diverse when the operation, and when the operation machinery is the common operating mode rather than the staff combination that corresponds, then represent man-machine collision risk this moment and be lower, otherwise when the operation machinery can't match with the staff combination, then represent the high risk operating mode that has appeared, needs early warning suggestion. The high risk condition has the following five conditions:

1) the operation machine is matched with the worker in a combined mode but is not common, the operation machine represents low-frequency correct operation, due to the fact that the operation machine is low in relative frequency, proficiency of workers and managers is low, and early warning is needed to improve attention of the worker;

2) a visual blind area exists, and the existence of the blind area needs to be early warned at the moment;

3) workers in the non-operation area enter, and due to the fact that the mobile machinery is relatively flexible in operation, the workers in the non-operation area enter the operation area, and at the moment, early warning is needed to drive the workers in the non-operation area;

4) lack of workers or supervision personnel in the operation area, and need early warning and prompt high-risk operation at the moment

5) The number of the supervisors is too much, the supervisors are sometimes gathered in a risk area to bring unnecessary risks, and at the moment, the unnecessary supervisors need to be driven by early warning.

At present, an effective technology or management means is lacked in a construction site for identifying and early warning high-risk working conditions related to the mobile machinery.

In recent years, artificial intelligence technology has been rapidly developed, and the artificial intelligence technology has been successful in various fields such as image recognition, semantic analysis, data mining and the like. The artificial intelligence technology can provide support for identifying high-risk working conditions of a construction site, but the artificial intelligence technology depends on the strong computing power of hardware equipment, and collision accidents require that the accident risk identification does not exceed 1 s. In addition, because the construction site conditions are complex and large-size and high-power embedded equipment is difficult to accommodate, a mobile mechanical man-machine collision accident recognition and early warning system on the construction site is required to be implemented in a small-size embedded edge computing platform.

Disclosure of Invention

In order to overcome the problems in the related art, the invention provides a man-machine collision early warning method and system for a construction site.

According to a first aspect of the embodiments of the present disclosure, there is provided a human-computer collision early warning system for a construction site based on an embedded edge computing platform, including:

the data acquisition module is used for respectively acquiring current image data, depth image data and motion data of the mechanical equipment on a construction site through a plurality of binocular cameras and a plurality of sensors which are arranged on the movable mechanical equipment on the construction site;

the processing module is connected to the data acquisition module and used for analyzing and processing each current image data to determine the contour pixel coordinates of workers in each current image data and determine the distance between the workers and the mechanical equipment and the color combination of the safety helmets of the workers in each current image data according to the contour pixel coordinates of the workers, the depth image data and the corresponding motion data of the mechanical equipment;

a risk identification module connected to the processing module and used for identifying risk levels corresponding to the current image data according to the distance between the worker and the mechanical equipment in each current image data and the color combination of the safety helmet of the worker, wherein the risk levels comprise a high risk level, a medium risk level and a low risk level;

and the early warning module is connected to the risk identification module and used for determining a corresponding human-computer collision early warning prompting mode for prompting according to the risk grade.

In one embodiment, preferably, the processing module includes:

the noise reduction unit is used for carrying out data noise reduction processing on each current image data to obtain processed image data;

the computing unit is used for computing the processed image data by adopting a MASK-RCNN algorithm so as to obtain the worker contour pixel coordinates of each worker in the image data;

the distance determining unit is used for determining the distance between each worker and the corresponding mechanical equipment according to the center point coordinates of the contour pixel coordinates of the workers and the depth image data;

the level determining unit is used for identifying the target risk distance level corresponding to each worker according to the distance between each worker and the corresponding mechanical equipment and the motion data of the mechanical equipment, and deleting the identification result of the worker with the distance larger than the risk distance threshold value;

the number determining unit is used for determining the number of target pixel points corresponding to the target risk distance level according to the target risk distance level of each undeleted worker and the corresponding relation between the preset risk distance level and the number of pixel points;

the numerical value determining unit is used for acquiring pixel points from the lower part of the vertex of the contour pixel coordinate of the worker as color anchor points according to the number of the target pixel points so as to determine the HSV value of the safety helmet of the worker;

and the color judging unit is used for judging the color of the safety helmet of the worker according to the HSV value of the safety helmet so as to obtain the color combination of the safety helmet corresponding to each current image data.

In one embodiment, preferably, the risk identification module comprises:

the acquiring unit is used for acquiring massive image data and color combinations of worker safety helmets corresponding to the image data;

the clustering unit is used for clustering the frequency of the color combination of the artificial safety helmet in the massive image data through a clustering algorithm so as to divide the massive image data into a low-frequency combination, a medium-frequency combination and a high-frequency combination;

the searching unit is used for searching a target combination corresponding to the safety helmet color combination of the worker in each current image data in the worker safety helmet color combinations corresponding to the massive image data to obtain a searching result;

and the risk determining unit is used for determining the risk level corresponding to the current image data according to the search result.

In one embodiment, preferably, the risk determination unit is further configured to:

and when the target combination is found, determining a corresponding risk level according to the frequency combination to which the target combination belongs. The low-frequency combination corresponds to a high risk level, the medium-frequency combination corresponds to a medium risk level, and the high-frequency combination corresponds to a low risk level;

and when the target combination is not found, determining that the corresponding risk level is a high risk level.

In one embodiment, preferably, the early warning module is configured to:

and when the risk level is a high risk level, outputting buzzing and voice prompt to prompt workers to reduce safety risks.

According to a second aspect of the embodiments of the present disclosure, there is provided a human-computer collision warning method for a construction site, including:

the method comprises the steps that current image data, depth image data and motion data of mechanical equipment of a construction site are collected through a plurality of binocular cameras and a plurality of sensors which are arranged on movable mechanical equipment of the construction site;

analyzing each current image data to determine the contour pixel coordinates of a worker in each current image data, and determining the distance between the worker and the mechanical equipment and the color combination of the safety helmet of the worker in each current image data according to the contour pixel coordinates of the worker, the depth image data and the motion data of the corresponding mechanical equipment;

identifying risk levels corresponding to the current image data according to the distance between the worker and the mechanical equipment and the color combination of the safety helmet of the worker in each current image data, wherein the risk levels comprise a high risk level, a medium risk level and a low risk level;

and determining a corresponding human-computer collision early warning prompting mode for prompting according to the risk level.

In one embodiment, preferably, the analyzing each current image data to determine the worker contour pixel coordinates in each current image data, and determining the distance between the worker and the mechanical device and the helmet color combination of the worker in each current image data according to the worker contour pixel coordinates, the depth image data and the motion data of the corresponding mechanical device, comprises:

performing data noise reduction processing on each current image data to obtain processed image data;

calculating the processed image data by adopting a MASK-RCNN algorithm to obtain the worker contour pixel coordinates of each worker in the image data;

determining the distance between each worker and the corresponding mechanical equipment according to the coordinates of the center point of the coordinates of the contour pixels of the workers and the depth image data;

identifying a target risk distance grade corresponding to each worker according to the distance between each worker and the corresponding mechanical equipment and the motion data of the mechanical equipment, and deleting the identification result of the worker with the distance greater than a risk distance threshold value;

determining the number of target pixel points corresponding to the target risk distance level according to the target risk distance level of each undeleted worker and the corresponding relation between the preset risk distance level and the number of pixel points;

according to the number of the target pixel points, acquiring pixel points from the lower part of the vertex of the contour pixel coordinate of the worker as color anchor points so as to determine the HSV value of the safety helmet of the worker;

and judging the color of the safety helmet of the worker according to the HSV value of the safety helmet so as to obtain the color combination of the safety helmet corresponding to each current image data.

In one embodiment, preferably, identifying the risk level corresponding to each current image data according to the distance between the worker and the mechanical equipment and the color combination of the helmet of the worker in each current image data comprises:

acquiring massive image data and a color combination of a worker safety helmet corresponding to each image data;

clustering the frequency of the color combination of the artificial safety helmet in the massive image data through a clustering algorithm to divide the massive image data into a low-frequency combination, a medium-frequency combination and a high-frequency combination;

searching a target combination corresponding to the safety helmet color combination of the worker in each current image data in the worker safety helmet color combinations corresponding to the massive image data to obtain a search result;

and determining the risk level corresponding to the current image data according to the search result.

In one embodiment, preferably, the determining the risk level corresponding to the current image data according to the search result includes:

In one embodiment, preferably, the determining, according to the risk level, a corresponding human-computer collision warning prompting manner for prompting includes:

According to a third aspect of the embodiments of the present disclosure, there is provided a human-computer collision warning system for a construction site, including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

in the embodiment of the invention, whether peripheral operators form risks or not can be actively judged aiming at the mobile machinery, early warning can be carried out, safety management personnel can be helped to identify the safety risks, and the operators can be helped to find high-risk working conditions so as to take measures to reduce the risks.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a schematic structural diagram illustrating a man-machine collision warning system of a construction site according to an exemplary embodiment.

Fig. 2 is a schematic structural diagram illustrating a data acquisition module in a human-computer collision warning system in a construction site according to an exemplary embodiment.

Fig. 3 is a schematic diagram illustrating binocular vision algorithm calculations according to an exemplary embodiment.

FIG. 4 is a schematic diagram of a Cartesian coordinate system shown in accordance with an exemplary embodiment.

Fig. 5 is a schematic diagram illustrating a processing module of a man-machine collision warning system in a construction site according to an exemplary embodiment.

FIG. 6 is a schematic diagram illustrating histogram clipping, according to an example embodiment.

FIG. 7 is a schematic diagram illustrating a worker contour pixel in accordance with one illustrative embodiment.

FIG. 8 is a block diagram illustrating a risk identification module of a human-machine collision warning system at a job site in accordance with an exemplary embodiment.

FIG. 9 is a flow chart illustrating a method for human-machine collision warning at a job site according to an exemplary embodiment.

Fig. 10 is a flowchart illustrating a step S102 in a man-machine collision warning method for a construction site according to an exemplary embodiment.

Fig. 11 is a flowchart illustrating a step S103 in a man-machine collision warning method for a construction site according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The man-machine collision early warning system and method for a construction site based on deep learning vision technology according to the embodiment of the invention are described below with reference to the accompanying drawings, and firstly, the man-machine collision early warning system for a construction site according to the embodiment of the invention will be described with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram illustrating a human-machine collision warning system of a construction site according to an exemplary embodiment, and as shown in fig. 1, a human-machine collision warning system 10 of a construction site includes: the system comprises a data acquisition module 11, a processing module 12, a risk identification module 13 and an early warning module 14;

the data acquisition module 11 is used for respectively acquiring current image data, depth image data and motion data of the mechanical equipment on a construction site through a plurality of binocular cameras and a plurality of sensors arranged on the mobile mechanical equipment on the construction site;

as shown in fig. 2, the data acquisition module includes a Realsense camera, a USB optical fiber extension line, and a USB auxiliary power supply line, and the Realsense camera includes a binocular camera and an IMU sensor. The data acquisition module is installed on portable machinery automobile body, and the height is at 1.5 meters, and the quantity is with covering staff's scope as the standard. The binocular camera captures a working image horizontally, a depth image is obtained through a binocular vision algorithm, and the distance between a worker and a vehicle body is calculated, wherein a specific algorithm schematic diagram is shown in fig. 3.

The object P in figure 3 is a distance object to be measured,

and

is two cameras of a binocular camera, T is the distance between the actual two cameras, f is the camera focal length, so T and f are known parameters. According to the principle of similar triangles

So solving the object distance problem becomes determining the parallax

To a problem of (a).

The parallax is the difference value of corresponding x coordinates of the same space point in the imaging of the two cameras, each pixel point has a gray value through encoding in the imaging, and the gray value closer to the lens is brighter. Corresponding pixel points can be found in the two photos through the gray value, so that the gray value is calculated, and the parallax D is obtained. Thereby obtaining a depth image through a binocular ranging algorithm.

The IMU sensor forms a cartesian coordinate system by a combined unit of 3 accelerometers and 3 gyroscopes as shown in fig. 4, having x, y and z axes, and is capable of measuring linear movements in each axis direction, as well as rotational movements around each axis. The method and the device are mainly used for judging whether the construction machine moves.

The data acquisition module finally outputs the image, the depth image and the motion signal obtained by the IMU sensor to the processor module every second. The image store is named XXXXXXX _ img _20AA _ BB _ CC _ DD __ EE _ FF _ H, where XXXXXXX is the camera number, AA is the year, BB is the month, CC is the date, DD is the hour, EE is the time in minutes, FF is the time in seconds, H is 0 or 1, 0 stands for still, 1 stands for motion. The depth image store is named depth _ XXXXXXXX _ img _20AA _ BB _ CC _ DD _ EE _ FF _ H in substantially the same way as the image.

The processing module 12 is connected to the data acquisition module 11 and is used for analyzing and processing each current image data to determine the worker contour pixel coordinates in each current image data and determine the distance between the worker and the mechanical equipment and the color combination of the safety helmet of the worker in each current image data according to the worker contour pixel coordinates, the depth image data and the motion data of the corresponding mechanical equipment;

a risk identification module 13 connected to the processing module 12, configured to identify a risk level corresponding to each current image data according to a distance between a worker and a mechanical device in each current image data and a color combination of a safety helmet of the worker, where the risk level includes a high risk level, a medium risk level, and a low risk level;

and the early warning module 14 is connected to the risk identification module 13 and is used for determining a corresponding human-computer collision early warning prompting mode for prompting according to the risk level.

In this embodiment, can initiatively judge whether peripheral operation personnel constitute the risk and carry out the early warning to portable machinery, help safety control personnel discernment safety risk, thereby help operation personnel to discover the high risk operating mode and take measures to reduce the risk.

As shown in fig. 5, in one embodiment, the processing module 12 preferably includes:

a denoising unit 51, configured to perform data denoising processing on each current image data to obtain processed image data;

in order to obtain the identity of the worker through the helmet color, data noise reduction, namely, color balance processing on the image, is firstly required. The color space of the picture is first converted from RGB to HSV because HSV performs better in natural light, where h (hue) is hue, s (saturation) is saturation, and v (value) is lightness. And selecting a cvSplit function in Opencv, wherein the image type is IPL-DEPTH _8UC, the value range of H is 0-180, the value range of S is 0-255, and the value range of V is 0-255. The value range of the RGB image R, G, B is 0-255, and preprocessing is needed before conversion into HSV, and the formula is as follows:

through the conversion of a COLOR _ RGB2HSV function in an OpenCV library, the conversion formula is as follows:

the algorithm selects contrast-limited adaptive histogram equalization (CLAHE) to perform color balance on the image, and the Histogram Equalization (HE) determines a gray distribution histogram of the image as a mapping curve named as cumulative distribution histogram (CDF) of the image. The CDF curve can carry out gray level transformation on the image, and the image contrast is improved. Adaptive Histogram Equalization (AHE) divides an image into a plurality of sub-blocks on the basis of histogram equalization, performs histogram equalization processing on the sub-blocks, and has a better effect on color balance of local details. The contrast-limited adaptive histogram equalization (CLAHE) limits the local contrast, and prevents the local contrast from being improved too much and prevents the image color from being distorted. The specific algorithm cuts the histogram obtained in each sub-block, so that the amplitude of the histogram is below a certain value, the cut part is uniformly distributed in the whole gray level interval, the total area of the histogram is unchanged, as shown in fig. 6, the influence of natural light and shadow on the color of the safety helmet in an HSV color space can be reduced by carrying out contrast-limited adaptive histogram equalization processing on the image transmitted by the data acquisition module, and the color identification of the safety helmet of a worker in the next step is facilitated.

A calculating unit 52, configured to calculate the processed image data by using a MASK-RCNN algorithm to obtain worker contour pixel coordinates of each worker in the image data;

the worker recognition adopts a Convolutional Neural Network (CNN) algorithm to realize the worker recognition, the algorithm does not use manually set rules to recognize the characteristics of a worker target in an image, but uses a large number of picture samples marked with workers to train, so that the algorithm obtains the capability of recognizing the workers through learning, and the accuracy is continuously improved. The indexes for judging the performance of the algorithm are a transmission Frame Per Second (FPS) and an average precision average value (mAP), the FPS represents the processing efficiency of the algorithm, the higher the numerical value is, the higher the efficiency is represented, the mAP represents the recognition precision of the algorithm, and the higher the numerical value is, the better the precision is represented under the same overlapping degree (IOU) for the same training set. In the stage, the identification of the workers is based on the data collected by the data acquisition module, and the analysis efficiency does not need to be considered, so that the MASK-RCNN algorithm with better precision at present is selected to obtain the contour pixels of the workers, and the specific effect is shown in fig. 7.

A distance determining unit 53 for determining a distance between each worker and the corresponding mechanical device according to the center point coordinates of the worker contour pixel coordinates and the depth image data;

a level determination unit 54 for identifying a target risk distance level corresponding to each worker according to a distance between each worker and the corresponding mechanical device and motion data of the mechanical device, and deleting a recognition result of a worker whose distance is greater than a risk distance threshold;

and (3) transferring all the contour pixel coordinates of the workers into an array, taking the coordinates of the center point of the contour image of the workers, transferring the coordinates into a depth image, and obtaining the distance between the workers and the vehicle body. And determining different risk distances according to different construction mobile machines, and deleting the worker identification results exceeding the risk distances.

The number determining unit 55 is configured to determine the number of target pixel points corresponding to the target risk distance level according to the target risk distance level of each undeleted worker and a preset corresponding relationship between the risk distance level and the number of pixel points;

a value determining unit 56, configured to obtain a pixel point from below a vertex of the worker contour pixel coordinate according to the number of the target pixel points, where the pixel point is used as a color anchor point, so as to determine an HSV value of the safety helmet of the worker;

as the headgear is typically worn on the head, the representation is in the picture as being represented at the top of the identified worker's contour. And finding the vertex of the contour through the pixel coordinates, taking 3-5 pixel points (taking 3 pixel points at a distance greater than 0.75 risk distance, taking 4 pixel points at a distance of 0.25-0.5 risk distance and taking 5 pixel points at a distance within 0.25 risk distance) under the vertex as color anchor points, and obtaining the HSV value of the worker safety helmet. And judging the color of the worker safety helmet according to the HSV value of the worker safety helmet. For each picture, the number of workers whose helmet color (red record is 1, blue record is 2, white record is 3, black record is 4, yellow record is 5, unidentified record is 6) corresponds to each helmet color is obtained by the above algorithm. The picture name and the number of safety helmets of each color are stored by a CSV file to form a worker combination per second. And counting the frequency of occurrence of each combination and inputting the frequency into a 1-dimensional array.

And the color judging unit 57 is configured to judge the color of the safety helmet of the worker according to the HSV value of the safety helmet, so as to obtain a color combination of the safety helmet corresponding to each current image data.

As shown in fig. 8, in one embodiment, preferably, the risk identification module 13 includes:

an acquiring unit 81 for acquiring a mass of image data and a color combination of a worker helmet corresponding to each image data;

the clustering unit 82 is used for clustering the frequency of the color combination of the artificial safety helmet in the massive image data through a clustering algorithm so as to divide the massive image data into a low-frequency combination, a medium-frequency combination and a high-frequency combination;

and clustering all the combinations according to the frequency, and clustering the frequency 1-dimensional arrays by using a k-means clustering algorithm and using the contour coefficient as a judging method. The k-means clustering algorithm is a clustering analysis algorithm for iterative solution, and comprises the steps of dividing data into k groups, taking k objects as initial clustering centers, calculating the distance between each object in the data and each clustering center, and dividing each data to the nearest clustering center. The cluster centers are recalculated for each data allocation until no data is reallocated again. In the invention, the value of k is 3, namely the final result is gathered into 3 types, which respectively represent high risk, medium risk and low risk. The contour coefficient is an evaluation mode of clustering effect, and after dividing data into k clusters by using a k-means clustering algorithm, the contour coefficient is calculated for each data in the clusters.

The specific algorithm is as follows, clustering a data set containing N data, and dividing data i into clusters A

Cluster C is different from cluster A

s (i) is the profile coefficient of data i

The distances are Euclidean distances of the combined data of the workers. For a data set containing N data, the contour coefficient is the average of the contour coefficients s (i) of each data. The value of the contour coefficient is between [ -1,1], when the contour coefficient is less than 0.25, no cluster structure exists in the data set, and clustering is invalid; when the contour coefficient is more than 0.25 and less than 0.5, the clustering result is weaker, and the cluster structure can be artificially formed; when the contour coefficient is more than 0.5 and less than 0.7, the clustering result has stronger persuasion; when the contour coefficient is more than 0.7 and less than 1.0, a stronger cluster structure is found, and the clustering result is most persuasive. The contour coefficient is larger than 0.5, representing successful clustering, the data set has convincing cluster structure, and is smaller than 0.5, and the system prompts: "data is wrong and more data is needed".

The searching unit 83 is configured to search a target combination corresponding to the helmet color combination of the worker in each current image data in the helmet color combinations corresponding to the massive image data to obtain a search result;

and a risk determining unit 84, configured to determine a risk level corresponding to the current image data according to the search result.

In the embodiment, the risk identification module extracts worker combination data from massive image data by using a deep learning algorithm, and finally obtains three clustering results of high frequency, medium frequency and low frequency through a clustering algorithm. The low frequency order represents a high risk, the medium frequency order represents a medium risk, and the high frequency order represents a low risk.

In one embodiment, preferably, the early warning module is configured to:

The early warning module can comprise three parts of a high-efficiency worker recognition algorithm, alarm judgment and buzzing and display alarm. The image and the depth image are acquired through a data acquisition module, and the image is subjected to color balance through contrast-limited adaptive histogram equalization (CLAHE). The workers need to be identified in real time, and when a Convolutional Neural Network (CNN) algorithm is applied, the identification efficiency of the algorithm, namely the number of frames transmitted per second (FPS), is preferably considered, and a Poly-Yolo algorithm is selected to obtain the outlines of the workers. And finding the vertex of the contour through the pixel coordinates, and taking 3-5 pixel points (taking 3 pixel points at a distance greater than 0.75 risk distance, taking 4 pixel points at a distance of 0.25-0.5 risk distance, and taking 5 pixel points at a distance within 0.25 risk distance) under the vertex as the color anchor points. And obtaining the color of the safety helmet in the HSV space to obtain the picture worker combination at the moment. And searching the worker combination in the worker combination with high risk, medium risk and low risk obtained in the risk identification module, if the worker combination can correspond to the worker combination, judging whether the worker combination has high risk, and if the worker combination has high risk, giving an alarm. If the high risk can not be responded to, the high risk is directly judged to alarm.

The alarm mode can be buzzing and voice, the attention of workers is aroused through buzzing, the workers are reminded to operate through voice, and the safety risk is reduced. The speech is: the risk of the area is high at present, and a safety person in charge is asked to improve the vigilance, pay attention to the blind area investigation and ask a non-operator to leave the area. Of course, an alarm may also be displayed on the display screen.

The processing module can adopt an embedded edge computing platform, the embedded edge computing platform module is used for processing and analyzing data in real time, and the embedded edge computing platform is formed by combining a special Ubuntu18.04 system of an embedded Arm framework based on a Jetson AGX Xavier artificial intelligence platform of Yingvida. The working environment of a construction site is complex, a large amount of dust can fly up all the time in the construction process, and in order to guarantee good construction environment and environmental protection requirements, a constructor can reduce the dust in the air by a water spraying mode. The invention relies on an embedded edge computing platform as a carrier for algorithm realization, and a platform case is required to meet the requirements of dustproof and waterproof computing cores. Meanwhile, the equipment is arranged on construction machinery equipment such as hoisting, transportation, excavation and the like, and the equipment can bring high-frequency vibration during operation, so the calculation core also has the requirement of shock resistance. And finally, in order to meet the condition that the heating causes the crash when the edge computing platform operates, the heat dissipation problem of the equipment is also considered. The size of the case is 21 cm in length, 18 cm in width and 32 cm in height, the case is composed of an aluminum heat dissipation shell, the thickness of the case is 3 mm, the height of the heat conduction dense teeth is 12 mm, a steel wire rope shock absorber and an internal circulation heat dissipation fan are arranged in the case, and the shock absorption and heat dissipation requirements of the embedded edge computing platform are met. The case comprises 4 wire outlets which are respectively a case power line, the display is connected with an HDMI high-definition video line, and an optical fiber video data transmission line is connected with an external USB expander. The whole equipment weight is 4 kg, and the top of the case shell is provided with a portable handle, so that the equipment is convenient to transport.

The power supply source in the construction site mainly has three modes, namely a site power supply, a mobile mechanical equipment power supply and an independent power supply.

The building site field power supply is mainly connected through the switch box and is connected with the distribution box through the line, and the advantages of the power supply stability and the highest safety factor are achieved. The power supply of the mobile mechanical equipment is mainly the power supply of the generator on the equipment, the advantages of being capable of carrying out data acquisition on the equipment, stable in power supply, and the disadvantages of being capable of expanding around the equipment due to the data acquisition, limiting the range of the data acquisition, and being capable of transforming the mobile mechanical equipment when being connected in addition, and having safety risks. The last mode is to supply power through an independent power supply, and has the advantages of strongest mobility and no limitation of sites, and has the disadvantages of limited capacity of the independent power supply and limited data acquisition time. But purely ranked from mobility: independent power supply > mobile machinery power supply > jobsite power supply.

The power supply of the invention mainly aims at an independent power supply and is compatible with a mobile mechanical equipment power supply and a construction site power supply. The power supply design is composed of an inverter, and the inverter is matched with a power supply of construction site mobile mechanical equipment (a mobile crane, a forklift and the like) to convert direct current (24V) into alternating current (220V). The power of the embedded edge computing platform is 30W, the capacity of the independent power supply battery is 23200mAh, the power ion battery with a battery core type power outputs 220V pure sine waves of 100W in an alternating current mode, and the requirement of data processing for 10 hours is met. The size of the independent power supply battery is 164 mm multiplied by 65 mm, and the requirement of the embedded edge computing platform on portability is met.

In the aspect of software, an embedded edge computing platform is based on an Arm version of an Ubuntu18.04 system, an artificial intelligence recognition algorithm is constructed by installing tenserflow-gpu 1.15, and an artificial intelligence recognition model is converted by installing tensorrT, so that the whole recognition algorithm is accelerated on a hardware layer. Based on Python3.6, the library and the pyrealsense2 library are constructed through compiling, and the data acquisition module is adapted. And an OpenCV library and a Scikit-Learn library are installed to realize the processing and data mining of the image. And installing a CSV library and storing the data.

Next, a man-machine collision warning method for a construction site according to an embodiment of the present invention will be described with reference to the drawings.

As shown in fig. 9, the man-machine collision early warning method for the construction site includes steps S101-S104:

step S101, respectively acquiring current image data, depth image data and motion data of mechanical equipment of a construction site through a plurality of binocular cameras and a plurality of sensors arranged on movable mechanical equipment of the construction site;

step S102, analyzing and processing each current image data to determine the contour pixel coordinates of workers in each current image data, and determining the distance between the workers and the mechanical equipment and the color combination of the safety helmets of the workers in each current image data according to the contour pixel coordinates of the workers, the depth image data and the motion data of the corresponding mechanical equipment;

step S103, identifying risk levels corresponding to the current image data according to the distance between the worker and the mechanical equipment in each current image data and the color combination of the safety helmet of the worker, wherein the risk levels comprise a high risk level, a medium risk level and a low risk level;

and step S104, determining a corresponding human-computer collision early warning prompting mode for prompting according to the risk level.

As shown in fig. 10, in one embodiment, the step S102 preferably includes steps S111-S117:

step S111, performing data noise reduction processing on each current image data to obtain processed image data;

step S112, calculating the processed image data by adopting a MASK-RCNN algorithm to obtain the worker contour pixel coordinates of each worker in the image data;

step S113, determining the distance between each worker and the corresponding mechanical equipment according to the center point coordinates of the contour pixel coordinates of the workers and the depth image data;

step S114, identifying a target risk distance grade corresponding to each worker according to the distance between each worker and the corresponding mechanical equipment and the motion data of the mechanical equipment, and deleting the identification result of the worker with the distance greater than the risk distance threshold value;

step S115, determining the number of target pixel points corresponding to the target risk distance level according to the target risk distance level of each undeleted worker and the corresponding relation between the preset risk distance level and the number of pixel points;

step S116, according to the number of the target pixel points, acquiring pixel points from the lower part of the vertex of the contour pixel coordinate of the worker as color anchor points to determine HSV values of the safety helmet of the worker;

and step S117, judging the color of the safety helmet of the worker according to the HSV value of the safety helmet so as to obtain the color combination of the safety helmet corresponding to each current image data.

As shown in fig. 11, in one embodiment, preferably, the step S103 includes:

step S121, acquiring massive image data and color combinations of worker safety helmets corresponding to the image data;

step S122, clustering the frequency of the color combination of the artificial safety helmet in the massive image data through a clustering algorithm so as to divide the massive image data into low-frequency combination, medium-frequency combination and high-frequency combination;

step S123, searching a target combination corresponding to the worker safety helmet color combination in each current image data in the worker safety helmet color combinations corresponding to the massive image data to obtain a search result;

and step S124, determining the risk level corresponding to the current image data according to the search result.

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

A non-transitory computer-readable storage medium having instructions therein, which when executed by a processor of a mobile terminal, enable the mobile terminal to perform a human-machine collision warning method for a construction site, the method comprising:

It is further understood that the use of "a plurality" in this disclosure means two or more, as other terms are analogous. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. The singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It will be further understood that the terms "first," "second," and the like are used to describe various information and that such information should not be limited by these terms. These terms are only used to distinguish one type of information from another and do not denote a particular order or importance. Indeed, the terms "first," "second," and the like are fully interchangeable. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure.

It is further to be understood that while operations are depicted in the drawings in a particular order, this is not to be understood as requiring that such operations be performed in the particular order shown or in serial order, or that all illustrated operations be performed, to achieve desirable results. In certain environments, multitasking and parallel processing may be advantageous.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A man-machine collision early warning system of job site, its characterized in that includes:

2. The human-computer collision warning system of a construction site according to claim 1, wherein the processing module comprises:

3. The human-computer collision warning system of a construction site according to claim 1, wherein the risk identification module comprises:

4. The human-computer collision warning system of a construction site according to claim 3, wherein the risk determination unit is further configured to:

when the target combination is found, determining a corresponding risk level according to the frequency combination to which the target combination belongs;

the low-frequency combination corresponds to a high risk level, the medium-frequency combination corresponds to a medium risk level, and the high-frequency combination corresponds to a low risk level;

5. The human-computer collision early warning system of a construction site according to claim 1, wherein the early warning module is configured to:

6. A man-machine collision early warning method for a construction site is characterized by comprising the following steps:

7. The human-computer collision warning method for the construction site according to claim 6, wherein the analyzing process is performed on each current image data to determine worker contour pixel coordinates in each current image data, and the distance between the worker and the mechanical equipment and the color combination of the safety helmet of the worker in each current image data are determined according to the worker contour pixel coordinates, the depth image data and the motion data of the corresponding mechanical equipment, and the method comprises the following steps:

8. The human-computer collision early warning method for the construction site according to claim 6, wherein identifying the risk level corresponding to each current image data according to the distance between the worker and the mechanical equipment and the color combination of the helmet of the worker in each current image data comprises:

9. The human-computer collision early warning method for the construction site according to claim 8, wherein the determining the risk level corresponding to the current image data according to the search result comprises:

10. The human-computer collision early warning method for the construction site according to claim 6, wherein the step of determining a corresponding human-computer collision early warning prompting mode for prompting according to the risk level comprises the following steps: