CN116029542A - Construction worker safety risk identification method based on computer vision and rational atlas - Google Patents

Abstract

The invention relates to a construction worker safety risk identification method based on computer vision and event map, on one hand, the construction worker safety risk identification method realizes the establishment of accident event map by carrying out the steps of event extraction, causal clause extraction, event probability transfer and the like on a construction report; on the other hand, the image information is translated into text information by utilizing a computer vision technology, so that an image semantic event consistent with the expression form of the accident situation map is formed, and the image semantic event is mapped into the accident situation map which is built in advance, so that the safety risk can be automatically identified. The method can grasp and evaluate the integrity of the construction scene, can automatically identify the safety risk, is easy to practice, takes pictures as input, does not need to install a sensor, reduces the cost and improves the efficiency.

Description

Construction worker safety risk identification method based on computer vision and rational atlas

Technical Field

The invention relates to the technical field of construction site management, in particular to a construction worker safety risk identification method based on computer vision and a rational map.

Background

Job site workers' occupational safety concerns are of great concern and the construction industry is always in the position of high-risk industries. All parties are devoted to improving the situation and guaranteeing the occupational safety of staff on the construction site. Technical intervention may offer the most important potential for eliminating safety accident hazards or reducing safety risks from the source, VR, BIM, sensor technology, etc. play a powerful role in construction safety management. The computer vision technology is characterized by non-invasion, proper cost and high recognition efficiency, and rapidly goes into the sight of researchers. In the field of construction safety management, although the computer vision technology has remarkable progress in aspects of target detection, resource tracking and the like, few researches related to overall mastering of scenes are often limited to researching certain computer vision specific underlying content, spatial relations and logical relations among various components are ignored, mining, extracting, reading and reasoning of semantic information of scenes are lacking, and advanced visual information is difficult to form. The method is particularly important for evaluation of construction scenes and worker occupational safety risks and prevention of occupational safety accidents. The challenge is overcome, and the degree of automation of safety supervision and management of the construction site by using a computer vision technology can be effectively improved. The dynamic scene understanding of the vision sensing equipment leading in the research category of the health and safety of the workers in the construction site is established, the information related to the health and safety of the workers is not only mined, the spatial relationship of elements in the monitoring scene is explained, the movement intention is also grasped, the evolution trend of the scene is deduced, the data information is analyzed to imply the safety risk, and the autonomous supervision is realized.

Safety accident intervention methods based on computer vision technology are also lacking in matching evaluation systems or rules with complex construction scene events. The existing evaluation method of the computer vision technology on the safety supervision of the construction site mainly evaluates the safety risk according to preset rules (norms, experiences, laws and the like), and the preset rules often have extremely strong subjectivity (such as according to management experiences) and are relatively single, so that the safety risk cannot be fully covered. Lack of integrity grasping and evaluation of construction scenes limits the accuracy and reliability of monitoring health and safety automation of construction site workers by computer vision technology, and is not beneficial to the practice of the technology.

Disclosure of Invention

In order to solve the problems, the invention provides a construction worker safety risk identification method based on computer vision and a rational map, which can grasp and evaluate the integrity of a construction scene, can automatically identify the safety risk and is easy to practice.

The invention is realized by the following scheme: a construction worker safety risk identification method based on computer vision and a rational map comprises the following steps:

collecting a construction accident report, and carrying out event triplet extraction, causal clause extraction and event relation extraction on the accident report to form a plurality of causal relation event tuple pairs;

performing coreference resolution on the events in all the causal event tuple pairs, and reserving causal relationships to form a plurality of generalized causal event tuple pairs;

carrying out causal event transition probability calculation on the generalized causal event tuple pairs;

constructing an accident event map according to all the generalized causal event tuple pairs and the causal event transition probabilities;

collecting a construction site picture, and organizing object semantics and attribute semantics reflected by the construction site picture into a plurality of image semantic events in the forms of < object, attribute and object > by utilizing an image semantic event model;

inputting the image semantic event into the accident event map, and finding out a corresponding mapping event from the accident event map;

and extracting a maximum risk event chain based on the mapping event and a principle of selecting a path with the maximum transition probability, and identifying the security risk of the image semantic event according to the maximum risk event chain.

The construction worker safety risk identification method based on the computer vision and the event map is further improved in that after a construction site picture is collected, a visual object in the construction site picture is identified by utilizing an image identification tool, the object semantics correspond to the visual object, and the attribute semantics comprise logic attributes for describing the relation between the visual objects and spatial position attributes for describing the spatial position relation between the visual objects.

The construction worker safety risk identification method based on computer vision and a rational map is further improved in that the spatial position relationship comprises a topological relationship, an azimuth relationship and a distance relationship.

A further improvement of the construction worker safety risk identification method based on computer vision and a rational map is that the topological relation comprises inclusion, disjoint, partial intersection and tangency.

The invention relates to a construction worker safety risk identification method based on computer vision and a rational map, which is further improved in that:

when the visual object in the construction site picture is identified by utilizing an image identification tool, carrying out frame selection on the identified visual object and forming a detection frame;

the spatial position attribute in the image semantic event reflected by the construction site picture is determined by the following steps:

firstly, determining a topological relation between two objects in the image semantic event, wherein the topological relation is a space connection relation between visual object detection frames corresponding to the two objects;

if the topological relation is any one of inclusion, tangency and partial intersection, determining an azimuth relation between the two objects, wherein the azimuth relation is an azimuth relation between the two detection frames;

and if the topological relation is disjoint, determining a distance relation between the two objects, wherein the distance relation is the distance between the center pixels of the two detection frames.

The invention relates to a construction worker safety risk identification method based on computer vision and a rational map, which is further improved in that the steps of carrying out coreference resolution on the events in all causal event tuple pairs comprise the following steps:

converting the triples of all the events into a structured word vector form from a word text form, correspondingly calculating the word vector similarity of three tuples in the event and three tuples in other events to obtain the word text similarity of the three tuples, and averaging the word text similarity of the three tuples to obtain the tuple similarity of the event and other events;

and comparing each obtained tuple similarity with a preset similarity threshold, and digesting the corresponding event when the tuple similarity is not smaller than the similarity threshold.

The invention further improves the construction worker safety risk identification method based on computer vision and a rational map, wherein the word vector similarity is obtained by adopting four vector similarity calculation indexes to calculate and average.

The invention relates to a construction worker safety risk identification method based on computer vision and a rational map, which is further improved in that four vector similarity calculation indexes are Jaccard similarity coefficient, cosine similarity, euclidean distance and Manhattan distance respectively.

The invention relates to a construction worker safety risk identification method based on computer vision and a rational map, which is further improved in that the step of finding a corresponding mapping event from the accident rational map comprises the following steps:

converting the triples of all events in the image semantic event and the accident event map from a word text form to a structured word vector form;

respectively carrying out similarity calculation on all events in the image semantic event and the accident event map by adopting two vector similarity calculation indexes and one text similarity calculation index, and then averaging three calculation results to obtain the final similarity of the two corresponding events;

and finding an event in the accident event map corresponding to the final similarity with the largest value in all the final similarities as the mapping event.

The construction worker safety risk identification method based on computer vision and a rational map is further improved in that two vector similarity calculation indexes are Jaccard similarity coefficients and cosine similarity respectively, and the text similarity calculation index is editing distance similarity.

The invention includes, but is not limited to, the following benefits:

1. the method has the advantages that the image information is translated into the text information by utilizing the computer vision technology, the image semantic event consistent with the expression form of the accident situation map is formed, and is mapped into the accident situation map, so that the safety risk can be automatically identified, the safety of the construction scene is evaluated in a non-contact mode, the method is easy to practice, and the picture is used as input, a sensor is not required to be installed, the cost is reduced, and the efficiency is improved.

2. The construction report is subjected to event extraction, causal clause extraction, event probability transfer and other steps to establish an accident event map, so that the construction scene can be grasped and evaluated integrally, and the construction method has a wider application range compared with the manual construction safety ontology or knowledge map establishment, can be popularized and applied to the establishment of a large-scale construction safety event map, and improves the application value.

3. The probability assignment is carried out on the reasoning result so as to reflect the confidence level of the evaluation result, and the method has the function of improving the safety management and early warning efficiency for the safety management practice of the construction site.

Drawings

Fig. 1 shows a flow chart of the method of the invention.

Figure 2 shows a causal clause extraction flow chart in the method of the invention.

FIG. 3 shows a schematic view of a partial representation of an accident situation map in an embodiment of the method of the present invention.

Figure 4 shows a schematic representation of a text sample of an incident report in an embodiment of the method of the present invention.

FIG. 5 is a schematic diagram of a causal clause extraction process record in an embodiment of the method of the present invention.

FIG. 6 is a schematic diagram of a causal clause extraction result display in an embodiment of the method of the present invention.

FIG. 7 shows a schematic diagram of the spatial relationship of objects in an embodiment of the method of the present invention.

Detailed Description

The construction worker safety risk identification method based on the computer vision and the rational map is further described below with reference to the accompanying drawings by using specific embodiments.

Referring to fig. 1, a construction worker safety risk identification method based on computer vision and a rational map includes the steps of:

and S11, collecting a public construction accident report, and performing event triplet extraction, causal clause extraction and event relation extraction on the accident report to form a plurality of causal relation event tuple pairs.

Specifically, referring to fig. 2, the construction accident report is in a document form, all documents are preprocessed after being collected, document segmentation and sentence segmentation are performed first, then whether a connective word exists in a clause of a document table is judged, if yes, a causal relation base is established, and a plurality of causal relation event tuple pairs are finally formed. The step of establishing the causal relation library mainly comprises the following steps:

1. event triples extraction aims to present unstructured event information contained in the raw text in a structured form. Event Triples (Triples), i.e., e= (S, P, O), expressed as event < subject, predicate, object > are extracted using dependency syntax analysis (Semantic Dependency Parsing, SDP). The method mainly comprises the following steps:

(i) A dictionary of syntactically dependent sub-nodes is maintained for each word in the sentence, and the relationship and the location of the corresponding sub-word are stored. And extracting and storing parent nodes and extracting the dependency relationship.

(ii) For the dependency structure of the parent-child array of the word, which is not generated, the part of speech of the word, the part of speech of the parent node and the relation between the part of speech and the parent node are recorded.

(iii) And (3) circularly searching a main and auxiliary relationship with a dynamic guest relationship, a post-fixed-language dynamic guest relationship and a mediate guest, and extracting.

(iv) For extracting words in the main guest, searching for words with related dependency structures, and eliminating unnecessary words.

(v) Identifying the found subject or object, and extracting event tuples.

2. And extracting causal clauses, namely extracting causal relations of the event by adopting a causal knowledge base-based method, and extracting boundary matching causal clauses according to a causal pattern knowledge base (see table 1 for details) and the causal relations.

Table 1 causal pattern knowledge base

/>

The causal relationship extraction boundary matching rule is as follows:

1)if t ₁ ∈cue1 and t _i ∈cue2，then s _i ∈p ₁ ，effect＝{t ₂ ，…，t _i-1 }，cause＝{t _i+1 ，…，t _n }

2)if t ₁ ∈cue3 and t _i ∈cue4，then7 s _i ∈p ₂ ，cause＝{t ₂ ，…，t _i-1 }，effect＝{t _i+1 ，…，t _n }

3)if t ₁ ∈cue5 and

then s _i ∈p ₃ ，cause＝{t ₁ ，…，t _i-1 }，effect＝{t _i+1 ，…，t _n }

4)if t ₁ ∈cue6 and

then s _i ∈p ₄ ，cause＝{t ₁ ，…，t _i-1 }，effect＝{t _i+1 ，…，t _n }

5)if t ₁ ∈cue7 and t _i ∈punc.，then s _i ∈p ₅ ，cause＝{t ₂ ，…，t _i }，effect＝{t _i+1 ，…，t _n }

6)if t ₁ ∈cue8 and

then s _i ∈p ₆ ，cause＝{t ₁ ，…，t _i-1 }，effect＝{ _ti+1 ，…，t _n }

7)if t ₁ ∈cue9 and t _i ∈punc.，then s _i ∈p ₇ ，cause＝{t ₂ ，…，t _i }，effect＝{t _i+1 ，…，t _n }

8)if t ₁ ∈cue10 and

then s _i ∈p8，effect＝{t ₂ ，…，t _i-1 }，cause＝{t _i+1 ，…，t _n }

9)if t ₁ ∈cue10 and t _i ∈punc.，then s _i ∈p5，effect＝{t ₂ ，…，t _i }，cause＝{t _i+1 ，…，t _n }

wherein: si represents the i-th sentence of the text, the content of which is { t1, … tn }, ti represents the i-th word (token) in the sentence, cause represents the reason clause, effect represents the result clause, ue1 represents the type of connective corresponding to the syntax pattern P1 of the prefrontal syntax pattern table 1, and so on.

And step S12, performing coreference resolution on the events in all the causal event tuple pairs, and reserving causal relations to form a plurality of generalized causal event tuple pairs.

The main flow comprises the following steps:

1. word Embedding (Word Embedding) of events. The method is characterized in that Word vectors are embedded (Word 2Vec, words (Word) of a text class are converted into structured vectors (vectors) in a mathematical space so that the structured vectors (vectors) can participate in calculation and are used for measuring similarity among words, a CBOW model of Hierarchical Softmax is adopted, words which are 2c above and below specific words of a corpus text are defined and input, wherein Word Vector dimension M, step length and Word Vector are w, and algorithm flow of the method is summarized as follows (1) Huffman Tree (Huffman Tree, also called optimal binary Tree) is built based on training samples { context (w), parameter initialization is carried out, parameter theta and Word Vector w are initialized randomly, gradient iteration is entered, and a random gradient rising method is selected to complete the iteration process.

2. Event tuple similarity is computed. Word vector similarity of the computation event triples, i.e., similarity of the respective argument of e= (S, P, O), is selected. Let E _simi (E _i ，E _j ) Representing event E _i And event E _j Similarity of (E) _{simi(Ei，Ej)} The calculation formula of (2) is as follows:

four common vector similarity calculation indexes are selected: the first is Jaccard similarity coefficient, the formula is as follows:

the second is cosine similarity, the formula is as follows:

the third is Euclidean distance, and the formula is as follows:

the fourth chapter is manhattan distance (Manhattan Distance), the formula is as follows:

and calculating the similarity of the word vectors for each tuple through the four vector similarity calculation indexes respectively, and taking an average value to obtain the similarity of the final word vectors of the single tuple, wherein the formula is as follows:

from the similarity of the final word vector of the single tuple, the text similarity of the single tuple can be obtained

The text similarity of the three tuples is obtained in this way and substituted into formula (1) to average, finally the event E is obtained _i And event E _j Similarity E of (2) _{simi(Ei，Ej)} 。

3. By adopting a similarity threshold method, the method in this embodiment considers the similarity between event tuples and covers text content as much as possible, reveals the relationship between events, and finally sets the threshold to 0.7 for the setting of the threshold. That is, the causal event tuple pair satisfying the judgment condition of the formula (7) is resolved, and finally the generalized causal event tuple pair is obtained.

And step S13, performing causal event transition probability calculation on the generalized causal event tuple pair. The transition probability calculation formula is as follows:

where COUNT (a, B) represents the frequency of events a occurring simultaneously with events B throughout the corpus material, and COUNT (a) represents the frequency of events a occurring throughout the corpus material. However, in this case, the event a and the event B are both co-reference resolved events, that is, the frequency of occurrence before co-reference resolution needs to be considered when calculating the frequency.

And S14, constructing an accident event map according to all the generalized causal event tuple pairs and the corresponding causal event transition probabilities. Referring specifically to fig. 3, fig. 3 shows a schematic view of a partial representation of the incident map. Each node represents a cause clause or a result clause, the arrow represents a causal relationship between the two clauses, and the data on the arrow represents the transition probability of the relationship.

S2, collecting a construction site picture, and organizing object semantics and attribute semantics reflected by the construction site picture into a plurality of image semantics events in the forms of object, attribute and object > by utilizing an image semantics event model.

Specifically, the object in the image semantic event, i.e., the object semantic reflected by the job site picture, corresponds to the "participant" in the event triplet, i.e., the subject and object of the event. In the ontology, it is mapped as a visual object under the "scene" of the construction site, i.e. object layer information. After collecting the job site picture, the visual object in the job site picture is identified by an image recognition tool (the present embodiment selects to employ a YOLO v3 model which should be trained and detected by the model of the relevant picture before use), the visual object is annotated, and the identified visual object is framed and a detection frame is formed. The attribute in the image semantic event, namely attribute semantics reflected by the construction site picture, mainly comprises a logic attribute and a spatial position attribute. The logical attribute mainly refers to a relationship between objects to link the objects or the objects to other relationships, and mainly includes data attributes such as Distance (Distance) and object attributes such as worker and construction tool. The specific definition is shown in Table 2.

Table 2 attribute semantic definition

The spatial position attribute is mainly used for describing the spatial position relation among objects, and the spatial position relation mainly comprises a topological relation, an azimuth relation and a distance relation. The topological relation is used for describing the mutual spatial connection and adjacency relation between object entities, and the spatial relation description does not give specific positions, namely the topological relation is expressed in a topological logic way, and comprises the containing, disjoint, partially intersected and tangent relations. And the azimuth relation adopts a four-direction (east, south, west and north) and eight-direction (including northeast, southeast, northwest and southwest) partition description model. The specific definitions and compositions are shown in Table 3. For the acquisition of the image semantics of the three spatial position relations, the acquisition is mainly determined according to the relation between the visual object detection frames, and specifically the following steps are followed: firstly, determining a topological relation between two objects in the image semantic event, wherein the topological relation is a space connection relation between visual object detection frames corresponding to the two objects; if the topological relation is any one of inclusion, tangency and partial intersection, determining the azimuth relation between the two objects, wherein the azimuth relation is the azimuth relation between the two detection frames; if the topological relation is disjoint, determining a distance relation between the two objects, wherein the distance relation is the distance between the center pixels of the two detection frames.

TABLE 3 spatial semantic definition

And S3, inputting the image semantic event into the accident event map, and finding out a corresponding mapping event from the accident event map.

Specifically, converting the triples of the image semantic event and all events in the accident event map from a word text form to a structured word vector form; respectively carrying out similarity calculation on the semantic event of the image and all events in the accident event map by adopting two vector similarity calculation indexes and one text similarity calculation index, and then averaging three calculation results to obtain the final similarity of the two corresponding events; and finding an event in the accident event map corresponding to the final similarity with the largest value in all the final similarities as the mapping event. The two vector similarity calculation indexes are Jaccard similarity coefficient and cosine similarity, respectively, and the calculation method is the same as the method in the event coreference resolution (i.e. refer to formula (1) and formula (2)), and will not be described herein. The text similarity calculation index is edit distance similarity. The calculation flow of the edit distance similarity is as follows:

1. calculate two character strings S _A ，S _B Length len (S) _A )，len(S _B) ；

2. Establishing an array space which is one (+1) longer than the length of the character string, and giving an initial value;

3. calculating the editing distance D of two character strings by using a dynamic programming algorithm _edit See formula (9), wherein d [ i, j ]]Representing the edit distance of the A character string from the 0 th character to the i character and the B character string from the 0 th character to the j character;

4. similarity is calculated using the minimum edit distance, see equation (10):

Simi _De ＝1-D _edit /max{len(S _A ),len(S _B )} (10)

the similarity of events is that of each eventThe synthesis of the argument similarity, i.e. the similarity of the respective argument of e= (S, P, O), let at this point E _simi (E _i ，E _j ) Representing event pairs E _i And E is _j Event facies of (1) at which point E is specified _i For image semantic events, E _j For the event in the accident situation map, E _{simi(Ei，Ej)} The calculation formula of (2) is a formula (12), and the calculation formula is derived from a formula (11):

mapping the image semantic event to an accident event map, traversing the whole accident event map, calculating event similarity, and selecting the event with the highest similarity as a mapping event according to the principle of maximum similarity.

And S4, extracting a maximum risk event chain based on the mapping event and a principle of selecting a path with the maximum transition probability, and identifying the security risk of the image semantic event according to the maximum risk event chain. Preferably, in the step S3, the similarity may be initialized to 0.5, the similarity calculated each time is compared with the initialized similarity, if the similarity is greater than or equal to the initialized similarity, the corresponding events are stored as mapping events in the candidate set, the risk event chain is extracted according to all the mapping events, the maximum risk event chain is found according to the principle of maximum transition probability, and the corresponding security risk is further determined and is used as the security risk of the image semantic time. Further, the security risk probability may be expressed in terms of a comprehensive transition probability in order to evaluate the security risk.

The method will be specifically described with reference to a specific example.

Firstly, collecting a public construction accident report, and preparing a foundation for building a safety accident situation map; and (3) writing a web crawler program, collecting safety accident investigation reports issued by residents of the people's republic of China, urban and rural construction parts, emergency management parts and related departments and units of all levels, and finally collecting 1025 accident reports.

And secondly, extracting the event. The processing of the document, besides the steps of sentence segmentation and the like shown in fig. 2, also needs to perform operations such as format conversion, effective content interception (deleting irrelevant content) and the like on the text, and finally stores the text into a txt document format according to a mode of 'title-body', as shown in fig. 4.

Sentence segmentation processing and word segmentation preprocessing are performed on all text materials according to the steps of fig. 2. The sentence segmentation and word segmentation are completed by adopting the methods of natural language processing tools of 'jieba', 'chon','re' and regular expressions, and the segmented data material is subjected to explicit 'causal' judgment on sentence by sentence, namely scanning judgment on whether the data material has causal connection words shown in table 1. Fig. 5 shows the processing of the text material causal relationship extraction flow.

And thirdly, extracting a causal clause. And obtaining a causal relation extraction result by using the extraction method of the explicit causal event based on the causal knowledge base. FIG. 6 is a drawing of a result of extraction in which a reason clause is stored in a cause (e 1) column and a result clause is stored in an effect (e 2) column.

Fourth, the event is co-referenced. And converting the event co-fingers into event similarity calculation, and then measuring the similarity of the events. Event coreference resolution is accomplished using the methods described above.

And fifthly, calculating transition probability. And (5) scanning and counting all reasons and result clauses, and calculating event transition probability according to a formula (8). The established rational map is shown in fig. 3.

And sixthly, collecting the construction site pictures. And acquiring an image video by placing a camera with a fixed focal length on a construction site, editing and acquiring an image key frame in the later period or performing data acquisition by shooting in the field, wherein the acquired data set comprises 10000 images.

And seventh, marking the collected construction pictures by construction workers and construction machinery. The image sample data was annotated using LabelImg.

Eighth, in the TensorFlow architecture, object detection of the Yolo v3 model is performed. The image quantity ratio on the training and testing set is 8:2, training is completed under a TensorFlow architecture, the testing set pictures are evaluated, and the detection accuracy of the excavator, workers, safety helmets and trucks is 0.9467, 0.9452, 0.9620 and 0.9356 respectively.

And ninth, generating image semantic event description. The object semantics and attribute semantics reflected by the image are organized into an image semantic event description model in the form of < object, attribute, object >, wherein the attribute semantics mainly comprise logic attributes and spatial location attributes. Selecting a picture as an example, performing target recognition detection through an image pattern recognition tool, and obtaining coordinates shown in table 4 according to a recognition detection result. The spatial positional relationship can be derived from the pixel coordinates.

Table 4 rectangular frame coordinates

From fig. 7, it can be found that "worker 3" does not have the spatial semantics of "including a helmet", and thus is semantically expressible in terms of object properties: "worker" and "helmet" are "unworn". Through a preset ontology relationship, the image events can be represented as follows in sequence: { worker, contain, helmet }; { worker, not worn, helmet }; { worker, approach, excavator }; { worker, wear, helmet }; { worker, wear, helmet }; { worker away, helmet }; { worker, distance, worker }, etc.

And tenth, event similarity calculation, namely taking an image semantic event as input of a rational map reasoning system, and calculating the similarity of the image semantic event and the event in the accident rational map by adopting a formula (12).

And eleventh, mapping the image semantic event into an accident event map. And traversing the whole accident theory map, calculating the similarity of the events, and selecting the event tuple with the highest similarity to finish mapping according to the principle of maximum similarity. The mapped events include: { worker, approach, excavator }, { worker, not worn, helmet }.

Twelfth, risk reasoning based on transition probabilities. And taking the mapping event as a root event, always selecting a path with the maximum transition probability, and extracting a maximum risk event chain. Respectively extracting event chains according to the accident event map transfer probability: [ workman, near, excavator ] -excavator, work, area ] -boom, rotation, ] -bucket, bump, ] -wound, ]; [ workman, not wearing, safety helmet ] -workman, construction, operation ] -slag bear, drop, ] -head, bleeding, ]. The safety risk for workers is therefore that the absence of safety helmets is prone to bleeding from head injuries.

The present invention has been described in detail with reference to the embodiments of the drawings, and those skilled in the art can make various modifications to the invention based on the above description. Accordingly, certain details of the illustrated embodiments are not to be taken as limiting the invention, which is defined by the appended claims.

Claims (10)

1. The construction worker safety risk identification method based on computer vision and a rational map is characterized by comprising the following steps:

collecting a construction accident report, and carrying out event triplet extraction, causal clause extraction and event relation extraction on the accident report to form a plurality of causal relation event tuple pairs;

performing coreference resolution on the events in all the causal event tuple pairs, and reserving causal relationships to form a plurality of generalized causal event tuple pairs;

carrying out causal event transition probability calculation on the generalized causal event tuple pairs;

constructing an accident event map according to all the generalized causal event tuple pairs and the causal event transition probabilities;

collecting a construction site picture, and organizing object semantics and attribute semantics reflected by the construction site picture into a plurality of image semantic events in the forms of < object, attribute and object > by utilizing an image semantic event model;

inputting the image semantic event into the accident event map, and finding out a corresponding mapping event from the accident event map;

and extracting a maximum risk event chain based on the mapping event and a principle of selecting a path with the maximum transition probability, and identifying the security risk of the image semantic event according to the maximum risk event chain.

2. The construction worker safety risk recognition method based on computer vision and a rational map according to claim 1, wherein after collecting a construction site picture, a visual object in the construction site picture is recognized by an image recognition tool, the object semantics correspond to the visual object, and the attribute semantics include a logical attribute for describing a relationship between the visual objects and a spatial location attribute for describing a spatial location relationship between the visual objects.

3. The construction worker safety risk identification method based on computer vision and a rational map according to claim 2, wherein the spatial positional relationship includes a topological relationship, an azimuth relationship, and a distance relationship.

4. A method of construction worker safety risk identification based on computer vision and a rational map as in claim 3 wherein said topological relationships include containment, disjoint, partially intersecting and tangential.

5. The construction worker safety risk identification method based on computer vision and a rational map according to claim 4, wherein:

when the visual object in the construction site picture is identified by utilizing an image identification tool, carrying out frame selection on the identified visual object and forming a detection frame;

the spatial position attribute in the image semantic event reflected by the construction site picture is determined by the following steps:

firstly, determining a topological relation between two objects in the image semantic event, wherein the topological relation is a space connection relation between visual object detection frames corresponding to the two objects;

if the topological relation is any one of inclusion, tangency and partial intersection, determining an azimuth relation between the two objects, wherein the azimuth relation is an azimuth relation between the two detection frames;

and if the topological relation is disjoint, determining a distance relation between the two objects, wherein the distance relation is the distance between the center pixels of the two detection frames.

6. The method for identifying safety risk of construction workers based on computer vision and a rational map according to claim 1, wherein the step of co-reference resolving the events in all the causal event tuple pairs comprises:

converting the triples of all the events into a structured word vector form from a word text form, correspondingly calculating the word vector similarity of three tuples in the event and three tuples in other events to obtain the word text similarity of the three tuples, and averaging the word text similarity of the three tuples to obtain the tuple similarity of the event and other events;

and comparing each obtained tuple similarity with a preset similarity threshold, and digesting the corresponding event when the tuple similarity is not smaller than the similarity threshold.

7. The method for identifying safety risk of construction workers based on computer vision and a rational map according to claim 6, wherein the word vector similarity is obtained by calculating and averaging four vector similarity calculation indexes respectively.

8. The method for identifying safety risk of construction workers based on computer vision and a rational map according to claim 7, wherein the four vector similarity calculation indexes are Jaccard similarity coefficient, cosine similarity, euclidean distance and manhattan distance, respectively.

9. The construction worker safety risk identification method based on computer vision and a rational map according to claim 1, wherein the step of finding a corresponding mapping event from the accident rational map comprises:

converting the triples of all events in the image semantic event and the accident event map from a word text form to a structured word vector form;

respectively carrying out similarity calculation on all events in the image semantic event and the accident event map by adopting two vector similarity calculation indexes and one text similarity calculation index, and then averaging three calculation results to obtain the final similarity of the two corresponding events;

and finding an event in the accident event map corresponding to the final similarity with the largest value in all the final similarities as the mapping event.

10. The method for identifying safety risk of construction workers based on computer vision and a rational map according to claim 9, wherein the two vector similarity calculation indexes are Jaccard similarity coefficient and cosine similarity, respectively, and the text similarity calculation index is edit distance similarity.

Images