CN114283379A - Subway construction information identification method, device, equipment and storage medium - Google Patents

Abstract

The application relates to a subway construction information identification method, a subway construction information identification device, subway construction information identification equipment and a storage medium. The method comprises the following steps: acquiring a target image acquired from a subway protection area; inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features; inputting the target image into a construction equipment recognition model for processing, so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment, and recognizes the target construction equipment according to the image features of the construction equipment; and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment. By adopting the method, the identification efficiency can be improved.

Description

Subway construction information identification method, device, equipment and storage medium

Technical Field

The application relates to the technical field of industrial intelligence, in particular to a subway construction information identification method, device, equipment and storage medium.

Background

With the development of social economy and cities, traffic networks are gradually perfected in various big cities in order to solve traffic congestion. The subway is a public transport which is not influenced by weather and is punctual, and is popular with people in large cities. However, the continuous construction of subway lines inevitably increases the mileage of subway tunnels. Because the position of the subway tunnel is not clear, the safety of the tunnel may be damaged by the construction of the earth surface, and therefore the inspection above the tunnel becomes the key point for the safe operation of the subway.

At present, construction inspection is mainly carried out by adopting a manual inspection method, so that a large number of inspection personnel are required to be configured to carry out construction inspection on a protection area related to a subway, and the construction identification efficiency is low.

Disclosure of Invention

In view of the above, it is necessary to provide a method, an apparatus, a computer device, a computer readable storage medium and a computer program product capable of identifying subway construction information in view of the above technical problems.

In a first aspect, the application provides a subway construction information identification method. The method comprises the following steps: acquiring a target image acquired from a subway protection area; inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features; inputting the target image into a construction equipment recognition model for processing, so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment, and recognizes the target construction equipment according to the image features of the construction equipment; and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

In one embodiment, the determining, based on the target construction behavior and the target construction equipment, a dangerous construction identification result corresponding to the subway protection area includes: comparing the target construction equipment with dangerous construction equipment in a preset dangerous construction equipment set; if the dangerous construction equipment set comprises the target construction equipment; and if the target construction behavior is the construction behavior corresponding to the target construction equipment, determining that a dangerous construction source exists in the subway protection area.

In one embodiment, the determining, based on the target construction behavior and the target construction equipment, a dangerous construction identification result corresponding to the subway protection area includes: determining a behavior degree identification model corresponding to the target construction behavior; inputting the target image into the behavior degree identification model for degree identification, and identifying the construction behavior degree corresponding to the target construction behavior; determining a construction damage level based on the construction behavior degree, the target construction behavior and the target construction equipment; and when the construction damage level exceeds a damage level threshold, the dangerous construction identification result is dangerous construction.

In one embodiment, the method further comprises: and if the dangerous construction identification result indicates that dangerous construction exists, dangerous prompt information is sent to a terminal corresponding to a subway management user, and the dangerous prompt information is used for describing the construction behavior degree, the target construction behavior and the target construction equipment.

In one embodiment, the method further comprises: acquiring target voice corresponding to the subway protection area acquired by synchronously acquiring the target image; inputting the target voice into a voice recognition model, extracting tone features of the template voice by the voice recognition model, and recognizing based on the tone features to obtain voice output equipment corresponding to the target voice; and determining the voice output duration corresponding to the voice output equipment, and if the voice output duration is greater than a preset duration threshold, outputting the alarm prompt information corresponding to the voice output equipment.

In one embodiment, the method further comprises: arranging the target construction behaviors identified by the construction behavior identification model at each moment according to a time sequence to obtain a construction behavior sequence; acquiring a standard behavior sequence corresponding to standard construction; and comparing the construction behavior sequence with the standard behavior sequence, and if the comparison is consistent, performing behavior prompt information.

In a second aspect, the application further provides a subway construction information identification device. The device comprises: the system comprises a target image acquisition module, a data acquisition module and a data processing module, wherein the target image acquisition module is used for acquiring a target image acquired from a subway protection area; the construction behavior recognition module is used for inputting the target image into a construction behavior recognition model for processing so that the construction behavior recognition model performs feature extraction on the image to obtain construction behavior image features, and a target construction behavior is obtained according to the construction behavior image features; the construction equipment identification module is used for inputting the target image into a construction equipment identification model for processing so that the construction equipment performs characteristic extraction on the image to obtain construction equipment image characteristics, and the target construction equipment is obtained according to the construction equipment image characteristics; and the dangerous construction identification result determining module is used for determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

In one embodiment, the dangerous construction identification result determining module includes a comparison unit, which compares the target construction equipment with dangerous construction equipment in a preset dangerous construction equipment set; a dangerous construction source determining unit, configured to determine whether the dangerous construction equipment set includes the target construction equipment; and if the target construction behavior is the construction behavior corresponding to the target construction equipment, determining that a dangerous construction source exists in the subway protection area.

In one embodiment, the dangerous construction identification result determining module is configured to: determining a behavior degree identification model corresponding to the target construction behavior; inputting the target image into the behavior degree identification model for degree identification, and identifying the construction behavior degree corresponding to the target construction behavior; determining a construction damage level based on the construction behavior degree, the target construction behavior and the target construction equipment; and when the construction damage level exceeds a damage level threshold, the dangerous construction identification result is dangerous construction.

In one embodiment, the apparatus further comprises: a danger sending module: and sending danger prompt information to a terminal corresponding to a subway management user if the dangerous construction identification result indicates that dangerous construction exists, wherein the danger prompt information is used for describing the construction behavior degree, the target construction behavior and the target construction equipment.

In one embodiment, the device further comprises a voice recognition module, configured to acquire a target voice corresponding to the subway protection area acquired synchronously with the target image; a voice output device obtaining module, configured to input the target voice into a voice recognition model, where the voice recognition model extracts a tone characteristic of the template voice, and obtains a voice output device corresponding to the target voice based on the tone characteristic recognition; and the alarm prompt information output module is used for determining the voice output duration corresponding to the voice output equipment, and outputting the alarm prompt information corresponding to the voice output equipment if the voice output duration is greater than a preset duration threshold.

In one embodiment, the device further comprises a construction sequence arrangement module, which is used for arranging the target construction behaviors identified by the construction behavior identification model at each moment according to a time sequence to obtain a construction behavior sequence; the construction sequence acquisition module is used for acquiring a standard behavior sequence corresponding to dangerous construction; and the construction sequence comparison module is used for comparing the construction behavior sequence with the standard behavior sequence, and if the comparison is consistent, outputting behavior prompt information.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program: acquiring a target image acquired from a subway protection area; inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features; inputting the target image into a construction equipment recognition model for processing, so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment, and recognizes the target construction equipment according to the image features of the construction equipment; and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of: acquiring a target image acquired from a subway protection area; inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features; inputting the target image into a construction equipment recognition model for processing, so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment, and recognizes the target construction equipment according to the image features of the construction equipment; and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which when executed by a processor performs the steps of: acquiring a target image acquired from a subway protection area; inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features; inputting the target image into a construction equipment recognition model for processing, so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment, and recognizes the target construction equipment according to the image features of the construction equipment; and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

According to the subway construction information identification method, the subway construction information identification device, the computer equipment, the storage medium and the computer program product, the target image acquired by collecting the subway protection area is acquired; inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features; inputting the target image into a construction equipment recognition model for processing so that the construction equipment performs feature extraction on the image to obtain construction equipment image features, and recognizing according to the construction equipment image features to obtain target construction equipment; and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment. The image acquisition can be carried out on the subway protection area, the construction behaviors and the construction equipment corresponding to the image are respectively identified and obtained on the basis of the image and the corresponding model, and the danger source identification is carried out on the basis of the construction behaviors and the construction equipment obtained through identification, so that the danger construction source can be rapidly and accurately determined, and the identification efficiency is improved.

Drawings

Fig. 1 is an application environment diagram of a subway construction information identification method in one embodiment;

fig. 2 is a schematic flow chart of a subway construction information identification method in one embodiment;

fig. 3 is a schematic flow chart illustrating steps of determining a dangerous construction recognition result corresponding to a subway protection area by a target construction behavior and target construction equipment in one embodiment;

fig. 4 is a schematic flowchart illustrating a step of determining a dangerous construction recognition result corresponding to a subway protection area based on a target construction behavior and target construction equipment in one embodiment;

fig. 5 is a schematic flow chart of a subway construction information identification method in one embodiment;

fig. 6 is a schematic flow chart of a subway construction information identification method in one embodiment;

fig. 7 is a block diagram showing the structure of a subway construction information recognition apparatus according to an embodiment;

FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The embodiment of the application provides a subway construction information identification method which can be applied to an application environment shown in figure 1. Wherein the photographing apparatus 102 communicates with the server 104 through a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104, or may be located on the cloud or other network server. The terminal device 106 may be used to receive information sent by the server 104. The server 104 acquires an image from the shooting device 102 through a communication network, the image is used as an identification model in the server 104 to extract features, the extracted features are used as an identification model in the server 104 to perform feature identification, information of construction behaviors and construction devices is obtained, judgment is performed based on the construction behaviors and the construction devices, if the construction behaviors or the construction devices are found to be dangerous, the construction behaviors, the construction devices, geographic positions and the like are sent to the terminal device 106 through the communication network, relevant information is displayed for a user, and workers can quickly and accurately judge dangers brought by construction danger sources above the subway tunnel. The shooting device 102 may be, but is not limited to, a drone and a camera, and the server 104 may be implemented by an independent server or a server cluster composed of a plurality of servers. The terminal device 106 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart car devices, and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. It can be understood that the subway construction information identification method provided by the present application may also be executed on other devices, for example, may be executed in a terminal device.

In one embodiment, as shown in fig. 2, a subway construction information identification method is provided, which is described by taking the method as an example applied to the server in fig. 1, and comprises the following steps:

step S202, acquiring a target image acquired from a subway protection area;

the subway protection area is an area related to safe operation of a subway, construction in the subway protection area may affect operation of the subway, for example, the subway protection area may be an area of the earth surface above the subway protection area, for example, the area may be above a subway tunnel. The target image is an image obtained by shooting a subway protection area. The target image can be shot by a camera installed in a subway protection area, and can also be shot by an unmanned aerial vehicle, and the no-fly area adopts the camera to shoot and collect characteristics such as constructors, construction materials and field appearances.

Specifically, the subway protection area can be shot by utilizing the shooting equipment to obtain a target image, the shooting equipment can send the shot image to the server in real time, and the server acquires the target image so as to analyze whether dangerous construction behaviors exist in the subway protection area or not and analyze whether dangerous construction equipment exists in the subway protection area or not through the target image.

In one embodiment, the target image may be an image in a video, for example, a video may be captured by a drone, and a partial image, for example, an image including at least one of a device or a portrait, may be extracted therefrom as the target image. And each target image can be identified to obtain corresponding dangerous construction behaviors and dangerous construction equipment.

And step S204, inputting the target image into the construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features.

The construction behavior recognition model can extract the characteristics of the image corresponding to the construction behavior and can recognize the construction behavior according to the extracted construction behavior characteristics. The construction behavior recognition model may be an artificial intelligence model obtained by training a large number of pictures of the type. The construction activities refer to activities of constructing a subway protection area, for example, the construction activities may include at least one of drilling, excavating, or piling. Feature extraction is the extraction of some characteristic representation of the construction behavior within the image, which may be a neural network model.

Specifically, the construction behavior recognition model may include a feature extraction layer and a behavior classification layer, the server obtains a target image from the terminal, a trained construction behavior recognition model with artificial intelligence is built in the server, feature extraction is performed on an input image through the feature extraction layer, obtained image features are further given to the behavior classification layer, the behavior classification layer obtains probabilities of the image corresponding to various construction behaviors based on the extracted features, and a construction behavior with the maximum probability and larger than a probability threshold is selected as the target construction behavior.

For example, the construction behavior recognition model may be used to recognize various behaviors such as drilling, excavation, and piling, and the target image is input to the construction behavior recognition model, and the construction behavior recognition model outputs probabilities of the behaviors of the image as drilling, excavation, and piling, respectively, assuming that the probabilities corresponding to drilling, excavation, and piling are 0.8, 0.15, and 0.05, and assuming that the preset threshold is 0.6, since 0.8 is greater than 0.6, the recognized target construction behavior is drilling.

In one embodiment, the construction behavior image may be obtained in advance, the construction behavior corresponding to the image is labeled manually, and the labeled construction behavior is used as the behavior label of the construction behavior image. The construction behavior image can be input into a construction behavior recognition model to be trained, the construction behavior recognition model to be trained recognizes the construction behavior image to obtain the probability corresponding to the behavior tag, the model loss value is determined based on the probability corresponding to the behavior tag, wherein the larger the probability is, the smaller the loss value is, the parameter of the model can be adjusted towards the direction that the loss value becomes smaller, and the trained construction behavior recognition model is obtained.

And step S206, inputting the target image into the construction equipment recognition model for processing so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment and recognizes the target construction equipment according to the image features of the construction equipment.

The construction equipment identification model can extract the characteristics of the image corresponding to the construction equipment and can identify the construction equipment according to the extracted construction equipment characteristics. The construction equipment identification model is an artificial intelligence model obtained by training a large number of pictures of the type. The construction equipment refers to equipment for constructing a subway protection area, and for example, the construction equipment may include at least one action of a drilling machine, an excavator or a pile driver. Feature extraction is the extraction of some characteristic representation of the construction equipment inside the image, and the model is a neural network model.

Specifically, the construction equipment identification model can comprise a feature extraction layer and an equipment classification layer, the server obtains a target image from the terminal, a trained construction equipment identification model with artificial intelligence is arranged in the server, feature extraction is carried out on an input picture through the feature extraction layer, the obtained image features are further given to the equipment classification layer, the equipment classification layer obtains the probability that the image corresponds to various construction equipment based on the extracted features, and construction equipment with the maximum probability and larger than a probability threshold value is selected from the probability and used as the target construction equipment.

For example, the construction equipment recognition model may be used to recognize various types of equipment such as a drilling machine, an excavator, a pile driver, and the like, and the target image is input to the construction equipment recognition model, and the equipment from which the construction equipment recognition model outputs the image has probabilities of the drilling machine, the excavator, and the pile driver being respectively, and assuming that the probabilities of the drilling machine, the excavator, and the pile driver being respectively corresponding are 0.7, 0.2, and 0.1, and assuming that the preset threshold value is 0.6, since 0.7 is greater than 0.6, the target construction equipment recognized is the drilling machine.

In one embodiment, the construction equipment image may be obtained in advance, the construction equipment corresponding to the image may be manually labeled, and the labeled construction equipment may be used as the equipment label of the construction equipment image. The construction equipment image can be input into the construction equipment identification model to be trained, the construction equipment identification model to be trained identifies the construction equipment image to obtain the probability corresponding to the equipment label, the model loss value is determined based on the probability corresponding to the equipment label, the larger the probability is, the smaller the loss value is, the parameters of the model can be adjusted towards the direction of the loss value becoming smaller, and the trained construction equipment identification model is obtained.

And S208, determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

The dangerous construction identification result refers to an identification result of whether a target construction behavior and target construction equipment cause danger to a subway protection area, and the dangerous identification result can be dangerous or non-dangerous. For example: the excavator digs above, and dangerous construction exists.

The target construction behavior refers to the behaviors above the subway protection area, which can be not limited to drilling, digging and piling, and the construction behavior can be defined according to different behaviors of protection requirements. The target construction equipment refers to equipment above a subway protection area, which can be not limited to a drilling rig, an excavator and a pile driver, and the construction equipment can have different equipment definitions according to protection requirements. The recognition result represents the construction behavior or the influence of the construction equipment on the subway tunnel.

Specifically, the target construction behavior is compared with the dangerous construction behavior, and the target construction equipment is compared with the dangerous construction equipment, which can be the comparison of the independent construction behavior or the independent construction equipment, or the comparison of the independent construction behavior and the independent construction equipment, or the comparison of both, if the target construction behavior and the dangerous construction equipment are consistent, the danger exists. If not, then no danger exists. If the danger exists, information such as construction behaviors, construction equipment, specific places, damage degree and the like is given, and if no danger source is found, the identification is continued.

In one embodiment, the construction behavior recognition model finds that drilling behaviors exist, compares the drilling behaviors with preset dangerous construction behaviors, immediately sends a signal to an alarm center under the condition that the comparison results are the same, and the alarm center is linked with a terminal on hand of an inspection worker and sends information such as construction behaviors, construction equipment, specific places, damage degrees and the like to inform the inspection worker to process in time.

In the subway construction information identification method, a target image acquired by collecting a subway protection area is acquired; inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features; inputting the target image into a construction equipment recognition model for processing so that the construction equipment performs feature extraction on the image to obtain construction equipment image features, and recognizing according to the construction equipment image features to obtain target construction equipment; and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment. The image acquisition can be carried out on the subway protection area, the construction behaviors and the construction equipment corresponding to the image are respectively identified and obtained on the basis of the image and the corresponding model, and the danger source identification is carried out on the basis of the construction behaviors and the construction equipment obtained through identification, so that the danger construction source can be rapidly and accurately determined, and the identification efficiency is improved.

In one embodiment, as shown in fig. 3, determining a dangerous construction recognition result corresponding to the subway protection area based on the target construction behavior and the target construction equipment includes:

step S302, comparing the target construction equipment with dangerous construction equipment in a preset dangerous construction equipment set.

The construction equipment in the dangerous construction equipment set is construction equipment which is preset and can cause danger to subway operation, and specifically, the dangerous construction equipment can be set according to needs, for example, the dangerous construction equipment refers to at least one of a drilling rig, an excavator or a pile driver. The non-hazardous construction equipment may be, for example, a road finisher.

Specifically, a dangerous construction equipment set is preset in the server, when a relevant machine enters a subway protection area and is constructed, the type of the equipment is obtained based on a target image, the type of the equipment is compared with the dangerous construction equipment set, and whether a characteristic identification result is the same as the dangerous construction equipment set is judged.

In one embodiment, the subway control center presets a set of, but not limited to, drilling rigs, excavators, and drivers, for example, a drilling rig is identified above a subway tunnel, and outputs are compared to the set.

Step S304, if the dangerous construction equipment set comprises target construction equipment and the target construction behavior is the construction behavior corresponding to the target construction equipment, determining that a dangerous construction source exists in the subway protection area.

Wherein, the target construction behavior is the construction behavior corresponding to the target construction equipment, and the target construction behavior is as follows: the target construction action is an action performed by the target construction equipment, that is, in the image, the target construction equipment is performing the related construction action in the subway protection area, for example, an excavator performs excavation and a pile driver performs pile driving. The subway protection area is the area of the earth's surface above the subway protection area. The dangerous construction source is a behavior having a dangerous construction equipment and a dangerous construction behavior.

Specifically, the construction equipment identification model identifies the construction equipment, and the construction behavior identified by the construction behavior identification model has relevance with the identified equipment, so that whether the construction equipment is dangerous construction equipment or not and whether the construction behavior is dangerous construction behavior or not are respectively judged, if the construction equipment and the construction behavior are judged to be dangerous construction source, the dangerous construction source is considered, and a signal is output to the alarm center. For example, a pile driver is placed above a subway tunnel and a piling action is performed, which indicates that the target construction action is a corresponding action of the target construction equipment; if the pile driver reaches the upper part of the subway tunnel and the pile driving action is not carried out, the joint condition of the pile driver and the subway tunnel is not met.

In one embodiment, the subway protection area is constructed above, the construction equipment with the pictures collected by the camera is identified as a pile driver, the identified construction behavior is pile driving, the pile driving behavior and the construction behavior are judged to be positive at the same time, the server considers that the construction behavior is a dangerous construction source, and a signal is output to the alarm center.

In the embodiment, the construction can be further determined to be dangerous construction through the combined comparison of the target construction equipment and the target construction behaviors, and the identification accuracy is improved.

In one embodiment, as shown in fig. 4, determining a dangerous construction recognition result corresponding to the subway protection area based on the target construction behavior and the target construction equipment includes:

step S402, determining a behavior degree identification model corresponding to the target construction behavior.

The target construction behavior is a construction behavior made by target construction equipment, the behavior degree identification model can perform feature extraction on the image to obtain construction behavior features capable of embodying the construction degree, and meanwhile, the degree of the construction behavior can be identified according to the extracted construction behavior features, for example: the excavator corresponds to high, medium and low excavation degrees. The behavior degree identification model is an artificial intelligence model obtained by training a large number of pictures corresponding to the construction behaviors and corresponding behavior degree labels.

Specifically, the behavior degree identification model is provided with a behavior degree classification layer, the server acquires a target image, a trained behavior degree identification model with artificial intelligence is arranged in the server, the acquired image characteristics are further given to the behavior degree classification layer, the equipment classification layer obtains corresponding probabilities for various behavior degrees based on the image, and the construction behavior degree with the maximum probability and larger than a probability threshold value is selected as the target behavior degree.

For example, the different construction activities correspond to different behavior degree models, for example, the behavior degree recognition model may include a plurality of models such as a model for recognizing the degree of drilling by a drilling machine, a model for recognizing the degree of digging by an excavator, and a model for recognizing the degree of piling by a pile driver, the target image is input to the behavior degree recognition model corresponding to the target construction activity, the behavior degree recognition model outputs probabilities of the behavior of the image at various degrees, and the recognized target construction activity degree is high assuming that the probabilities of the high, medium, and low degrees of drilling by the drilling machine are 0.9, 0.05, and 0.05, respectively.

Step S404, inputting the target image into the behavior degree identification model for degree identification, and identifying the construction behavior degree corresponding to the target construction behavior.

The construction behavior degree refers to a degree reached by the behavior of the equipment, the degree can be divided into a high level, a medium level and a low level, and the corresponding destructiveness is also from large to small. For example, a high degree of construction activity generally corresponds to high destructiveness.

Specifically, the target image is input into the behavior degree identification model, and the behavior degree identification model discriminates the degree of the construction behavior according to the image to obtain probabilities corresponding to the high degree, the medium degree and the low degree, for example, if the behavior degree model recognizes that the construction degree of the excavator is 0.1, 0.1 and 0.8 respectively, the construction degree is discriminated to be low.

In one embodiment, complex construction is carried out above a subway protection area, a machine for entering construction comprises a pile driver, and the identification model identifies the construction degree of the shot images. If the equipment is constructed, the corresponding construction degree is given, and if the equipment is not constructed, the monitoring is continued.

Step S406, determining a construction damage level based on the construction behavior degree, the target construction behavior and the target construction equipment.

The damage level is divided according to the destructiveness of the construction equipment to the subway protection area, the destructiveness is divided into three levels, namely high, medium and low, and the damage degree of the construction equipment to the protection area and the damage degree of the construction equipment to the subway operation are reflected.

The construction behavior degree, the construction behavior and the construction equipment corresponding to each construction damage level are preset, that is, each damage level is determined according to the behavior degree, the construction behavior and the construction equipment in a combined mode, for example, the level corresponding to the condition that the behavior degree is low, the construction behavior is drilling and the construction equipment is drilling equipment is set.

Specifically, the destructiveness of the construction area is judged according to the three recognition results of the construction behavior degree, the target construction behavior and the target construction equipment in the construction area.

In one embodiment, the captured images are respectively input into different models to obtain the construction behavior degree, the target construction behavior and the target construction equipment, then the system analyzes and judges the destructiveness, and the single destructiveness and the joint destructiveness are output. The mono-destructiveness is judged according to only one of the construction behavior and the construction equipment.

In one embodiment, the construction damage level may be determined by combining the construction behavior degree, the target construction behavior, and the target construction device respectively corresponding to the plurality of target images. For example, a behavior sequence corresponding to each damage level is preset, when the behaviors obtained by target image recognition are sorted according to time and are the same as the behavior sequence, the degree corresponding to the behaviors obtained by target image recognition and whether the degree of the behaviors of the damage level corresponding to the behavior sequence and the construction equipment are the same are judged, and if the degrees are the same, the damage level is determined.

And step S408, when the construction damage level exceeds the damage level threshold, the dangerous construction identification result is dangerous construction.

The damage level threshold refers to the lowest value of the destructiveness of the construction, and when the damage level threshold is higher than the threshold, the destructiveness of the construction is indicated, and when the damage level threshold is lower than the threshold, the possible destruction of the construction is indicated, and the monitoring is to be strengthened. Dangerous construction refers to construction which damages a subway protection area, and when the construction is higher than a damage level threshold value, the construction belongs to dangerous construction.

Specifically, when the damage level exceeds a set threshold, the construction is determined as dangerous construction; when the damage level does not exceed the threshold value, but the construction action exists, the construction is intensively monitored, and once the damage level is higher than the threshold value, the construction is determined to be dangerous. For example, the region corresponding to the construction behavior may be set as a highlight detection region, and the image capturing frequency for the highlight detection region may be increased. Or a fixed area detection instruction is sent to the unmanned aerial vehicle, the unmanned aerial vehicle is instructed to fly to the key detection area to shoot, and the server continues to execute construction behavior detection construction equipment detection based on the shot image, so that the detection efficiency can be improved.

In one embodiment, where there is a road leveling construction above the metro protection area, the image input into the model does not identify the target construction equipment, and therefore the construction is not considered a hazardous construction.

In this embodiment, through adopting the construction degree recognition model to discern the construction degree, can the high, well, low construction degree of this construction action of rapid judgement, for providing destruction degree and repairing provide important reference.

In one embodiment, if the dangerous construction identification result indicates that dangerous construction exists, dangerous prompt information is sent to a terminal corresponding to a subway management user, and the dangerous prompt information is used for describing the construction behavior degree, the target construction behavior and the target construction equipment.

The subway management user refers to an inspection worker in a subway protection area, the terminal refers to equipment carried by the inspection worker, and information such as construction behavior degree, target construction behavior and target construction equipment can be displayed.

Specifically, when the dangerous construction source is identified by the identification model, information such as the degree of work behavior, the target construction equipment and the like is sent to the patrol personnel of the subway through the server, a specific position is given, and the degree of harm of the dangerous construction source is automatically evaluated. For example: and (4) carrying out drilling construction by a drilling machine, obtaining a dangerous construction source by shooting input of a terminal and then identifying, immediately sending an alarm signal to inspection personnel, and marking real-time position source information.

In one embodiment, as shown in fig. 5, the method further comprises:

step S502, acquiring target voice corresponding to the subway protection area acquired by synchronously acquiring the target image.

The target voice refers to voice collected while collecting videos above a subway protection area, and the voice is synchronous with the videos and can assist in recognizing drilling, excavating and piling.

Specifically, when the video is collected, the voice collecting device is opened, the device can have a voice filtering effect, and can filter sounds except the sounds emitted by the device, so that voices synchronized with the target image are obtained.

In one embodiment, the voice acquisition equipment is installed to unmanned aerial vehicle and camera, opens voice acquisition equipment when shooing, can obtain with the synchronous pronunciation of image.

Step S504, the target voice is input into the voice recognition model, the voice recognition model extracts the tone characteristic of the template voice, and the voice output equipment corresponding to the target voice is obtained based on the tone characteristic recognition.

The voice recognition model can extract the characteristics of the voice corresponding to the construction equipment, and can recognize the construction equipment according to the extracted voice characteristics. The tone features are features representing the perceptual characteristics of sound, and the speech recognition model is an artificial intelligence model obtained by training a large number of voices of the type. The construction voice refers to a sound emitted from equipment that performs construction on the subway protection area, for example, the construction equipment may include a sound emitted from at least one of a drilling machine, an excavator, or a pile driver. Feature extraction is the extraction of some characteristic acoustic properties of the construction equipment within the speech.

Specifically, the voice recognition model may include a feature extraction layer and an equipment layer, the server obtains a target voice from the terminal, a trained voice recognition model with artificial intelligence is built in the server, feature extraction is performed on the input voice through the feature extraction layer, the obtained voice features are further provided to the equipment classification layer, the equipment classification layer obtains probabilities that the voice corresponds to various construction equipment based on the extracted features, and the construction equipment with the highest probability and greater than a probability threshold is selected from the probabilities to serve as the target construction equipment.

For example, the speech recognition model may be used to recognize various sounds such as a drilling rig, a shovel, and a pile driver, and target speech is input to the speech recognition model, and the probability that the device from which the speech is output by the speech recognition model is the drilling rig, the shovel, and the pile driver, respectively.

Step S506, determining the voice output duration corresponding to the voice output device, and if the voice output duration is greater than a preset duration threshold, outputting the alarm prompt information corresponding to the voice output device.

The time length threshold value refers to a time length threshold value set during voice output, an alarm is triggered when the time length threshold value is exceeded, and the fact that the time length is not exceeded indicates normal.

Specifically, the voice output devices output by the voice recognition model at various times can be acquired, so that the voice duration of each voice output device is determined, the output duration is compared with a preset duration threshold, if the obtained voice duration exceeds the threshold, the construction equipment is constructed for a long time and is possibly damaged, and if the obtained voice duration is shorter than the threshold, the construction equipment does not reach the damaged degree.

In one embodiment, after a section of sound collected by a voice collecting device of an unmanned aerial vehicle or a camera is identified by a voice identification model, it is found that an excavator excavates, if the voice duration exceeds a set threshold, the excavator may have excavated a deeper position and may have damage to a subway tunnel, and if the voice duration is shorter than the threshold, it indicates that no damage has occurred, but a shutdown is required in time.

In this embodiment, synchronous collection pronunciation when gathering the image can reach and assist the supplementary construction action of judging in the aspect of following pronunciation, prevents that image acquisition from missing dangerous construction action.

In one embodiment, as shown in fig. 6, the method further comprises:

step S602, arranging the target construction behaviors identified by the construction behavior identification model at each moment according to a time sequence to obtain a construction behavior sequence.

The construction behaviors are arranged according to the time sequence, namely the construction behaviors identified at different moments are arranged according to the time sequence from first to last, and the construction behavior sequence refers to a sequence of the construction behaviors arranged from first to last according to the time.

Specifically, the construction behaviors obtained by inputting the images shot at different times into the construction behavior recognition model are arranged according to the time sequence, and the sequence obtained by arrangement is a set of each construction behavior and is not the construction behavior not participating in arrangement.

In one embodiment, the excavator excavates and needs to drill holes, then excavates, and finally sends the excavated soil to a certain position for stacking. The steps are arranged according to the time sequence, and the obtained sequence is punching, digging and stacking.

And step S604, acquiring a standard behavior sequence corresponding to dangerous construction.

The standard behavior sequence refers to a sequence obtained from first to last according to standard construction behaviors.

Specifically, a sequence obtained according to a standard construction sequence from first to last in time is input into the server to obtain a standard behavior sequence. The standard action sequence is a preset standard action sequence and represents dangerous construction action sequence.

In one embodiment, the standard order in which excavations are performed by an excavator is punch, dig, and stack, which is entered into the server.

And step S606, comparing the construction behavior sequence with the standard behavior sequence, and outputting behavior prompt information if the comparison is consistent.

The construction action sequence refers to a construction sequence in which actual construction actions are carried out from first to last in time sequence.

Specifically, the construction behaviors identified by the construction behavior identification model are arranged from first to last according to the time sequence, and the construction behavior sequence of the construction is obtained. And comparing the sequence with a standard construction behavior sequence, and if the comparison is consistent, indicating that the construction is dangerous construction.

In one embodiment, the excavator excavates, the construction behavior recognition model recognizes the punching, digging and stacking behaviors, the punching, digging and stacking behaviors are obtained by sequencing according to time, then the punching, digging and stacking behaviors are compared with a standard construction behavior sequence, the behaviors are found to be consistent, and behavior prompt information is output.

In this embodiment, by comparing the construction behavior sequence with the standard behavior sequence, it is possible to identify the construction that is not standard from the construction sequence, and output a prompt message to the corresponding terminal to prompt that the behavior sequence is dangerous.

It should be understood that, although the steps in the flowcharts related to the embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the application also provides a subway construction information identification device for realizing the subway construction information identification method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the method, so that specific limitations in one or more embodiments of the subway construction information identification device provided below can be referred to the limitations on the subway construction information identification method in the above, and details are not repeated herein.

In one embodiment, as shown in fig. 7, there is provided a subway construction information recognition apparatus including: image acquisition module, construction action identification module, construction equipment identification module and dangerous construction identification result determine module, wherein:

an image obtaining module 702, configured to obtain a target image acquired from a subway protection area;

the construction behavior recognition module 704 is used for inputting the target image into the construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes and obtains the target construction behavior according to the construction behavior image features;

the construction equipment identification module 706 is used for inputting the target image into the construction equipment identification model for processing, so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment, and identifies the target construction equipment according to the image features of the construction equipment;

and a dangerous construction identification result determining module 708, configured to determine a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

In one embodiment, the dangerous construction recognition result determining module includes:

the comparison unit is used for comparing the target construction equipment with dangerous construction equipment in a preset dangerous construction equipment set;

the dangerous construction source determining unit is used for determining whether the dangerous construction equipment set comprises target construction equipment; and determining that the dangerous construction source exists in the subway protection area if the target construction behavior is the construction behavior corresponding to the target construction equipment.

In one embodiment, the dangerous construction identification result determining module is configured to: determining a behavior degree identification model corresponding to the target construction behavior; inputting the target image into a behavior degree identification model for degree identification, and identifying the construction behavior degree corresponding to the target construction behavior; determining a construction damage level based on the construction behavior degree, the target construction behavior and the target construction equipment; and when the construction damage level exceeds the damage level threshold, the dangerous construction identification result is that dangerous construction exists.

In one embodiment, the apparatus further comprises a hazard sending module: and sending danger prompt information to a terminal corresponding to the subway management user if the dangerous construction identification result indicates that dangerous construction exists, wherein the danger prompt information is used for describing the construction behavior degree, the target construction behavior and the target construction equipment.

In one embodiment, the device further comprises a voice recognition module, which is used for acquiring target voice corresponding to the subway protection area acquired by synchronously acquiring the target image; the voice output equipment obtaining module is used for inputting the target voice into the voice recognition model, extracting the tone characteristic of the template voice by the voice recognition model, and obtaining the voice output equipment corresponding to the target voice based on the tone characteristic recognition; and the alarm prompt information output module is used for determining the voice output duration corresponding to the voice output equipment, and outputting the alarm prompt information corresponding to the voice output equipment if the voice output duration is greater than a preset duration threshold.

In one embodiment, the device further comprises a construction sequence arrangement module, which is used for arranging the target construction behaviors identified by the construction behavior identification model at each moment according to a time sequence to obtain a construction behavior sequence; the construction sequence acquisition module is used for acquiring a standard behavior sequence corresponding to dangerous construction; and the construction sequence comparison module is used for comparing the construction behavior sequence with the standard behavior sequence, and if the comparison is consistent, outputting behavior prompt information.

All modules in the subway construction information identification device can be completely or partially realized through software, hardware and a combination of the software and the hardware. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data to be processed. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a subway construction information recognition method.

Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments. It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims (10)

1. A subway construction information identification method, characterized in that the method comprises:

acquiring a target image acquired from a subway protection area;

inputting the target image into a construction behavior recognition model for processing, so that the construction behavior recognition model performs feature extraction on the image, extracts the construction behavior image features, and recognizes the target construction behavior according to the construction behavior image features;

inputting the target image into a construction equipment recognition model for processing, so that the construction equipment performs feature extraction on the image, extracts the image features of the construction equipment, and recognizes the target construction equipment according to the image features of the construction equipment;

and determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

2. The method of claim 1, wherein determining a dangerous construction identification result corresponding to the subway protection zone based on the target construction behavior and the target construction equipment comprises:

comparing the target construction equipment with dangerous construction equipment in a preset dangerous construction equipment set;

if the dangerous construction equipment set comprises the target construction equipment; and if the target construction behavior is the construction behavior corresponding to the target construction equipment, determining that a dangerous construction source exists in the subway protection area.

3. The method of claim 1, wherein the determining a dangerous construction identification result corresponding to the subway protection zone based on the target construction behavior and the target construction equipment comprises:

determining a behavior degree identification model corresponding to the target construction behavior;

inputting a target image into the behavior degree identification model for degree identification, and identifying the construction behavior degree corresponding to the target construction behavior;

determining a construction damage level based on the construction behavior degree, the target construction behavior and the target construction equipment;

and when the construction damage level exceeds a damage level threshold, the dangerous construction identification result is dangerous construction.

4. The method of claim 3, further comprising:

and if the dangerous construction identification result indicates that dangerous construction exists, dangerous prompt information is sent to a terminal corresponding to a subway management user, and the dangerous prompt information is used for describing the construction behavior degree, the target construction behavior and the target construction equipment.

5. The method of claim 1, further comprising:

acquiring target voice corresponding to the subway protection area acquired by synchronously acquiring the target image;

inputting the target voice into a voice recognition model, extracting tone features of the template voice by the voice recognition model, and recognizing based on the tone features to obtain voice output equipment corresponding to the target voice;

and determining the voice output duration corresponding to the voice output equipment, and if the voice output duration is greater than a preset duration threshold, outputting the alarm prompt information corresponding to the voice output equipment.

6. The method of claim 1, further comprising:

arranging the target construction behaviors identified by the construction behavior identification model at each moment according to a time sequence to obtain a construction behavior sequence;

acquiring a standard behavior sequence corresponding to dangerous construction;

and comparing the construction behavior sequence with the standard behavior sequence, and outputting behavior prompt information if the comparison is consistent.

7. A subway construction information recognition apparatus, said apparatus comprising:

the image acquisition module is used for acquiring a target image acquired from a subway protection area;

the construction behavior recognition module is used for inputting the target image into a construction behavior recognition model for processing so that the construction behavior recognition model performs feature extraction on the image to obtain construction behavior image features, and a target construction behavior is obtained according to the construction behavior image features;

the construction equipment identification module is used for inputting the target image into a construction equipment identification model for processing so that the construction equipment performs characteristic extraction on the image to obtain construction equipment image characteristics, and the target construction equipment is obtained according to the construction equipment image characteristics;

and the dangerous construction identification result determining module is used for determining a dangerous construction identification result corresponding to the subway protection area based on the target construction behavior and the target construction equipment.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6.

9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 6 when executed by a processor.

Images